Skip to content

[BUG] scoped package bandwidth performance is poor and will just get worse in the future #1648

Closed
@mikkorantalainen

Description

@mikkorantalainen

Current Behavior:

The existing API for scoped packages causes HUGE overhead for loading packages. The overhead is expected to raise in the future, if nothing is done.

Example: have dependency such as "@atlaskit/editor-core": "^112.44.8". To fulfill this, npm needs to fetch the editor-core package. This requires downloading 9789k of data just to figure out the URL for the package because this is a "scoped" package and as such, the URL http://registry.npmjs.org/@atlaskit/editor-core needs to be fully fetched just to acquire the URL for the actual package. If http://registry.npmjs.org/@atlaskit/editor-core/latest (or other more specific matches) were supported, there wouldn't be need to transfer so much data. Note that because the main URL must return full history for the package, the problem is expected to get worse in long run.

Expected Behavior:

$ curl http://registry.npmjs.org/@atlaskit/editor-core/latest
should work allowing clients to transfer lots of unneeded data.

Steps To Reproduce:

$ curl -I http://registry.npmjs.org/@atlaskit/editor-core/latest
HTTP/1.1 401 Unauthorized
...
npm-notice: ERROR: you cannot fetch versions for scoped packages
...
$ curl -I http://registry.npmjs.org/@atlaskit/editor-core/^112.44.8
HTTP/1.1 404 Not Found
...

curl http://registry.npmjs.org/@atlaskit/editor-core | wc -c
10024038

(or 9789k of data just to get the URL for the actual package which is 1911k)
curl https://registry.npmjs.org/@atlaskit/editor-core/-/editor-core-112.44.8.tgz | wc -c

Note that many packages have similar overhead: fetching the URL for specific version of the scoped package often takes 10x the data transfer of the full size for the actual package!

Environment:

Ubuntu 16.04.6 LTS
$ apt policy nodejs
...
Installed: 10.22.0-1nodesource1
...
500 https://deb.nodesource.com/node_10.x xenial/main amd64 Packages

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugthing that needs fixing

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions