Skip to content

fix(nextjs): Stop SentryWebpackPlugin from uploading unnecessary files #3845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 5, 2021

Conversation

lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Jul 27, 2021

Though sourcemap uploading using SentryWebpackPlugin is generally quite fast, even when there are a lot of files, there are situations in which it can slow things down a lot.

There are a number of reasons for that, but one of them is that, in the current state, @sentry/nextjs's use of the plugin is pretty indiscriminate - it just uploads anything and everything in the .next folder (which is where nextjs puts all of its built files), during both client and server builds. The lack of specificity leads us to upload files we don't need, and the fact that we don't distinguish between client and server builds means that whichever one happens second not only uploads its own unnecessary files, it also uploads all of the files from the other build (which have already been uploaded), both necessary and unnecessary. More details here.

This PR makes it so that we're much more specific in terms of what we upload, in order to fix both the specificity and the duplication problems.

Notes:

  • There's discussion in the linked issue about which non-user sources/maps to upload (things like code from webpack, nextjs itself, etc). In the end, I chose to restrict it to user code (almost - see below), for two reasons. First, non-user code is unlikely to change much (or often) between releases, so if third-party code were included, we'd be uploading lots and lots of copies of the same files. Second, though it's occasionally helpful to be able to see such code in the stacktrace, the vast majority of the time the problem lies in user code. For both reasons, then, including third-party files didn't feel worth it , either in terms of time spent uploading them or space spent storing them.

    (I say it's "almost" restricted to user code because, among other files, the server build generates bundles which are labeled only by numbers (1226.js and such), some of which are user code and some of which are third-party code, and - in this PR at least - I didn't include a way to filter out the latter while making sure to still include the former. This is potentially still a worthwhile thing to do, but the fact that it would mean actually examining the contents of files (rather than just their paths) makes it a more complicated change, which will have to wait for a future PR.)

  • In that issue, the topic of HMR (hot module reloading) files also came up. For the moment I also chose to skip those (even though they contain user code), as a) the plugin isn't meant for use in a dev environment, where every change triggers a new build, potentially leading to hundreds of sets of release files being uploaded, and b) we'd face a similar issue of needing to examine more than just the path in order to know what to upload (to avoid uploading all previous iterations of the code at each rebuild).

  • Another small issue I fixed, while I was in the relevant code, was to prevent the webpack plugin from injecting the release into bundles which don't need it (like third-party code, or any bundle with no Sentry.init() call). Since this lines up exactly with the files into which we need to inject sentry.server.config.js or sentry.client.config.js, I pulled it out into a function, shouldAddSentryToEntryPoint().

Fixes #3769
Fixes vercel/next.js#26588

@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-stop-uploading-too-much-stuff branch from 0e84895 to 3357356 Compare July 27, 2021 15:18
@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-stop-uploading-too-much-stuff branch from 3357356 to 75a7c20 Compare August 5, 2021 04:48
@github-actions
Copy link
Contributor

github-actions bot commented Aug 5, 2021

size-limit report

Path Size
@sentry/browser - CDN Bundle (gzipped) 21.56 KB (0%)
@sentry/browser - Webpack 22.56 KB (0%)
@sentry/react - Webpack 22.59 KB (0%)
@sentry/browser + @sentry/tracing - CDN Bundle (gzipped) 28.99 KB (+0.01% 🔺)

@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-stop-uploading-too-much-stuff branch from aa7953e to e60b1bb Compare August 5, 2021 05:51
@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-stop-uploading-too-much-stuff branch from e60b1bb to b919e55 Compare August 5, 2021 19:05
@lobsterkatie lobsterkatie merged commit 8ef39cc into master Aug 5, 2021
@lobsterkatie lobsterkatie deleted the kmclb-nextjs-stop-uploading-too-much-stuff branch August 5, 2021 19:34
lobsterkatie added a commit that referenced this pull request Aug 18, 2021
…3878)

In order to cut down on the number of unnecessary files which get uploaded  by the Sentry webpack plugin during nextjs builds, we're quite specific about which filepaths to scan. The change which introduced that specificity[1] missed the fact that under webpack 4, `.next/server/chunks` is not generated. This leads the Sentry webpack plugin to throw an error when it tries to upload it, even though nothing is actually amiss.

This PR fixes that, by checking the webpack version before deciding which files to upload with the plugin.

[1] #3845
lobsterkatie added a commit that referenced this pull request Oct 11, 2021
…ig` (#4056)

This removes the logic which manipulates the user's webpack plugin options to include the nextjs config's `distDir` option (if set) in the plugin option's `include` property, as that manipulation results in more files than we want being uploaded (by essentially undoing #3845). 

A more selective version of this merging will be included in #4017.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants