Skip to content

ci: Streamline CI browser playwright tests #13276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 8, 2024
Merged

Conversation

mydea
Copy link
Member

@mydea mydea commented Aug 8, 2024

This PR streamlines our browser integration tests a bit more:

  1. We now only install chromium for playwright tests in most places. This may help cut down test run time a bit, as we do not need to install native dependencies for webkit and firefox everywhere.
  2. I changed how we split the matrix jobs for the browser integration tests. Previously, we sharded this via playwright. Now, we only do this for esm tests, which we now only run in chromium, not all browsers. Only the full bundle tests are run in all browsers (they are considerably faster than the esm tests...), and there we now do not use sharding, but have one matrix job for chromium (default), one for webkit and one for firefox (where each can only install the native dependencies it needs).
  3. I renamed the jobs a bit so that we can see a bit more of the job name in the Github UI... it still truncates but you see more than before, at least...
  4. For all other playwright tests (e.g. e2e tests etc) we only install chromium, AFAIK we do not have any non-chromium tests anywhere there.

Some data points:

@mydea mydea self-assigned this Aug 8, 2024
Copy link
Contributor

github-actions bot commented Aug 8, 2024

size-limit report 📦

Path Size
@sentry/browser 22.47 KB (0%)
@sentry/browser (incl. Tracing) 34.26 KB (0%)
@sentry/browser (incl. Tracing, Replay) 70.31 KB (0%)
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 63.64 KB (0%)
@sentry/browser (incl. Tracing, Replay with Canvas) 74.71 KB (0%)
@sentry/browser (incl. Tracing, Replay, Feedback) 87.32 KB (0%)
@sentry/browser (incl. Tracing, Replay, Feedback, metrics) 89.16 KB (0%)
@sentry/browser (incl. metrics) 26.77 KB (0%)
@sentry/browser (incl. Feedback) 39.42 KB (0%)
@sentry/browser (incl. sendFeedback) 27.09 KB (0%)
@sentry/browser (incl. FeedbackAsync) 31.75 KB (0%)
@sentry/react 25.24 KB (0%)
@sentry/react (incl. Tracing) 37.25 KB (0%)
@sentry/vue 26.62 KB (0%)
@sentry/vue (incl. Tracing) 36.1 KB (0%)
@sentry/svelte 22.61 KB (0%)
CDN Bundle 23.69 KB (0%)
CDN Bundle (incl. Tracing) 35.93 KB (0%)
CDN Bundle (incl. Tracing, Replay) 70.36 KB (0%)
CDN Bundle (incl. Tracing, Replay, Feedback) 75.61 KB (0%)
CDN Bundle - uncompressed 69.52 KB (0%)
CDN Bundle (incl. Tracing) - uncompressed 106.43 KB (0%)
CDN Bundle (incl. Tracing, Replay) - uncompressed 218.27 KB (0%)
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 231.16 KB (0%)
@sentry/nextjs (client) 37.1 KB (0%)
@sentry/sveltekit (client) 34.81 KB (0%)
@sentry/node 115.47 KB (+0.36% 🔺)
@sentry/node - without tracing 89.86 KB (+0.48% 🔺)
@sentry/aws-serverless 99.3 KB (+0.46% 🔺)

@mydea mydea marked this pull request as ready for review August 8, 2024 12:04
@@ -528,7 +528,7 @@ jobs:
run: yarn lerna run test --scope @sentry/profiling-node

job_browser_playwright_tests:
name: Playwright (${{ matrix.bundle }}${{ matrix.shard && format(' {0}/{1}', matrix.shard, matrix.shards) || ''}}) Tests
name: PW ${{ matrix.bundle }}${{ matrix.project && matrix.project != 'chromium' && format(' {0}', matrix.project) || ''}}${{ matrix.shard && format(' ({0}/{1})', matrix.shard, matrix.shards) || ''}} Tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really prefer if we kept Playwright as PW is not obvious for an onlooker.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this is that as of now this is so long that you often miss a lot of context:
image

making it very hard to find what kind of test this actually is :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I named it back to Playwright ...!

@mydea mydea force-pushed the fn/ci-chromium-only branch from 38e9aa0 to eccc335 Compare August 8, 2024 13:42
@mydea mydea merged commit 21830b1 into develop Aug 8, 2024
126 checks passed
@mydea mydea deleted the fn/ci-chromium-only branch August 8, 2024 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants