[DO MERGE] PGO new solver #141453

compiler-errors · 2025-05-23T15:01:28Z

🤔

I hope I did this right...

compiler-errors · 2025-05-23T15:01:40Z

@bors try @rust-timer queue

bors · 2025-05-23T15:02:54Z

⌛ Trying commit 8c50bba with merge d8dcadd...

[DONT MERGE] PGO new solver 🤔 I hope I did this right...

Kobzol · 2025-05-23T15:06:24Z

Looks about right. I was planning to do this, so thanks! We'll see how much longer it takes to run these new benchmarks, maybe it will be fine to just have 1-2 of the new-solver crates in the training set.

compiler-errors · 2025-05-23T15:09:16Z

Yep, I'm not prepared to land this yet, I just want to see how much PGO actually matters for the new solver. I think just including a couple stressy crates like nalgebra would be sufficient :>

bors · 2025-05-23T15:30:03Z

💔 Test failed - checks-actions

Kobzol · 2025-05-23T15:39:43Z

Oh, sorry, this is essentially a bug in the rustc-perf CLI parser, it tries to detect unused benchmarks, but it doesn't handle well situations where one benchmark is a prefix of another. Removing bitmaps-3.2.1-new-solver from the PGO set should fix this.

compiler-errors · 2025-05-23T15:45:24Z

@bors try @rust-timer queue

bors · 2025-05-23T15:46:34Z

⌛ Trying commit e1d250f with merge 1fde446...

[DONT MERGE] PGO new solver 🤔 I hope I did this right...

bors · 2025-05-23T18:12:54Z

☀️ Try build successful - checks-actions
Build commit: 1fde446 (1fde4461620ecd56d23e8cc2559cc54591ba40f7)

rust-timer · 2025-05-24T03:28:59Z

Finished benchmarking commit (1fde446): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.1%, 1.7%]	123
Regressions ❌ (secondary)	0.9%	[0.1%, 4.2%]	122
Improvements ✅ (primary)	-0.4%	[-0.8%, -0.1%]	44
Improvements ✅ (secondary)	-5.8%	[-17.5%, -0.1%]	36
All ❌✅ (primary)	0.2%	[-0.8%, 1.7%]	167

Max RSS (memory usage)

Results (primary -2.1%, secondary -3.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.7%	[1.6%, 9.9%]	8
Improvements ✅ (primary)	-2.1%	[-6.3%, -0.9%]	43
Improvements ✅ (secondary)	-4.9%	[-9.0%, -1.8%]	32
All ❌✅ (primary)	-2.1%	[-6.3%, -0.9%]	43

Cycles

Results (primary -1.1%, secondary -11.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.6%	[2.6%, 2.6%]	1
Improvements ✅ (primary)	-1.1%	[-1.3%, -0.9%]	2
Improvements ✅ (secondary)	-11.7%	[-22.2%, -1.7%]	21
All ❌✅ (primary)	-1.1%	[-1.3%, -0.9%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 776.992s -> 774.276s (-0.35%)
Artifact size: 365.49 MiB -> 368.39 MiB (0.79%)

Kobzol · 2025-05-24T06:10:13Z

Yup, looks like it helps quite a lot! We should land the general benchmark update and rustc-perd bump separately from adding the new solver benchmarks.

compiler-errors · 2025-05-24T11:02:36Z

Blocked on #141490

@rustbot author

bors · 2025-05-29T23:53:56Z

☔ The latest upstream changes (presumably #141490) made this pull request unmergeable. Please resolve the merge conflicts.

compiler-errors · 2025-05-30T07:55:32Z

@bors try @rust-timer queue

rust-timer · 2025-05-30T22:32:05Z

Finished benchmarking commit (9ef33a0): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.4%]	9
Regressions ❌ (secondary)	0.2%	[0.1%, 0.4%]	5
Improvements ✅ (primary)	-0.2%	[-0.2%, -0.1%]	3
Improvements ✅ (secondary)	-7.1%	[-15.1%, -0.2%]	16
All ❌✅ (primary)	0.2%	[-0.2%, 0.4%]	12

Max RSS (memory usage)

Results (secondary -0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.2%	[0.7%, 9.0%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	[-4.0%, -0.5%]	19
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary -5.0%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.0%	[0.5%, 2.0%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-6.4%	[-18.5%, -0.4%]	25
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: missing data
Artifact size: 370.19 MiB -> 371.51 MiB (0.36%)

compiler-errors · 2025-05-30T22:35:09Z

I think the results are basically similar.

Let me squash this and this should be good? Or is this still affecting bootstrap time (idk how to check)?

Kobzol · 2025-05-31T06:42:50Z

I think it's fine like this, thanks.

compiler-errors · 2025-05-31T10:03:03Z

@rustbot ready

rustbot · 2025-05-31T10:03:32Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Kobzol · 2025-05-31T10:47:36Z

r? kobzol

@bors r+

bors · 2025-05-31T10:47:38Z

📌 Commit 3253de6 has been approved by Kobzol

It is now in the queue for this repository.

bors · 2025-05-31T11:34:08Z

⌛ Testing commit 3253de6 with merge 738c08b...

Kobzol · 2025-05-31T11:58:55Z

Funny, apparently bors doesn't understand DONT MERGE xD

compiler-errors · 2025-05-31T12:00:04Z

lol

bors · 2025-05-31T14:55:26Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing 738c08b to master...

github-actions · 2025-05-31T14:58:14Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing e0d014a (parent) -> 738c08b (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 738c08b63c4f9e3ebdaec5eece7b6fbc354f6467 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-linux: 5730.5s -> 7727.0s (34.8%)
dist-various-1: 4382.0s -> 4887.9s (11.5%)
x86_64-apple-2: 4757.4s -> 5246.1s (10.3%)
x86_64-apple-1: 8881.7s -> 9750.8s (9.8%)
x86_64-rust-for-linux: 2591.4s -> 2840.5s (9.6%)
dist-x86_64-apple: 8900.4s -> 8047.2s (-9.6%)
x86_64-msvc-ext1: 6875.2s -> 7281.0s (5.9%)
dist-x86_64-musl: 7127.7s -> 7470.8s (4.8%)
aarch64-gnu: 7031.3s -> 6708.3s (-4.6%)
x86_64-gnu-llvm-19-1: 3951.3s -> 3771.4s (-4.6%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-05-31T17:18:02Z

Finished benchmarking commit (738c08b): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

If the regression was expected or you think it can be justified,
please write a comment with sufficient written justification, and add
@rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
If you think that you know of a way to resolve the regression, try to create
a new PR with a fix for the regression.
If you do not understand the regression or you think that it is just noise,
you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.4%]	10
Regressions ❌ (secondary)	0.2%	[0.2%, 0.3%]	6
Improvements ✅ (primary)	-0.2%	[-0.2%, -0.2%]	1
Improvements ✅ (secondary)	-8.1%	[-15.0%, -3.3%]	14
All ❌✅ (primary)	0.3%	[-0.2%, 0.4%]	11

Max RSS (memory usage)

Results (secondary 0.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.2%	[0.4%, 9.0%]	12
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.8%	[-4.0%, -0.5%]	25
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary -4.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.6%	[0.5%, 9.2%]	9
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-7.4%	[-19.7%, -0.4%]	22
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 777.196s -> 776.294s (-0.12%)
Artifact size: 370.47 MiB -> 372.28 MiB (0.49%)

Mark-Simulacrum · 2025-06-02T12:47:21Z

Regressions are acceptable given the wins and intent of the PR.

rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label May 23, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 23, 2025

bors added a commit that referenced this pull request May 23, 2025

Auto merge of #141453 - compiler-errors:pgo-solver, r=<try>

d8dcadd

[DONT MERGE] PGO new solver 🤔 I hope I did this right...

This comment has been minimized.

Sign in to view

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label May 23, 2025

compiler-errors force-pushed the pgo-solver branch from 8c50bba to e1d250f Compare May 23, 2025 15:45

This comment has been minimized.

Sign in to view

bors added a commit that referenced this pull request May 23, 2025

Auto merge of #141453 - compiler-errors:pgo-solver, r=<try>

1fde446

[DONT MERGE] PGO new solver 🤔 I hope I did this right...

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 24, 2025

compiler-errors force-pushed the pgo-solver branch from e1d250f to 0a84dfa Compare May 24, 2025 11:02

compiler-errors force-pushed the pgo-solver branch from 0a84dfa to f7d42c9 Compare May 30, 2025 07:55

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 30, 2025

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 30, 2025

Add some new solver to PGO

3253de6

compiler-errors force-pushed the pgo-solver branch from 0ab1217 to 3253de6 Compare May 31, 2025 09:54

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 31, 2025

compiler-errors marked this pull request as ready for review May 31, 2025 10:03

rustbot assigned Mark-Simulacrum May 31, 2025

rustbot assigned Kobzol May 31, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 31, 2025

rustbot unassigned Mark-Simulacrum May 31, 2025

compiler-errors changed the title ~~[DONT MERGE] PGO new solver~~ [DO MERGE] PGO new solver May 31, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label May 31, 2025

bors merged commit 738c08b into rust-lang:master May 31, 2025
10 checks passed

rustbot added this to the 1.89.0 milestone May 31, 2025

Mark-Simulacrum added the perf-regression-triaged The performance regression has been triaged. label Jun 2, 2025

[DO MERGE] PGO new solver #141453

[DO MERGE] PGO new solver #141453

Conversation

compiler-errors commented May 23, 2025

Uh oh!

compiler-errors commented May 23, 2025

Uh oh!

This comment has been minimized.

bors commented May 23, 2025

Uh oh!

Kobzol commented May 23, 2025

Uh oh!

compiler-errors commented May 23, 2025

Uh oh!

This comment has been minimized.

bors commented May 23, 2025

Uh oh!

Kobzol commented May 23, 2025

Uh oh!

compiler-errors commented May 23, 2025

Uh oh!

This comment has been minimized.

bors commented May 23, 2025

Uh oh!

bors commented May 23, 2025

Uh oh!

This comment has been minimized.

rust-timer commented May 24, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Kobzol commented May 24, 2025

Uh oh!

compiler-errors commented May 24, 2025

Uh oh!

bors commented May 29, 2025

Uh oh!

compiler-errors commented May 30, 2025

Uh oh!

This comment has been minimized.

rust-timer commented May 30, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

compiler-errors commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented May 31, 2025

Uh oh!

compiler-errors commented May 31, 2025

Uh oh!

rustbot commented May 31, 2025

Uh oh!

Kobzol commented May 31, 2025

Uh oh!

bors commented May 31, 2025

Uh oh!

bors commented May 31, 2025

Uh oh!

Kobzol commented May 31, 2025

Uh oh!

compiler-errors commented May 31, 2025

Uh oh!

bors commented May 31, 2025

Uh oh!

Uh oh!

github-actions bot commented May 31, 2025

Test differences

Job duration changes

Uh oh!

rust-timer commented May 31, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Mark-Simulacrum commented Jun 2, 2025

Uh oh!

Uh oh!

compiler-errors commented May 30, 2025 •

edited

Loading