Skip to content

Triage 2023 12 18 #1776

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 23, 2023
Merged

Triage 2023 12 18 #1776

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ca85215
Initial triage report template.
pnkfelix Dec 19, 2023
e90ea74
Triage report template with shortened commit ids
pnkfelix Dec 19, 2023
b9ecf1a
triage for 2023-12-18.
pnkfelix Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
254 changes: 254 additions & 0 deletions triage/2023-12-18.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
# 2023-12-19 Triage Log

A lot of noise in the results this week; there was an lull in the noise
recently, so our auto-inferred noise threshold went down, and thus five PR's
were artificially flagged this week (and three supposed improvements were just
reverting to the mean). Beyond that, we had three nice improvements: the first
to debug builds in #117962 (by ceasing emission of expensive+unused
`.debug_pubnames` and `.debug_pubtypes`), a second to diesel and serde in
#119048 (by avoiding some unnecessary work), and a third to several benchmarks
in #117749 (by adding some caching of an internal compiler structure).


Triage done by **@pnkfelix**.
Revision range: [57010939..bf9229a2](https://perf.rust-lang.org/?start=57010939ed1d00076b4af0ed06a81ec69ea5e4a8&end=bf9229a2e366b4c311f059014a4aa08af16de5d8&absolute=false&stat=instructions%3Au)

**Summary**:

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.3% | [0.2%, 0.4%] | 9 |
| Regressions ❌ <br /> (secondary) | 0.4% | [0.2%, 0.9%] | 21 |
| Improvements ✅ <br /> (primary) | -1.8% | [-10.8%, -0.1%] | 103 |
| Improvements ✅ <br /> (secondary) | -1.2% | [-2.6%, -0.2%] | 93 |
| All ❌✅ (primary) | -1.6% | [-10.8%, 0.4%] | 112 |


6 Regressions, 9 Improvements, 3 Mixed; 5 of them in rollups
67 artifact comparisons made in total

#### Regressions

rustc_passes: Enforce `rustc::potential_query_instability` lint [#118870](https://github.com/rust-lang/rust/pull/118870) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=56d25ba5ea3515ba2b361b6c4636373734eb27cc&end=c3def263a44e07e09ae6d57abfc8650227fb4972&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:-------------:|:-----:|
| Regressions ❌ <br /> (primary) | 13.5% | [0.5%, 44.5%] | 8 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 13.5% | [0.5%, 44.5%] | 8 |

* already marked as triaged by lqd; this PR did not regress things, the reported change is due to a periodic "slow s3 uploads on the collector", i.e. instrument noise/bias.

fix --dry-run when the change-id warning is printed [#118789](https://github.com/rust-lang/rust/pull/118789) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1aa6aefdc92555b3fbc5ae4c99365df9845a3e31&end=529047cfc3f4f7b3ea5aaac054408f368d153727&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.9% | [0.8%, 1.0%] | 4 |
| Regressions ❌ <br /> (secondary) | 0.5% | [0.3%, 0.8%] | 18 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.9% | [0.8%, 1.0%] | 4 |

* already marked as triaged by lqd; this PR did not regress things. lqd hypothesized that the inferred noise threshold was lowered (due to a period of low noisiness?), and thus normal amounts of noise are tripping the threshold.

Rollup of 3 pull requests [#118982](https://github.com/rust-lang/rust/pull/118982) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=4d1bd0db7f489b22c6d8aa2385937a95412c015b&end=e6707df0de337976dce7577e68fc57adcd5e4842&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.8% | [0.8%, 0.9%] | 4 |
| Regressions ❌ <br /> (secondary) | 0.4% | [0.0%, 0.8%] | 32 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.8% | [0.8%, 0.9%] | 4 |


* already marked as triaged by lqd. We think this is another case where the inferred noise threshold was low and we are seeing regression reports as the noise ramps up again.

Rollup of 5 pull requests [#119002](https://github.com/rust-lang/rust/pull/119002) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=a96d57bdb6d2bb6d233d7d5aaefc2995ab99be01&end=ca5c68a110099b17c3537e6e5840dbcc21b0e3ca&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.2% | [0.1%, 0.5%] | 40 |
| Regressions ❌ <br /> (secondary) | 0.4% | [0.1%, 0.7%] | 36 |
| Improvements ✅ <br /> (primary) | -0.3% | [-0.3%, -0.3%] | 1 |
| Improvements ✅ <br /> (secondary) | -0.1% | [-0.1%, -0.1%] | 2 |
| All ❌✅ (primary) | 0.2% | [-0.3%, 0.5%] | 41 |

* lqd identified PR #118396, "Collect lang items from AST, get rid of GenericBound::LangItemTrait" as the root cause for the regressions here.
* not marking as triaged

More unescaping cleanups [#118897](https://github.com/rust-lang/rust/pull/118897) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1c6a06183a1452c2772db370b572e38bac9c5d86&end=3ad8e2d129a67b0dc450d6ef1100ae41d2cd4b8a&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.6% | [0.1%, 0.8%] | 5 |
| Regressions ❌ <br /> (secondary) | 0.4% | [0.2%, 0.7%] | 26 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.6% | [0.1%, 0.8%] | 5 |

* lqd theorized this is another instance of noise threshold wriggling
* pnkfelix agrees and marked as triaged.

coverage: Regression test for `assert!(!false)` [#119011](https://github.com/rust-lang/rust/pull/119011) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=445177724ac2beebf4faa5a3ffacb2adbe7e7846&end=9022e6a39c2714d2f447285c4ee7513bf153e3a9&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:----:|:------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.7% | [0.7%, 0.8%] | 4 |
| Regressions ❌ <br /> (secondary) | 0.5% | [0.3%, 0.6%] | 2 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | 0.7% | [0.7%, 0.8%] | 4 |

* yet again, helloworld check is being flagged as regressing (in four scenarios), probably due to wriggling noise threshold.
* pnkfelix marking as triaged

#### Improvements

Rollup of 8 pull requests [#118838](https://github.com/rust-lang/rust/pull/118838) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=57010939ed1d00076b4af0ed06a81ec69ea5e4a8&end=21cce21d8c012f14cf74d5afddd795d324600dac&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | - | - | 0 |
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.0%, -0.9%] | 6 |
| All ❌✅ (primary) | - | - | 0 |


fix: stop emitting `.debug_pubnames` and `.debug_pubtypes` [#117962](https://github.com/rust-lang/rust/pull/117962) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=21cce21d8c012f14cf74d5afddd795d324600dac&end=e2a3c9b3f0895c866c104bd2fff2a8bf16eaf964&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -2.1% | [-5.2%, -0.3%] | 24 |
| Improvements ✅ <br /> (secondary) | -1.6% | [-3.1%, -0.2%] | 61 |
| All ❌✅ (primary) | -2.1% | [-5.2%, -0.3%] | 24 |

* nice improvement to debug builds, especially for ripgrep and serde_derive

Unbreak non-unix non-windows bootstrap [#118894](https://github.com/rust-lang/rust/pull/118894) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=c3def263a44e07e09ae6d57abfc8650227fb4972&end=7176b8babdc55d68a485ab61940b10611cfa6c57&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:------:|:---------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -10.4% | [-30.8%, -0.6%] | 8 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | -10.4% | [-30.8%, -0.6%] | 8 |

* (don't get excited; this is just a return-to-normal after the artificial regression flagged with PR #118870 above)

cache param env canonicalization [#117749](https://github.com/rust-lang/rust/pull/117749) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e6d1b0ec9859e6f5c29aaa3b6525fb625bf354ad&end=d23e1a689426638a5146c204e09278433c509610&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -2.0% | [-4.8%, -0.2%] | 32 |
| Improvements ✅ <br /> (secondary) | -1.3% | [-1.4%, -1.3%] | 2 |
| All ❌✅ (primary) | -2.0% | [-4.8%, -0.2%] | 32 |

* pretty solid improvement here to serde, diesel, bitmaps, hyper, image.

`-Ztrait-solver=next` to `-Znext-solver` [#118937](https://github.com/rust-lang/rust/pull/118937) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=529047cfc3f4f7b3ea5aaac054408f368d153727&end=2ecba0fa00b75e7291978c50bece407f17296f45&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.9% | [-1.0%, -0.9%] | 4 |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.8%, -0.3%] | 21 |
| All ❌✅ (primary) | -0.9% | [-1.0%, -0.9%] | 4 |

* (just noise)

Use the LLVM option NoTrapAfterNoreturn [#110494](https://github.com/rust-lang/rust/pull/110494) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e223c41028def5123ffd2a4a51a9109b1aae2c2e&end=02ad6676ddc7ea7bc3acd73f383180bc80185aeb&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.9% | [-0.9%, -0.8%] | 4 |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.8%, -0.3%] | 25 |
| All ❌✅ (primary) | -0.9% | [-0.9%, -0.8%] | 4 |

* (just noise)

Add support for `--env` on `tracked_env::var` [#118830](https://github.com/rust-lang/rust/pull/118830) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=9022e6a39c2714d2f447285c4ee7513bf153e3a9&end=5e7025419d1b1359e4cb7cd365ba5900c8299089&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.7% | [-0.8%, -0.7%] | 3 |
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.7%, -0.3%] | 7 |
| All ❌✅ (primary) | -0.7% | [-0.8%, -0.7%] | 3 |

* (just noise)

don't fold ParamEnv in register_predicate_obligation [#119048](https://github.com/rust-lang/rust/pull/119048) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=d14e52b47ab27bcc3c3e083ed5f009a57db63eff&end=4283aeaca5c0c14864d4af3369a42c6b8b21f80f&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -2.1% | [-7.3%, -0.2%] | 23 |
| Improvements ✅ <br /> (secondary) | -0.3% | [-0.3%, -0.3%] | 1 |
| All ❌✅ (primary) | -2.1% | [-7.3%, -0.2%] | 23 |

* nice improvement to diesel and serde.

resolve: Replace visibility table in resolver outputs with query feeding [#118657](https://github.com/rust-lang/rust/pull/118657) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=43dcc9b786c922251c3fbe75a0142f50f07053ca&end=321b6565a5f8ebf6f23622fc91ea1191a54f2a38&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.3% | [-0.5%, -0.1%] | 70 |
| Improvements ✅ <br /> (secondary) | -0.6% | [-1.2%, -0.1%] | 42 |
| All ❌✅ (primary) | -0.3% | [-0.5%, -0.1%] | 70 |


#### Mixed

codegen_llvm: set `DW_AT_accessibility` [#115165](https://github.com/rust-lang/rust/pull/115165) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e6707df0de337976dce7577e68fc57adcd5e4842&end=3f39cae1199a2a0217c3646a16d1ae7fa599130b&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.2% | [0.2%, 0.3%] | 4 |
| Regressions ❌ <br /> (secondary) | - | - | 0 |
| Improvements ✅ <br /> (primary) | -0.7% | [-0.7%, -0.6%] | 4 |
| Improvements ✅ <br /> (secondary) | -0.4% | [-0.7%, -0.2%] | 22 |
| All ❌✅ (primary) | -0.2% | [-0.7%, 0.3%] | 8 |

* the improvements are helloworld noise; the regressions are minor.

Rollup of 6 pull requests [#119009](https://github.com/rust-lang/rust/pull/119009) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=5c927ab99946c312eed65f28aea15d2a035441f0&end=1c6a06183a1452c2772db370b572e38bac9c5d86&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | 0.2% | [0.2%, 0.2%] | 1 |
| Regressions ❌ <br /> (secondary) | 0.4% | [0.2%, 1.2%] | 8 |
| Improvements ✅ <br /> (primary) | -1.3% | [-1.9%, -0.9%] | 8 |
| Improvements ✅ <br /> (secondary) | - | - | 0 |
| All ❌✅ (primary) | -1.1% | [-1.9%, 0.2%] | 9 |

* The single tiny regression to exa doc full does not warrant spending time trying to tease apart this rollup, IMO.
* marked as triaged

Rollup of 5 pull requests [#119069](https://github.com/rust-lang/rust/pull/119069) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=cda4736f1eaad8af6f49388baa9b7e480df8e329&end=e004adb5561b724ac18f5b24584648ca4e42b6ad&stat=instructions:u)

| (instructions:u) | mean | range | count |
|:----------------------------------:|:-----:|:--------------:|:-----:|
| Regressions ❌ <br /> (primary) | - | - | 0 |
| Regressions ❌ <br /> (secondary) | 0.3% | [0.3%, 0.3%] | 2 |
| Improvements ✅ <br /> (primary) | -0.2% | [-0.3%, -0.1%] | 12 |
| Improvements ✅ <br /> (secondary) | -0.3% | [-0.5%, -0.2%] | 13 |
| All ❌✅ (primary) | -0.2% | [-0.3%, -0.1%] | 12 |

* The two minor regressions to secondary benchmarks do not warrant spending time trying to tease apart this rollup, IMO.
* marked as triaged