Open
Description
Looking at the performance report linked here:
Benchmark & Profile | Scenario | % Change | Significance Factor ? |
---|---|---|---|
encoding opt | full | -0.80%? | 2.21x |
encoding opt | incr-full | -0.72%? | 2.36x |
coercions check | incr-patched: println | -0.64% | 4.82x |
helloworld debug | incr-full | -0.53%? | 2.07x |
helloworld debug | incr-unchanged | -0.52% | 4.79x |
coercions check | incr-unchanged | -0.51% | 7.92x |
helloworld debug | full | -0.45%? | 1.83x |
coercions opt | incr-unchanged | -0.43% | 5.67x |
ripgrep opt | incr-unchanged | -0.42% | 2.73x |
cranelift-codegen opt | incr-unchanged | 0.38%? | 1.81x |
coercions debug | incr-unchanged | -0.36% | 3.87x |
issue-58319 debug | incr-unchanged | -0.34% | 2.09x |
token-stream-stress debug | full | -0.31% | 1.74x |
token-stream-stress debug | incr-unchanged | -0.31% | 2.25x |
coercions doc | full | -0.25% | 2.89x |
unicode_normalization check | incr-unchanged | -0.24% | 3.79x |
coercions check | full | -0.24% | 3.79x |
issue-58319 debug | incr-full | -0.24% | 2.00x |
tuple-stress debug | incr-unchanged | -0.24% | 1.94x |
helloworld opt | incr-patched: println | -0.21% | 2.55x |
helloworld opt | incr-unchanged | -0.20% | 2.19x |
coercions check | incr-full | -0.20% | 5.63x |
why isn't this classified as a definite improvement in the performance triage script?
Yes, a lot of them have question marks on them (though I'm not sure the performance triage script includes the question marks in its weighting at this point, based on what I saw elsewhere), but even if you remove the question mark cases, you're still left with e.g.
Benchmark & Profile | Scenario | % Change | Significance Factor ? |
---|---|---|---|
coercions check | incr-patched: println | -0.64% | 4.82x |
helloworld debug | incr-unchanged | -0.52% | 4.79x |
coercions check | incr-unchanged | -0.51% | 7.92x |
which seems like it should have been enough for being classified as a performance win, not a "probably" ?
Metadata
Metadata
Assignees
Labels
No labels