Skip to content

perf triage classification of PR 89611 was "probably changed"; why not definite?  #1109

Open
@pnkfelix

Description

@pnkfelix

Looking at the performance report linked here:

Benchmark & Profile Scenario % Change Significance Factor ?
encoding opt full -0.80%? 2.21x
encoding opt incr-full -0.72%? 2.36x
coercions check incr-patched: println -0.64% 4.82x
helloworld debug incr-full -0.53%? 2.07x
helloworld debug incr-unchanged -0.52% 4.79x
coercions check incr-unchanged -0.51% 7.92x
helloworld debug full -0.45%? 1.83x
coercions opt incr-unchanged -0.43% 5.67x
ripgrep opt incr-unchanged -0.42% 2.73x
cranelift-codegen opt incr-unchanged 0.38%? 1.81x
coercions debug incr-unchanged -0.36% 3.87x
issue-58319 debug incr-unchanged -0.34% 2.09x
token-stream-stress debug full -0.31% 1.74x
token-stream-stress debug incr-unchanged -0.31% 2.25x
coercions doc full -0.25% 2.89x
unicode_normalization check incr-unchanged -0.24% 3.79x
coercions check full -0.24% 3.79x
issue-58319 debug incr-full -0.24% 2.00x
tuple-stress debug incr-unchanged -0.24% 1.94x
helloworld opt incr-patched: println -0.21% 2.55x
helloworld opt incr-unchanged -0.20% 2.19x
coercions check incr-full -0.20% 5.63x

why isn't this classified as a definite improvement in the performance triage script?

Yes, a lot of them have question marks on them (though I'm not sure the performance triage script includes the question marks in its weighting at this point, based on what I saw elsewhere), but even if you remove the question mark cases, you're still left with e.g.

Benchmark & Profile Scenario % Change Significance Factor ?
coercions check incr-patched: println -0.64% 4.82x
helloworld debug incr-unchanged -0.52% 4.79x
coercions check incr-unchanged -0.51% 7.92x

which seems like it should have been enough for being classified as a performance win, not a "probably" ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions