compiletest does not signal when a ui (or compile-fail) test with `//~ WARN` gets unexpected warnings

[pnkfelix]

In compiletest, a test that has a //~ WARN ... comment will not tell you if other undocumented warnings were also emitted.

If I recall correctly, we have a policy that if a test has annotations of the form //~ KIND (where KIND is one of ERROR/WARN/HELP/INFO`, then we will check that the set of annotations of that form properly matches what the compiler is emitting.

But it looks like we only actually follow the above policy completely for KIND == ERROR, not the others.

[memoryruins]

cc #55596

[pnkfelix]

Is this actually still a problem, given that #55596 has been resolved by PR #66213?

[pnkfelix]

assigning to self to follow-up and determine if this is even a problem at this point.

[osa1]

I think this is fixed now. To check, I created this file in src/test/ui/issue-55693.rs:

// check-pass

#[warn(unused_variables)]
#[warn(non_snake_case)]

fn main() {
    let unused: usize = 123;
    let WEIRD: usize = 456;
}

I first run this with --bless which created an .stderr file, with expected contents.

Then added //~WARN in the first let. It failed with

unexpected errors (from JSON output): [
    Error {
        line_num: 8,
        kind: Some(
            Warning,
        ),
        msg: "8:9: 8:14: unused variable: `WEIRD` [unused_variables]",
    },
    Error {
        line_num: 8,
        kind: Some(
            Warning,
        ),
        msg: "8:9: 8:14: variable `WEIRD` should have a snake case name [non_snake_case]",
    },
]

So it seems like we now expect the //~WARN lines to match the compiler generated warnings.

[pnkfelix]

closing as fixed! Thanks @osa1 for looking into it!