Non-deterministic diagonstics output

[weiznich]

Given the following code: diesel-rs/diesel@49b39fc

Diesel has a larger amount of compile test to check whether some constructs are rejected at compile time and to check for specific error messages. These tests are implemented using trybuild. We pin specific compiler versions as the output obviously might change with newer compiler versions. From time to time an update of the pinned version is required. While doing such an update we noticed that newer versions of rustc do emit non-deterministic error messages. These sometimes include more or less information.

To reproduce this behaviour perform the following steps:

git clone https://github.com/diesel-rs/diesel
cd diesel 
git checkout 49b39fc9f92018b41b48bba2b9401dad953899fe
cd diesel_compile_tests
# edit the rust-toolchain file to a "recent" nightly compiler
TRYBUILD=overwrite cargo test #adopt the output for the chosen compiler version
cargo test # execute the tests again to observe the tests failing again

We do not observe this behaviour with nightly-2022-08-12. I've observed this behaviour at least since nightly-2022-09-15 (that's the earliest nightly I've checked, so there might be earlier versions as well)

Exemplary changes between two runs (obviously that changes as well):

diff --git a/diesel_compile_tests/tests/fail/find_requires_correct_type.stderr b/diesel_compile_tests/tests/fail/find_requires_correct_type.stderr
index 91374aa0e3..9a6bd16fe6 100644
--- a/diesel_compile_tests/tests/fail/find_requires_correct_type.stderr
+++ b/diesel_compile_tests/tests/fail/find_requires_correct_type.stderr
@@ -60,6 +60,8 @@ error[E0277]: the trait bound `{integer}: diesel::Expression` is not satisfied
              (T0, T1, T2, T3, T4, T5, T6, T7)
            and $N others
    = note: required for `diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>` to implement `diesel::Expression`
+   = note: 1 redundant requirement hidden
+   = note: required for `diesel::expression::grouped::Grouped<diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>>` to implement `diesel::Expression`
 note: required for `string_primary_key::columns::id` to implement `EqAll<{integer}>`
   --> tests/fail/find_requires_correct_type.rs:13:9
    |
@@ -91,6 +93,8 @@ error[E0277]: the trait bound `{integer}: ValidGrouping<()>` is not satisfied
              <(T0, T1, T2, T3, T4, T5, T6, T7) as ValidGrouping<__GroupByClause>>
            and $N others
    = note: required for `diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>` to implement `ValidGrouping<()>`
+   = note: 1 redundant requirement hidden
+   = note: required for `diesel::expression::grouped::Grouped<diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>>` to implement `ValidGrouping<()>`
    = note: required for `diesel::expression::grouped::Grouped<diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>>` to implement `NonAggregate`
    = note: required for `SelectStatement<FromClause<string_primary_key::table>>` to implement `FilterDsl<diesel::expression::grouped::Grouped<diesel::expression::operators::Eq<string_primary_key::columns::id, {integer}>>>`
    = note: 1 redundant requirement hidden
diff --git a/diesel_compile_tests/tests/fail/join_with_explicit_on_requires_valid_boolean_expression.stderr b/diesel_compile_tests/tests/fail/join_with_explicit_on_requires_valid_boolean_expression.stderr
index 8522dc94c6..b116c48165 100644
--- a/diesel_compile_tests/tests/fail/join_with_explicit_on_requires_valid_boolean_expression.stderr
+++ b/diesel_compile_tests/tests/fail/join_with_explicit_on_requires_valid_boolean_expression.stderr
@@ -26,8 +26,6 @@ error[E0277]: the trait bound `diesel::sql_types::Integer: BoolOrNullableBool` i
              Bool
              Nullable<Bool>
    = note: required for `JoinOn<query_source::joins::Join<users::table, posts::table, Inner>, users::columns::id>` to implement `QuerySource`
-   = note: required for `SelectStatement<FromClause<JoinOn<query_source::joins::Join<users::table, posts::table, Inner>, users::columns::id>>>` to implement `Query`
-   = note: required for `SelectStatement<FromClause<JoinOn<query_source::joins::Join<users::table, posts::table, Inner>, users::columns::id>>>` to implement `AsQuery`
    = note: required for `SelectStatement<FromClause<users::table>>` to implement `InternalJoinDsl<posts::table, Inner, users::columns::id>`
    = note: 1 redundant requirement hidden
    = note: required for `users::table` to implement `InternalJoinDsl<posts::table, Inner, users::columns::id>`

[jyn514]

If you have time, it would be helpful to find exactly which commit regressed this (using https://github.com/rust-lang/cargo-bisect-rustc).

[weiznich]

searched nightlies: from nightly-2022-08-12 to nightly-2022-09-15
regressed nightly: nightly-2022-08-23
searched commit range: c0941df...015a824
regressed commit: 0b71ffc

bisected with cargo-bisect-rustc v0.6.5

Host triple: x86_64-unknown-linux-gnu
Reproduce with:

cargo bisect-rustc --start=2022-08-12 --end=2022-09-15 --script=./test.sh 

test.sh:

#!/bin/sh
set -x 

TRYBUILD=overwrite cargo test

cargo test

cc @compiler-errors as author of the relevant PR

[compiler-errors]

I'll take a look.

@rustbot claim