Stabilize allow_fail test flag

[GuillaumeGomez]

Implemented in #42219.

I'll make the stabilization.

[gnzlbg]

So I just had the following problem with #[allow_fail].

I have a test that should raise a cpu exception (e.g. segfault) and fail, so from the docs it looked like #[allow_fail] was exactly what I needed.

This does not work: the test segfaulting brings the whole test harness down (I assume that a test aborting the process with an error code also brings the whole test harness down).

In hindsight, after having read the source code, it is clear what is going on, but users should not have to read the source code to know what is meant by "fail" (it is a pretty general term).

As I see it the best solution would be to rename #[allow_fail] to #[allow_assert_fail] or similar and to document in the docs exactly which kind of "failures" are supported.

An alternative that I would prefer more (since it would cover my use case) but involves significant work that will delay stabilization would be to allow more kinds of failures by spawning the #[allow_fail] tests on a different process and documenting exactly which kind of failures are allowed.

[jonhoo]

@gnzlbg interesting... That's almost more in the domain of #[should_panic], but even that wouldn't be enough to handle a cpu exception. I can see how the naming of #[allow_fail] might make you think that it could work for your use-case though. I don't love #[allow_assert_fail], as it's fairly long, and prone to misremembering. I also don't know that #[allow_fail] currently only allows fails as a result of calls to assert*! macros (maybe someone can clarify).

[nrc]

Discussed at the dev-tools meeting today: we plan to support this via external test frameworks, and so will not stabilise this. It will be deprecated and removed once such test frameworks exist.

[killercup]

Notes from meeting: macro_rules! possibly_failing_test { ($name:ident, $body:expr) => { #[test] fn $name() { let _ = ::std::panic::catch_unwind(|| $body ); } } } is probably a stable alternative.

[m-ou-se]

This hasn't gone anywhere in many years, with no clear use cases.

A new similar feature could probably use a RFC with clear use cases and a discussion of all possible solutions.

@rfcbot close

[rfcbot]

Team member @m-ou-se has proposed to close this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se
• @yaahc

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.