Oh rust doctest lints, where art þou? (Add a way to run clippy on doctests)

[llogiq]

Currently, doctests benefit from being code in numerous ways, such as being tested. However, this unfortunately does not (yet?) apply to clippy lints. For (an atmittedly contrived) example:

/// is the given number odd?
///
/// # Examples
///
/// ```rust
///# use testdoclints::is_odd;
/// let mut a = 1;
/// a = a + 1; // this should lint `clippy::assign_op_pattern`
/// assert!(!is_odd(a));
/// ```
pub fn is_odd(x: usize) -> bool {
    (x & 1) == 1
}

Running cargo clippy shows no lint.

To solve this, we'd need to be able to hook into the test code generation and present the resulting AST and HIR to our lints. I am unsure where to put this issue, but as clippy is not the only source of custom lints, I think solving it within rust/rustdoc makes sense.

cc @Manishearth @oli-obk

[Nemo157]

It seems like having cargo check able to check doctests as well would be handy (and would probably get clippy 99% of the way to working).

[llogiq]

@QuietMisdreavus @GuillaumeGomez any idea how we could coax rustdoc into extracting the doctests for us to lint?

[GuillaumeGomez]

Well, we already extract the doctest but just for compilation/run at the moment. The only question I have is: what interface do you want rustdoc to provide to do so? Running specifically clippy on the returned code samples or something more generic? And if so, then what would it look like?

[llogiq]

I think there might be interest in running both check (for quick checks) and clippy (for deeper inspection). However, we may need to somehow flag the code for compile-fail tests (& possibly avoid them in clippy runs by default, as clippy doesn't like non-compiling code very much).

[GuillaumeGomez]

So something more generic. I see a few different possibilities. Like providing a Rust file like build.rs that would be run on the different code samples provided. How does it sound?

[Manishearth]

That's not really useful for tooling, tooling shouldn't have to add new files to the tree to be able to run tests.

[GuillaumeGomez]

I don't have other ideas to do this nicely. But if you have any, please tell so! :)

[llogiq]

@GuillaumeGomez we have a custom driver (as has cargo check, IIRC), so the most elegant solution in terms of UI would be giving an expansion of the doctests with the respective spans in the code to our custom driver.

[Manishearth]

Actually I think this should wait until cargo-clippy has fully moved into Cargo, and then we can add a cargo doc --test --clippy mode that does this correctly.

Ultimately clippy behaves just like another rustc binary, and we're slowly consolidating all of the actual cargo logic in cargo itself.

[GuillaumeGomez]

Fine by me!

[Nemo157]

@Manishearth currently examples are tested by running cargo test --doc, with that as precedence it seems like cargo check --doc and cargo clippy --doc would be the appropriate flags to lint the examples as well.

Since cargo check is already built into Cargo, it seems like whatever path the clippy driver would take could already be developed as part of that and then just extended to cargo clippy once the move is complete.