Skip to content

rustdoc runs some markdown tests twice #42726

Closed
@ollie27

Description

@ollie27

Running rustdoc --test with 1.18.0 and nightly on the following markdown file:

# Foo

```
fail1
```

```
fail2
```

bar
```
fail3
```

Produces the following:

output
running 5 tests
test foo.md - Foo (line 3) ... FAILED
test foo.md - Foo (line 7) ... FAILED
test foo.md - Foo (line 12) ... FAILED
test foo.md - Foo (line 8) ... FAILED
test foo.md - Foo (line 3) ... FAILED

failures:

---- foo.md - Foo (line 3) stdout ----
        error[E0425]: cannot find value `fail1` in this scope
 --> <anon>:2:1
  |
2 | fail1
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- foo.md - Foo (line 7) stdout ----
        error[E0425]: cannot find value `fail2` in this scope
 --> <anon>:2:1
  |
2 | fail2
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 12) stdout ----
        error[E0425]: cannot find value `fail3` in this scope
 --> <anon>:2:1
  |
2 | fail3
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 8) stdout ----
        error[E0425]: cannot find value `fail2` in this scope
 --> <anon>:2:1
  |
2 | fail2
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 3) stdout ----
        error[E0425]: cannot find value `fail1` in this scope
 --> <anon>:2:1
  |
2 | fail1
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218


failures:
    foo.md - Foo (line 12)
    foo.md - Foo (line 3)
    foo.md - Foo (line 3)
    foo.md - Foo (line 7)
    foo.md - Foo (line 8)

test result: FAILED. 0 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out

The first 2 tests are run twice (one with the wrong line number though). The last test is only run once. It looks as though it's running all the tests found by hoedown and all the tests run by pulldown-cmark. I'm pretty sure this wasn't the intention of #41290 and #41431 cc. @GuillaumeGomez. Note that this doesn't happen for tests in rust files.

This is also technically a stable to stable regression because the following file passes rustdoc --test on 1.17.0 but not on 1.18.0 or later:

# Foo

bar
```
fail3
```

Metadata

Metadata

Assignees

No one assigned

    Labels

    T-dev-toolsRelevant to the dev-tools subteam, which will review and decide on the PR/issue.T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions