rustdoc crashes parsing strange doc-comment on 1.36

[bobbobbio]

After upgrading to 1.36 rustdoc is crashing like this when run on my crate:

thread 'rustc' panicked at 'unexpected item body MaybeEmphasis(2, true, false)', /work/toolchain/packages/rust/rustc-1.36.0-src/vendor/pulldown-cmark/src/parse.rs:2418:14                                                                                                                                                                            
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.                                                                                                                                                                                                                                                               
stack backtrace:                                                                                                                                                                                                                                                                                                                                      
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace                                                                                                                                                                                                                                                                                       
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39                                                                                                                                                                                                                                                                                     
   1: std::sys_common::backtrace::print                                                                                                                                                                                                                                                                                                               
             at src/libstd/sys_common/backtrace.rs:71                                                                                                                                                                                                                                                                                                 
             at src/libstd/sys_common/backtrace.rs:59                                                                                                                                                                                                                                                                                                 
   2: std::panicking::default_hook::{{closure}}                                                                                                                                                                                                                                                                                                       
             at src/libstd/panicking.rs:197                                                                                                                                                                                                                                                                                                           
   3: std::panicking::default_hook                                                                                                                                                                                                                                                                                                                    
             at src/libstd/panicking.rs:211                                                                                                                                                                                                                                                                                                           
   4: std::panicking::rust_panic_with_hook                                                                                                                                                                                                                                                                                                            
             at src/libstd/panicking.rs:478                                                                                                                                                                                                                                                                                                           
   5: std::panicking::continue_panic_fmt                                                                                                                                                                                                                                                                                                              
             at src/libstd/panicking.rs:381                                                                                                                                                                                                                                                                                                           
   6: std::panicking::begin_panic_fmt                                                                                                                                                                                                                                                                                                                 
             at src/libstd/panicking.rs:336                                                                                                                                                                                                                                                                                                           
   7: pulldown_cmark::parse::item_to_event                                                                                                                                                                                                                                                                                                            
             at /work/toolchain/packages/rust/rustc-1.36.0-src/vendor/pulldown-cmark/src/parse.rs:2418                                                                                                                                                                                                                                                
   8: <pulldown_cmark::parse::Parser as core::iter::traits::iterator::Iterator>::next                                                                                                                                                                                                                                                                 
             at /work/toolchain/packages/rust/rustc-1.36.0-src/vendor/pulldown-cmark/src/parse.rs:2487                                                                                                                                                                                                                                                
   9: rustdoc::html::markdown::find_testable_code                                                                                                                                                                                                                                                                                                     
             at src/librustdoc/html/markdown.rs:539                                                                                                                                                                                                                                                                                                   
  10: rustdoc::passes::look_for_tests                                                                                                                                                                                                                                                                                                                 
             at src/librustdoc/passes/mod.rs:338                                                                                                                                                                                                                                                                                                      
  11: <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item                                                                                                                                                                                                                                     
             at src/librustdoc/passes/private_items_doc_tests.rs:35                                                                                                                                                                                                                                                                                   
  12: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter                                                                                                                                                                                                                                                                                  
             at src/librustdoc/fold.rs:51                                                                                                                                                                                                                                                                                                             
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826                                                                                                                                                                                                                                                   
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1572                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826                                                                                                                                                                                                                                                   
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1609                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:812                                                                                                                                                                                                                                                   
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1939                                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1836                                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1831                                                                                                                                                                                                                                                               
  13: rustdoc::fold::DocFolder::fold_inner_recur                                                                                                                                                                                                                                                                                                      
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1731                                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1465                                                                                                                                                                                                                                               
             at src/librustdoc/fold.rs:100                                                                                                                                                                                                                                                                                                            
             at src/librustdoc/fold.rs:27                                                                                                                                                                                                                                                                                                             
  14: <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item                                                                                                                                                                                                                                     
             at src/librustdoc/fold.rs:90                                                                                                                                                                                                                                                                                                             
             at src/librustdoc/passes/private_items_doc_tests.rs:37                                                                                                                                                                                                                                                                                   
  15: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter                                                                                                                                                                                                                                                                                  
             at src/librustdoc/fold.rs:51                                                                                                                                                                                                                                                                                                             
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826                                                                                                                                                                                                                                                   
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1572                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1609
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:812
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1939
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1836
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1831
13: rustdoc::fold::DocFolder::fold_inner_recur                                                                                                                                                                                                                                                                                                      
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1731                                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1465                                                                                                                                                                                                                                               
             at src/librustdoc/fold.rs:100                                                                                                                                                                                                                                                                                                            
             at src/librustdoc/fold.rs:27                                                                                                                                                                                                                                                                                                             
  14: <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item                                                                                                                                                                                                                                     
             at src/librustdoc/fold.rs:90                                                                                                                                                                                                                                                                                                             
             at src/librustdoc/passes/private_items_doc_tests.rs:37                                                                                                                                                                                                                                                                                   
  15: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter                                                                                                                                                                                                                                                                                  
             at src/librustdoc/fold.rs:51                                                                                                                                                                                                                                                                                                             
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826                                                                                                                                                                                                                                                   
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1572                                                                                                                                                                                                                                               
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:826
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1609
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/adapters/mod.rs:812
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1939
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1836
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1831
  16: rustdoc::fold::DocFolder::fold_inner_recur
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/vec.rs:1731
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/iter/traits/iterator.rs:1465
             at src/librustdoc/fold.rs:100
             at src/librustdoc/fold.rs:27
  17: <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item
             at src/librustdoc/fold.rs:90
             at src/librustdoc/passes/private_items_doc_tests.rs:37
  18: rustdoc::passes::private_items_doc_tests::check_private_items_doc_tests
             at src/librustdoc/fold.rs:105
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/option.rs:624
             at src/librustdoc/fold.rs:105
             at src/librustdoc/passes/private_items_doc_tests.rs:27
  19: rustc::ty::context::tls::enter_global
             at src/librustdoc/core.rs:466
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_interface/passes.rs:807
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:2000
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1967
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1900
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1966
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1999
  20: rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}}
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_interface/passes.rs:807
             at /work/toolchain/packages/rust/rustc-1.36.0-src/<::rustc_data_structures::box_region::declare_box_region_type macros>:17
  21: rustc_interface::passes::create_global_ctxt::{{closure}}
             at src/librustc_interface/passes.rs:873
  22: rustc_interface::interface::run_compiler_in_existing_thread_pool
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/liballoc/boxed.rs:910
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_data_structures/box_region.rs:52
             at /work/toolchain/packages/rust/rustc-1.36.0-src/<::rustc_data_structures::box_region::declare_box_region_type macros>:19
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_interface/passes.rs:807
             at src/librustdoc/core.rs:355
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_interface/interface.rs:122
  23: rustdoc::core::run_core
             at src/librustdoc/core.rs:341
  24: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at src/librustdoc/lib.rs:455
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/panic.rs:315
  25: std::panicking::try::do_call
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/panicking.rs:293
  26: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:85
  27: rustc_driver::report_ices_to_stderr_if_any
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/panicking.rs:272
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/panic.rs:394
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_driver/lib.rs:1115
  28: rustdoc::main_args
             at src/librustdoc/lib.rs:452
             at src/librustdoc/lib.rs:408
  29: std::thread::local::LocalKey<T>::with
             at src/librustdoc/lib.rs:98
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libcore/option.rs:416
             at src/librustdoc/lib.rs:98
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc_interface/util.rs:186
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1955
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/thread/local.rs:299
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/thread/local.rs:245
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/librustc/ty/context.rs:1947
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/thread/local.rs:299
             at /work/toolchain/packages/rust/rustc-1.36.0-src/src/libstd/thread/local.rs:245

It doesn't produce the line that caused it to crash and the crate is quite large.

It seems to correspond with perhaps this issue in pulldown-cmark:
pulldown-cmark/pulldown-cmark#306

[bobbobbio]

by process of elimination I determined this was the offending doc-comment:

#[doc = "  _   _           _        ___    _"]                                                      
#[doc = " | \\ | | ___   __| | ___  |_ _|__| |"]                                                    
#[doc = " |  \\| |/ _ \\ / _` |/ _ \\  | |/ _` |"]                                                  
#[doc = " | |\\  | (_) | (_| |  __/  | | (_| |"]                                                    
#[doc = " |_| \\_|\\___/ \\__,_|\\___| |___\\__,_|"]                                                
#[doc = ""]    

[hellow554]

Excuse me? What kind of bug is this? ^^ Thanks for finding. I managed to reduce it to this:

/// __|_ _|__||_|
pub struct A;

cargo doc then gives me:

Documenting foo v0.1.0 (/tmp/tmp.AjQQzlRVZ2/foo)
thread 'rustc' panicked at 'unexpected item body MaybeEmphasis(2, true, false)', /cargo/registry/src/github.com-1ecc6299db9ec823/pulldown-cmark-0.5.2/src/parse.rs:2418:14
stack backtrace:
   0:     0x7f8f9326519b - backtrace::backtrace::libunwind::trace::hd732a1add0f3c38f
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1:     0x7f8f9326519b - backtrace::backtrace::trace_unsynchronized::h4791df561e50498d
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2:     0x7f8f9326519b - std::sys_common::backtrace::_print::h5fa2d794cb884684
                               at src/libstd/sys_common/backtrace.rs:47
   3:     0x7f8f9326519b - std::sys_common::backtrace::print::h7cd3adf88f405cef
                               at src/libstd/sys_common/backtrace.rs:36
   4:     0x7f8f9326519b - std::panicking::default_hook::{{closure}}::h2b88c1fac98b01ce
                               at src/libstd/panicking.rs:200
   5:     0x7f8f93264e77 - std::panicking::default_hook::h983f8b3138244f75
                               at src/libstd/panicking.rs:214
   6:     0x7f8f93265910 - std::panicking::rust_panic_with_hook::h94f6d90998bef5c8
                               at src/libstd/panicking.rs:477
   7:     0x7f8f93265492 - std::panicking::continue_panic_fmt::hf49c6a853e8dcac9
                               at src/libstd/panicking.rs:384
   8:     0x7f8f932653df - std::panicking::begin_panic_fmt::ha8d124bef3f21240
                               at src/libstd/panicking.rs:339
   9:     0x560e43064143 - pulldown_cmark::parse::item_to_event::h95012e9d04fec4c5
  10:     0x560e4306423d - <pulldown_cmark::parse::Parser as core::iter::traits::iterator::Iterator>::next::hded5130fc70fc34f
  11:     0x560e42e87f1d - rustdoc::passes::look_for_tests::ha49ad2c19398a144
  12:     0x560e42ee68e9 - <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item::h2c34c33c7f518ff8
  13:     0x560e42de4bb5 - <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter::h1f5a786c5fe07bf7
  14:     0x560e42ee386f - rustdoc::fold::DocFolder::fold_inner_recur::h9d0af8fea7310887
  15:     0x560e42ee6b41 - <rustdoc::passes::private_items_doc_tests::PrivateItemDocTestLinter as rustdoc::fold::DocFolder>::fold_item::h2c34c33c7f518ff8
  16:     0x560e42ee63f4 - rustdoc::passes::private_items_doc_tests::check_private_items_doc_tests::h1a35ec8426427953
  17:     0x560e42e588e2 - rustc::ty::context::tls::enter_global::h004923b303546763
  18:     0x560e42ebe915 - rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}}::h0d82a7876664b846
  19:     0x7f8f967ef0da - rustc_interface::passes::create_global_ctxt::{{closure}}::h14ca905422d068e3
  20:     0x560e42ec189a - rustc_interface::interface::run_compiler_in_existing_thread_pool::hbef0f1e9fa8ef3ff
  21:     0x560e42fbb0e7 - rustdoc::core::run_core::h557abb113278db14
  22:     0x560e42ee78f9 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h1c3737142ce4c9e6
  23:     0x560e42e49c14 - std::panicking::try::do_call::h3feb55ab80dbb052
  24:     0x7f8f9327645a - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:82
  25:     0x560e42eea0c3 - rustc_driver::report_ices_to_stderr_if_any::hb507e043ac4ab7f6
  26:     0x560e42dbc724 - rustdoc::main_options::h73fd32b381361a98
  27:     0x560e42e90045 - std::thread::local::LocalKey<T>::with::h7b42eea3105d8426
  28:     0x560e42ee9991 - scoped_tls::ScopedKey<T>::set::h8ad6cbf25bee106b
  29:     0x560e42ea4692 - syntax::with_globals::ha68beff4b747d351
  30:     0x560e42fa31a3 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc52e7f8752451ef6
  31:     0x7f8f9327645a - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:82
  32:     0x560e42d9afe9 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h8af64b4dfe60ea20
  33:     0x7f8f93248e1f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::hfb7da7c10e37eb96
                               at /rustc/0beb2ba16a08dfa01569b5f4644da315dc4c806c/src/liballoc/boxed.rs:746
  34:     0x7f8f93275140 - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::hfa4f87915138ff10
                               at /rustc/0beb2ba16a08dfa01569b5f4644da315dc4c806c/src/liballoc/boxed.rs:746
  35:     0x7f8f93275140 - std::sys_common::thread::start_thread::h15a2ed152517f722
                               at src/libstd/sys_common/thread.rs:13
  36:     0x7f8f93275140 - std::sys::unix::thread::Thread::new::thread_start::h006cc43581e90d8f
                               at src/libstd/sys/unix/thread.rs:79
  37:     0x7f8f931ab182 - start_thread
  38:     0x7f8f930bab1f - __clone
  39:                0x0 - <unknown>

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (0beb2ba16 2019-07-02) running on x86_64-unknown-linux-gnu

[hellow554]

This is an issue in the pulldown-cmark parser and I think it has already been fixed in pulldown-cmark/pulldown-cmark@238d827, pulldown-cmark/pulldown-cmark@b931bb4 and pulldown-cmark/pulldown-cmark@a4e41df (see pulldown-cmark/pulldown-cmark#306)

@raphlinus are you planning to release a version 0.5.3 soon so rustdoc can use that to fix this issue?

[hellow554]

@Centril Are there any plans of introducing a new tag, a la S-waiting-for-upstream? This could be useful for LLVM issues and this one in particular? What do you think? Can you bring in that idea to the next Rust meeting?

[raphlinus]

We can release an 0.5.3 soon. The other thing I probably want to get in is pulldown-cmark/pulldown-cmark#356 (just the main emphasis, not the table stuff), but we don't have a patch for that yet. I'll see what I can do.

[Centril]

@Centril Are there any plans of introducing a new tag, a la S-waiting-for-upstream? This could be useful for LLVM issues and this one in particular? What do you think? Can you bring in that idea to the next Rust meeting?

Added to T-release agenda.

[GuillaumeGomez]

@raphlinus Please ping us once the release is done.

[pietroalbini]

Are there any plans of introducing a new tag, a la S-waiting-for-upstream? This could be useful for LLVM issues and this one in particular? What do you think? Can you bring in that idea to the next Rust meeting?

We discussed this in today's release team meeting, and we decided not to add the label. There isn't really a difference in the way we would triage S-blocked and S-waiting-on-upstream PRs, so splitting the label wouldn't help the triage efforts that much. Thanks for the proposal though!

[raphlinus]

We just released 0.5.3 and it contains this fix among others.

[GuillaumeGomez]

Thanks @raphlinus! Sending the update PR right away.