rustc is still failing with "index out of bounds" when incremental is on

[imp]

It looks quite similar to #48847, which was fixed some time ago.
Nonetheless, I am still hitting this on daily basis (and I am on fresh nightly on daily basis too).
Re-running with CARGO_INCREMENTAL=0 fixes the problem. It kinda feels like it has something to do with the fact that one of the dependencies I have uses #[proc_macro_attribute]. However, I don't have strong evidence to support this.

So, unless the fix in #49364 didn't make it into nightly the problem is still here or perhaps is triggered by something else.

$ RUST_BACKTRACE=full cargo test
   Compiling rexlog v0.3.0 (file:///../rexlog)
thread 'rustc' panicked at 'index out of bounds: the len is 346134 but the index is 347494', /Users/travis/build/rust-lang/rust/src/libcore/slice/mod.rs:871:14
stack backtrace:
thread 'rustc' panicked at 'index out of bounds: the len is 323469 but the index is 324710', /Users/travis/build/rust-lang/rust/src/libcore/slice/mod.rs:871:14
stack backtrace:
      00: :               0x0x10f0be4ab10c6704ab -  - stdstd::::syssys::::unixunix::::backtracebacktrace::::tracingtracing::::impimp::::unwind_backtraceunwind_backtrace::::h366610c5a5c43907h366610c5a5c43907

   1:          1 : 0x 10f09439b  -   std :: sys_common ::0xbacktrace10c64639b:: - print::stdh7a317efe25c5e753::
sys_common::backtrace::print::h7a317efe25c5e753
      22: :               0x0x10c660e2d10f0aee2d -  - stdstd::::panickingpanicking::::default_hookdefault_hook::::{{{{closureclosure}}}}::::h0ceb10b0333c30c9h0ceb10b0333c30c9

     3:     3 :       0x 10c660b8a  -   0x10f0aeb8a - std::panicking::default_hook::h52f1cafdb80838c7
std::panicking::default_hook::h52f1cafdb80838c7
      44: :               0x0x10b2043e010d7bc3e0 -  - corecore::::opsops::::functionfunction::::FnFn::::callcall::::h9b97d3244299c7efh9b97d3244299c7ef

   5  :   5 :           0x 10f0af2e0  - 0x10c6612e0std - ::panickingstd::::rust_panic_with_hookpanicking::rust_panic_with_hook::hb44d67051d9d88b2
::hb44d67051d9d88b2
   6:       6 :       0x 10c6610da  -   std0x::10f0af0dapanicking - ::begin_panic_fmtstd::::h1bc250036d2f0fcdpanicking
::begin_panic_fmt::h1bc250036d2f0fcd
   7:       7 :       0x 10c660fd2  -  rust_begin_unwind
0x10f0aefd2 - rust_begin_unwind
      88: :               0x0x10f11dd4510c6cfd45 -  - corecore::::panickingpanicking::::panic_fmtpanic_fmt::::h2e51865b8b30844fh2e51865b8b30844f

      99: :               0x0x10f11dcc310c6cfcc3 -  - corecore::::panickingpanicking::::panic_bounds_checkpanic_bounds_check::::h27f516c666ce87c3h27f516c666ce87c3

    1010: :               0x0x10ae06a5a10d3bea5a -  - rustcrustc::::dep_graphdep_graph::::graphgraph::::DepGraphDepGraph::::with_task_implwith_task_impl::::h44ec831076965f2dh44ec831076965f2d

  11  : 11 :             0x 10d42a4cf0x - 10ae724cf - rustc_errors::rustc_errorsHandler::::Handlertrack_diagnostics::::track_diagnosticsh1b6f9e82a9a76210::
h1b6f9e82a9a76210
    1212: :               0x0x10b0a9c5010d661c50 -  - rustcrustc::::tyty::::mapsmaps::::plumbingplumbing::::<<implimpl  rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt<<''aa,,  ''gcxgcx,,  ''tcxtcx>>>>::::cycle_checkcycle_check::::hbd8198d763eb22d5hbd8198d763eb22d5

    1313: :               0x0x10d74602510b18e025 -  - rustcrustc::::tyty::::mapsmaps::::<<implimpl  rustcrustc::::tyty::::mapsmaps::::queriesqueries::::crate_disambiguatorcrate_disambiguator<<''tcxtcx>>>>::::forceforce::::h03279f064836e36ch03279f064836e36c

  14:        0x10b18ea37 - rustc::ty::maps::<impl rustc::ty::maps::queries::crate_disambiguator<  'tcx14>: > :: try_get :: h200355d461c7bd90
  0x10d746a37 - rustc::ty::maps::<impl rustc::ty::maps::queries::crate_disambiguator<'tcx>>::try_get::h200355d461c7bd90
    1515: :               0x0x10d6a267b10b0ea67b -  - rustcrustc::::tyty::::mapsmaps::::TyCtxtAtTyCtxtAt::::crate_disambiguatorcrate_disambiguator::::h96062a13ebb040ach96062a13ebb040ac

    1616: :               0x0x10afcd5f710d5855f7 -  - corecore::::opsops::::functionfunction::::implsimpls::::<<implimpl  corecore::::opsops::::functionfunction::::FnOnceFnOnce<<AA>>  forfor  &&''aa  mutmut  FF>>::::call_oncecall_once::::h70d98388dbf3d01ch70d98388dbf3d01c

    1717: :               0x0x10d2d75a010ad1f5a0 -  - <<stdstd::::collectionscollections::::hashhash::::mapmap::::HashMapHashMap<<KK,,  VV,,  SS>>  asas  corecore::::iteriter::::traitstraits::::FromIteratorFromIterator<<((KK,,  VV))>>>>::::from_iterfrom_iter::::h466507e1c9931fcah466507e1c9931fca

  18:          180x: 10adef672  -   rustc :: dep_graph :: graph0x::10d3a7672DepGraph - ::with_ignorerustc::::h227baac199d5fd46dep_graph
::graph::DepGraph::with_ignore::h227baac199d5fd46
    1919: :               0x0x10affdf8d10d5b5f8d -  - rustcrustc::::tyty::::mapsmaps::::on_disk_cacheon_disk_cache::::OnDiskCacheOnDiskCache::::load_diagnosticsload_diagnostics::::he2d592279e7a62b0he2d592279e7a62b0

  20  : 20 :             0x 10d3ff5580x - 10ae47558 - rustcrustc::::dep_graphdep_graph::::graphgraph::::DepGraphDepGraph::::try_mark_greentry_mark_green::::heaf9c57c89e0da8dheaf9c57c89e0da8d

    2121: :               0x0x10b0c0c4c10d678c4c -  - rustcrustc::::tyty::::mapsmaps::::plumbingplumbing::::<<implimpl  rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt<<''aa,,  ''gcxgcx,,  ''tcxtcx>>>>::::try_mark_green_and_readtry_mark_green_and_read::::hea34e4d80140e3fahea34e4d80140e3fa

    2222: :               0x0x10b155c8c10d70dc8c -  - rustcrustc::::tyty::::mapsmaps::::<<implimpl  rustcrustc::::tyty::::mapsmaps::::queriesqueries::::trans_fn_attrstrans_fn_attrs<<''tcxtcx>>>>::::try_gettry_get::::h5a03b2357edcf5eeh5a03b2357edcf5ee

    2323: :               0x0x10b0e7e2110d69fe21 -  - rustcrustc::::tyty::::mapsmaps::::TyCtxtAtTyCtxtAt::::trans_fn_attrstrans_fn_attrs::::h583266aa8af834c8h583266aa8af834c8

    2424: :               0x0x10d40200810ae4a008 -  - <<rustcrustc::::hirhir::::check_attrcheck_attr::::CheckAttrVisitorCheckAttrVisitor<<''aa,,  ''tcxtcx>>  asas  rustcrustc::::hirhir::::intravisitintravisit::::VisitorVisitor<<''tcxtcx>>>>::::visit_itemvisit_item::::he74dfb870b548c09he74dfb870b548c09

    2525: :               0x0x10d9aab6a10b3f2b6a -  - rustcrustc::::hirhir::::CrateCrate::::visit_all_item_likesvisit_all_item_likes::::h3b674bd40b178a04h3b674bd40b178a04

  26:      26 :       0x 10d4033d7  -   rustc0x::10ae4b3d7hir - ::rustccheck_attr::::hircheck_crate::::check_attrh51e4802d29e36c00::
check_crate::h51e4802d29e36c00
    2727: :               0x0x108fd7c0810bb0ec08 -  - <<stdstd::::threadthread::::locallocal::::LocalKeyLocalKey<<TT>>>>::::withwith::::hdc24675849a08d33hdc24675849a08d33

  28  : 28 :             0x 108fd63fc0x - 10bb0d3fc< - std<::stdthread::::threadlocal::::localLocalKey::<LocalKeyT<>T>>::>with::::withh202d3d26a302feda::
h202d3d26a302feda
    2929: :               0x0x10bb7b5e91090445e9 -  - rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt::::create_and_entercreate_and_enter::::hb3bf8df16c466565hb3bf8df16c466565

    3030: :               0x0x10ba9f2c9108f682c9 -  - rustc_driverrustc_driver::::driverdriver::::compile_inputcompile_input::::h6fef5ec4253b03bdh6fef5ec4253b03bd

    3131: :               0x0x108ffcf8510bb33f85 -  - rustc_driverrustc_driver::::run_compiler_implrun_compiler_impl::::h93424a4b8ff05c37h93424a4b8ff05c37

    32: 32 :           0x 10ba7a60b  -  syntax0x::with_globals108f4360b:: - h692960019527b54c
syntax::with_globals::h692960019527b54c
    33: 33 :           0x 10bab967f  -  std0x::108f8267fsys_common:: - backtrace::std__rust_begin_short_backtrace::::sys_commonhae82898bcf3d911a::
backtrace::__rust_begin_short_backtrace::hae82898bcf3d911a
  34:        0x10f0d3b1e -   __rust_maybe_catch_panic34
:        0x10c685b1e - __rust_maybe_catch_panic
  35:        35 :   0x 10bac57ff  -     <0xF108f8e7ff  - as <allocF:: boxedas:: FnBoxalloc<::Aboxed>::>FnBox::<call_boxA::>hb601af265dd036e7>
::call_box::hb601af265dd036e7
    3636: :               0x0x10f0c6d9710c678d97 -  - stdstd::::sys_commonsys_common::::threadthread::::start_threadstart_thread::::h9a50dc631317cc98h9a50dc631317cc98

    3737: :               0x0x10c64512810f093128 -  - stdstd::::syssys::::unixunix::::threadthread::::ThreadThread::::newnew::::thread_startthread_start::::ha148513b715b08e3ha148513b715b08e3

    3838: :         0x0x7fff5d6246c07fff5d6246c0 -  - _pthread_body_pthread_body

    3939: :         0x0x7fff5d62456c7fff5d62456c -  - _pthread_start_pthread_start


error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.26.0-nightly (06fa27d7c 2018-04-01) running on x86_64-apple-darwin

note: compiler flags: -C debuginfo=2 -C incremental --crate-type lib

note: some of the compiler flags provided by cargo are hidden


error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.26.0-nightly (06fa27d7c 2018-04-01) running on x86_64-apple-darwin

note: compiler flags: -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `rexlog`.
warning: build failed, waiting for other jobs to finish...
error: Could not compile `rexlog`.

To learn more, run the command again with --verbose.

[wesleywiser]

cc @rust-lang/wg-compiler-performance as this is related to incremental compilation

[wesleywiser]

The stack trace appears garbled for some reason. Here's the relevant parts:

99: :               0x0x10f11dcc310c6cfcc3 -  - corecore::::panickingpanicking::::panic_bounds_checkpanic_bounds_check::::h27f516c666ce87c3h27f516c666ce87c3

1010: :               0x0x10ae06a5a10d3bea5a -  - rustcrustc::::dep_graphdep_graph::::graphgraph::::DepGraphDepGraph::::with_task_implwith_task_impl::::h44ec831076965f2dh44ec831076965f2d

11  : 11 :             0x 10d42a4cf0x - 10ae724cf - rustc_errors::rustc_errorsHandler::::Handlertrack_diagnostics::::track_diagnosticsh1b6f9e82a9a76210::
h1b6f9e82a9a76210
1212: :               0x0x10b0a9c5010d661c50 -  - rustcrustc::::tyty::::mapsmaps::::plumbingplumbing::::<<implimpl  rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt<<''aa,,  ''gcxgcx,,  ''tcxtcx>>>>::::cycle_checkcycle_check::::hbd8198d763eb22d5hbd8198d763eb22d5

1313: :               0x0x10d74602510b18e025 -  - rustcrustc::::tyty::::mapsmaps::::<<implimpl  rustcrustc::::tyty::::mapsmaps::::queriesqueries::::crate_disambiguatorcrate_disambiguator<<''tcxtcx>>>>::::forceforce::::h03279f064836e36ch03279f064836e36c

14:        0x10b18ea37 - rustc::ty::maps::<impl rustc::ty::maps::queries::crate_disambiguator<  'tcx14>: > :: try_get :: h200355d461c7bd90
  0x10d746a37 - rustc::ty::maps::<impl rustc::ty::maps::queries::crate_disambiguator<'tcx>>::try_get::h200355d461c7bd90
1515: :               0x0x10d6a267b10b0ea67b -  - rustcrustc::::tyty::::mapsmaps::::TyCtxtAtTyCtxtAt::::crate_disambiguatorcrate_disambiguator::::h96062a13ebb040ach96062a13ebb040ac

1616: :               0x0x10afcd5f710d5855f7 -  - corecore::::opsops::::functionfunction::::implsimpls::::<<implimpl  corecore::::opsops::::functionfunction::::FnOnceFnOnce<<AA>>  forfor  &&''aa  mutmut  FF>>::::call_oncecall_once::::h70d98388dbf3d01ch70d98388dbf3d01c

1717: :               0x0x10d2d75a010ad1f5a0 -  - <<stdstd::::collectionscollections::::hashhash::::mapmap::::HashMapHashMap<<KK,,  VV,,  SS>>  asas  corecore::::iteriter::::traitstraits::::FromIteratorFromIterator<<((KK,,  VV))>>>>::::from_iterfrom_iter::::h466507e1c9931fcah466507e1c9931fca

18:          180x: 10adef672  -   rustc :: dep_graph :: graph0x::10d3a7672DepGraph - ::with_ignorerustc::::h227baac199d5fd46dep_graph
::graph::DepGraph::with_ignore::h227baac199d5fd46
1919: :               0x0x10affdf8d10d5b5f8d -  - rustcrustc::::tyty::::mapsmaps::::on_disk_cacheon_disk_cache::::OnDiskCacheOnDiskCache::::load_diagnosticsload_diagnostics::::he2d592279e7a62b0he2d592279e7a62b0

20  : 20 :             0x 10d3ff5580x - 10ae47558 - rustcrustc::::dep_graphdep_graph::::graphgraph::::DepGraphDepGraph::::try_mark_greentry_mark_green::::heaf9c57c89e0da8dheaf9c57c89e0da8d

2121: :               0x0x10b0c0c4c10d678c4c -  - rustcrustc::::tyty::::mapsmaps::::plumbingplumbing::::<<implimpl  rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt<<''aa,,  ''gcxgcx,,  ''tcxtcx>>>>::::try_mark_green_and_readtry_mark_green_and_read::::hea34e4d80140e3fahea34e4d80140e3fa

2222: :               0x0x10b155c8c10d70dc8c -  - rustcrustc::::tyty::::mapsmaps::::<<implimpl  rustcrustc::::tyty::::mapsmaps::::queriesqueries::::trans_fn_attrstrans_fn_attrs<<''tcxtcx>>>>::::try_gettry_get::::h5a03b2357edcf5eeh5a03b2357edcf5ee

2323: :               0x0x10b0e7e2110d69fe21 -  - rustcrustc::::tyty::::mapsmaps::::TyCtxtAtTyCtxtAt::::trans_fn_attrstrans_fn_attrs::::h583266aa8af834c8h583266aa8af834c8

2424: :               0x0x10d40200810ae4a008 -  - <<rustcrustc::::hirhir::::check_attrcheck_attr::::CheckAttrVisitorCheckAttrVisitor<<''aa,,  ''tcxtcx>>  asas  rustcrustc::::hirhir::::intravisitintravisit::::VisitorVisitor<<''tcxtcx>>>>::::visit_itemvisit_item::::he74dfb870b548c09he74dfb870b548c09

2525: :               0x0x10d9aab6a10b3f2b6a -  - rustcrustc::::hirhir::::CrateCrate::::visit_all_item_likesvisit_all_item_likes::::h3b674bd40b178a04h3b674bd40b178a04

26:      26 :       0x 10d4033d7  -   rustc0x::10ae4b3d7hir - ::rustccheck_attr::::hircheck_crate::::check_attrh51e4802d29e36c00::
check_crate::h51e4802d29e36c00
2727: :               0x0x108fd7c0810bb0ec08 -  - <<stdstd::::threadthread::::locallocal::::LocalKeyLocalKey<<TT>>>>::::withwith::::hdc24675849a08d33hdc24675849a08d33

28  : 28 :             0x 108fd63fc0x - 10bb0d3fc< - std<::stdthread::::threadlocal::::localLocalKey::<LocalKeyT<>T>>::>with::::withh202d3d26a302feda::
h202d3d26a302feda
2929: :               0x0x10bb7b5e91090445e9 -  - rustcrustc::::tyty::::contextcontext::::TyCtxtTyCtxt::::create_and_entercreate_and_enter::::hb3bf8df16c466565hb3bf8df16c466565

3030: :               0x0x10ba9f2c9108f682c9 -  - rustc_driverrustc_driver::::driverdriver::::compile_inputcompile_input::::h6fef5ec4253b03bdh6fef5ec4253b03bd

[wesleywiser]

The crash isn't happening during incremental result deserialization so the fix in #49364 doesn't have any effect.

[ExpHP]

Might instead be #48923 (report) / #49300 (meta-issue for underlying bug)

[michaelwoerister]

This this still reproduce with a current nightly?

[SimonSapin]

I’ve seen this happen a couple times this week while working on std after rebasing a branch on top of master. ./x.py clean fixes it, but takes some time.

[michaelwoerister]

@SimonSapin, was that with a version that already contains #49695?

[SimonSapin]

merged […] a day ago

Probably not. Or at least the pre-rebase branches probably did not. Or possibly it only happened specifically when going from without to with this change?

[michaelwoerister]

Is this still an issue on nightly?

[imp]

Not anymore on the code base that was the source of this ticket.