rustc: Unbounded memory growth when missing `use`

[Jake-Shadle]

It seems that there is a problem related to missing use statements that occurs once a dependent library becomes large enough, or possibly it has enough transitive dependencies, but this is just conjecture as I was not able to find a repro for this issue on a smaller scale.

I have created a single repo that includes all of the code required to repro this issue, but basically the short of the problem is encapsulated in a very small amount of code.

extern crate dbobject;

// Uncommenting this line to import the type will compile successfully as it should
//use dbobject::types::Sha1;

struct StructWithTypeInsideDbObject {
    key: Sha1,
}

fn main() {
    println!("Hello, world!");
}

I expected this to happen:

src\main.rs:4:7: 4:11 error: type name `Sha1` is undefined or not in scope [E0412]
src\main.rs:4   key: Sha1
                     ^~~~
src\main.rs:4:7: 4:11 help: run `rustc --explain E0412` to see a detailed explanation
src\main.rs:4:7: 4:11 help: you can import several candidates into scope (`use ...;`):
src\main.rs:4:7: 4:11 help:   `dbobject::types::Sha1`

Instead, this happened:
rustc consumes ever increasing amounts of memory while maxing out a CPU core, until it is eventually killed by the OS.

Meta

rustc --version --verbose:

rustc 1.9.0 (e4e8b6668 2016-05-18)
binary: rustc
commit-hash: e4e8b666850a763fdf1c3c2c142856ab51e32779
commit-date: 2016-05-18
host: x86_64-pc-windows-msvc
release: 1.9.0

rustc 1.9.0 (e4e8b6668 2016-05-18)
binary: rustc
commit-hash: e4e8b666850a763fdf1c3c2c142856ab51e32779
commit-date: 2016-05-18
host: x86_64-unknown-linux-gnu
release: 1.9.0

More notes:

• I have tested this on Ubuntu x86_64, Windows x86_64 (gnu and msvc), on 1.8, 1.9, and 1.11 nightly (from a ~week ago)
• The crate type with the missing use can be either lib or bin
• One assumption I had was that the problem was due to name resolution looking in transitive dependencies, in this example, there is a Sha1 type in both dbobject as well as rust-crypto, but using a type that only exists in dbobject such as Null still exhibits the same problem.

Backtrace (captured at a random time as the compiler was running):

#0  0x00007f7cf98cd1cd in collections::slice::hack::to_vec::h1c0cc52b8aed0cc3
    ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_resolve-d16b8f0e.so
#1  0x00007f7cf98bd768 in rustc_resolve::Resolver::resolve_type::hefe358d1579d4388 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_resolve-d16b8f0e.so
#2  0x00007f7cf98cc905 in rustc::hir::intravisit::walk_item::h4d0fd788646d76e7
    ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_resolve-d16b8f0e.so
#3  0x00007f7cf98b79e7 in _$LT$Resolver$LT$$u27$a$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..hir..intravisit..Visitor$LT$$u27$v$GT$$GT$::visit_item::h3eb5e028bdb089d8 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_resolve-d16b8f0e.so
#4  0x00007f7cf98e39eb in rustc_resolve::resolve_crate::h8dee150c90a64334 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_resolve-d16b8f0e.so
#5  0x00007f7cfe7b15d3 in rustc_driver::driver::phase_3_run_analysis_passes::h83da042ec4b8ea10 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/lib---Type <return> to continue, or q <return> to quit---
rustc_driver-d16b8f0e.so
#6  0x00007f7cfe785fa0 in rustc_driver::driver::compile_input::h6491aaddd9e61258 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d16b8f0e.so
#7  0x00007f7cfe76c4e5 in rustc_driver::run_compiler::h80b2ba5e4d787c5f ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d16b8f0e.so
#8  0x00007f7cfe769942 in std::sys_common::unwind::try::try_fn::h34e27823ddd1d5e9 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d16b8f0e.so
#9  0x00007f7cfe264d0c in __rust_try ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
#10 0x00007f7cfe264c9e in std::sys_common::unwind::inner_try::h9eebd8dc83f388a6
    ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
#11 0x00007f7cfe76a18b in _$LT$F$u20$as$u20$std..boxed..FnBox$LT$A$GT$$GT$::call_box::h3d5d78986dfac5b2 ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/librustc_driver-d16b8f0e.so
---Type <return> to continue, or q <return> to quit---
#12 0x00007f7cfe272e05 in std::sys::thread::Thread::new::thread_start::h471ad90789353b5b ()
   from /home/jake/.multirust/toolchains/stable-x86_64-unknown-linux-gnu/lib/libstd-d16b8f0e.so
#13 0x00007f7cf6c316fa in start_thread (arg=0x7f7cf4fff700)
    at pthread_create.c:333
#14 0x00007f7cfdec8b5d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

[jseyfried]

cc me

[Jake-Shadle]

FWIW, I have also tried building with MIR enabled (-Zorbit) on the latest nightly with the same results. 😦

nightly-x86_64-pc-windows-msvc
rustc 1.11.0-nightly (ad7fe6521 2016-06-23)

[jonas-schievink]

@Jake-Shadle The bug happens in resolve, long before MIR is constructed

[Jake-Shadle]

This is still an issue on the latest nightly build.

nightly-x86_64-pc-windows-msvc (directory override for '\\?\D:\git\rust_repro')
rustc 1.15.0-nightly (71c06a56a 2016-12-18)

[jseyfried]

@Jake-Shadle sorry for the delay!
Fixed in #38539.

[Jake-Shadle]

Oh awesome, I guess this just fell through the cracks! 😄