Bootstrap build is failing on Mac OS X with "rust: task ... ran out of stack"

[pnkfelix]

Fresh clone of repo.
commit 1d53bab

% configure --disable-debug --disable-optimize  --enable-clang
% time make -j8
...
compile_and_link: x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/lib/libcore.dylib
rust: task 7fa031d00000 ran out of stack
compile_and_link: x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/lib/libstd.dylib
cp: x86_64-apple-darwin/stage2/lib/libcore.dylib
cp: x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/lib/libcore-*.dylib: No such file or directory
make: *** [x86_64-apple-darwin/stage2/lib/libcore.dylib] Error 1
make: *** Waiting for unfinished jobs....
rust: task 7fa058c0ad10 ran out of stack

real    4m54.362s
user    22m36.394s
sys 1m4.580s

This may be a duplicate of another issue, for example perhaps #6049. I'm honestly not sure.

At this point I have bisected the problem down to:

4a24f10 is the first bad commit
commit 4a24f10
Author: Huon Wilson dbau.pp+github@gmail.com
Date: Wed Apr 24 22:29:19 2013 +1000

(But it is possible that Huon's code is just exposing a latent bug in rustc, rather than actually having something fundamentally wrong with it itself.) I plan to investigate backing out Huon's change to see if that makes the problem go away, just so I can actually bootstrap a compiler and do more investigation.

[pnkfelix]

A theory from #rust IRC channel:

cuddling: @pnkfelix: regarding #6049/#6061 ... rustsqlite uses HashMap, so maybe gen() has a problem

[huonw]

Does RUST_LOG=::rt::backtrace make indicate where the infinite recursion is? (Assuming it is actually that.)

(Also, sorry for breaking everything :( )

[huonw]

And, does giving type hints, i.e. r.gen::<u64>() in hashmap, fix it? (I don't have an osx computer to experiment with.)

[catamorphism]

I'm checking on this.

[brson]

@pcwalton's recent ffi changes could have significantly changed the way stack accounting works. You could try running with RUST_MAX_STACK=20000000 (20MB, the default is 8MB) and see if that gives you enough overhead.

[brson]

More specifically, C stacks were never accounted for before when deciding whether there was enough stack. I assume that now when we use the fast_ffi stacks, that is attributing an additional 2MB of stack to the task.

[huonw]

Would #[fixed_stack_segment] on the rustrt functions in rand.rs help this?

(I'm still not quite sure what it does, but it's helped with stack troubles before.)

[huonw]

@catamorphism (or @brson, or anyone else who can reproduce this) I just opened #6073, which reduces the number of C calls that the rand module does (they are then only necessary for generating the initial seed), which may fix this if it that is the problem. I can't test it so I'd appreciate if someone could throw some spare CPU cycles at it.

[pnkfelix]

@huonw

% RUST_LOG=rustc=1,::rt::backtrace x86_64-apple-darwin/stage1/bin/rustc -Z verbose --cfg stage1    --target=x86_64-apple-darwin   -o x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/lib/libsyntax.dylib /tmp/rust/src/libsyntax/syntax.rc
rust: task 7f800b40ac80 ran out of stack

Maybe I am doing it wrong, or maybe our stack exhaustion handling code needs some hacking. Looking through the other suggestions.

[pnkfelix]

@huonw commit 6d50d55 does not fix the issue for me.

That is, after fetching that repository, rebasing that commit to my incoming (commit 1d53bab), cleaning my build tree (via: ( rm -rf objdir/* src/libuv src/llvm && git checkout src/libuv src/llvm ) to side-step the problems with submodules I have been encountering lately), and then doing a configure+make as I wrote at the top of this isue, the same problem persists:

% RUST_LOG=rustc=1,::rt::backtrace make -j1 VERBOSE=1
cfg: build triple x86_64-apple-darwin
cfg: host triples x86_64-apple-darwin
cfg: target triples x86_64-apple-darwin
cfg: disabling rustc optimization (CFG_DISABLE_OPTIMIZE)
cfg: host for x86_64-apple-darwin is x86_64
cfg: os for x86_64-apple-darwin is apple-darwin
cfg: using clang
x86_64-apple-darwin/stage1/bin/rustc --cfg stage1    --target=x86_64-apple-darwin  --cfg rustc -o x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc /Users/fklock/Dev/Mozilla/rust.git/src/driver/driver.rs
rust: task 7fca11d00000 ran out of stack
cp x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc x86_64-apple-darwin/stage2/bin/rustc
cp: x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc: No such file or directory
make: *** [x86_64-apple-darwin/stage2/bin/rustc] Error 1

@brson Increasing the stack size by various reasonable+unreasonable amounts does not fix the problem:

% time RUST_MAX_STACK=10000000 x86_64-apple-darwin/stage1/bin/rustc --cfg stage1    --target=x86_64-apple-darwin  --cfg rustc -o x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc /Users/fklock/Dev/Mozilla/rust.git/src/driver/driver.rs
rust: task 7fbb50d00000 ran out of stack

real    0m0.376s
user    0m0.364s
sys 0m0.011s
% time RUST_MAX_STACK=100000000 x86_64-apple-darwin/stage1/bin/rustc --cfg stage1    --target=x86_64-apple-darwin  --cfg rustc -o x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc /Users/fklock/Dev/Mozilla/rust.git/src/driver/driver.rs
rust: task 7f950ad00000 ran out of stack

real    0m3.734s
user    0m3.694s
sys 0m0.039s
% time RUST_MAX_STACK=1000000000 x86_64-apple-darwin/stage1/bin/rustc --cfg stage1    --target=x86_64-apple-darwin  --cfg rustc -o x86_64-apple-darwin/stage1/lib/rustc/x86_64-apple-darwin/bin/rustc /Users/fklock/Dev/Mozilla/rust.git/src/driver/driver.rs
rust: task 7fdde2d00000 ran out of stack

real    0m37.261s
user    0m36.967s
sys 0m0.295s

(I still haven't investigating what it is about the commits that causes the problem. I plan to do that now; I just wanted to make sure I addressed these comments first.)