rustc produces invalid IR with -O -g

[wthrowe]

When compiling rustc with "--enable-debug --enable-optimize" the stage0 compiler fails to build libsyntax with

!dbg attachment points at wrong subprogram for function
!157085 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !157069, file: !157069, line: 38, type: !42899, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1242*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69625E, templateParams: !1036, variables: !157086)
i64 (%closure.1242*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69625E
  tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.857, i64 0, metadata !157464, metadata !190803), !dbg !771025
!771025 = !DILocation(line: 113, scope: !157458, inlinedAt: !771026)
!771027 = distinct !DILexicalBlock(scope: !771028, file: !157069, line: 129, column: 56)
!160753 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !157069, file: !157069, line: 129, type: !160754, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1036, variables: !160756)
LLVM ERROR: Broken function found, compilation aborted!

This started with #28857, but it looks to me like that just happened to trigger a preexisting bug.

[pnkfelix]

I am able to reproduce this on my native linux system. I have not yet tried in a VM yet. Its a pretty bad show-stopper for Linux work (at least if one wants to use --enable-debug in a usable manner).

[dotdash]

JFYI, I started looking into this.

[pnkfelix]

(this problem does not reproduce on Mac OS X, which may be fortunate or unfortunate, depending on one's POV...)

Update: Now I have reproduced it on OS X. I am not sure where my earlier attempt to reproduce went wrong. Anyway, this heightens my desire to get this fixed considerably...

cc @nrc

[pnkfelix]

@dotdash have you made much progress, e.g. have you made a reduced test case, or have insight into what's wrong?

I am wondering in part because I am willing to spend time looking at this, but I do not want to waste or duplicate effort.

[dotdash]

Not much progress yet. For some reason, the inlinedAt chain for the
dbg.value calls is missing. AFAICT so far, our debug info looks fine (I
might be missing something though), so I'm looking at the LLVM code to see
how inlining handles debug info and how the conversion from dbg.declare
to dbg.value works.

2015-10-13 12:01 GMT+02:00 Felix S Klock II notifications@github.com:

@dotdash https://github.com/dotdash have you made much progress, e.g.
have you made a reduced test case, or have insight into what's wrong?

I am wondering in part because I am willing to spend time looking at this,
but I do not want to waste or duplicate effort.

—
Reply to this email directly or view it on GitHub
#28947 (comment).

[sanxiyn]

Something weird is going on. -verify-each causes the error to disappear.

$ rustc -g --emit llvm-bc src/libsyntax/lib.rs
$ opt -O2 -disable-output syntax.bc
LLVM ERROR: Broken function found, compilation aborted!
$ opt -O2 -verify-each -disable-output syntax.bc

Reproduced on a hardened gentoo linux (here's all you need to know about the system: http://dpaste.com/1DQ0EFB.txt ) as follows:

$ ./configure --prefix=/home/zazdxscf/build/1nonpkgs/rust/usr/local --disable-rpath --enable-manage-submodules --disable-clang --enable-ccache --enable-dist-host-only --disable-valgrind --disable-helgrind --disable-valgrind-rpass --python=/usr/bin/python2 --enable-optimize --enable-optimize-cxx --enable-optimize-llvm --enable-debug --enable-debuginfo --enable-debug-assertions --enable-debuginfo-tests --enable-llvm-assertions --enable-debug-jemalloc --disable-local-rust
...
$ time ( time make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 RUSTFLAGS="--verbose" && time make check )
cfg: version 1.5.0-dev (294ef5b15 2015-10-14)
cfg: build triple x86_64-unknown-linux-gnu
cfg: host triples x86_64-unknown-linux-gnu
cfg: target triples x86_64-unknown-linux-gnu
cfg: enabling debug assertions (CFG_ENABLE_DEBUG_ASSERTIONS)
cfg: enabling debuginfo (CFG_ENABLE_DEBUGINFO)
cfg: host for x86_64-unknown-linux-gnu is x86_64
cfg: os for x86_64-unknown-linux-gnu is unknown-linux-gnu
cfg: good valgrind for x86_64-unknown-linux-gnu is 1
cfg: using CC=ccache gcc (CFG_CC)
cfg: disabling valgrind run-pass tests
rustc: x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax
time: 0.381; rss: 58MB  parsing
time: 0.124; rss: 59MB  configuration 1
...
...
time: 25.515; rss: 929MB    translation
  time: 9.382; rss: 631MB   llvm function passes
  time: 202.128; rss: 738MB llvm module passes
!dbg attachment points at wrong subprogram for function
!156164 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156148, file: !156148, line: 38, type: !42689, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69460E, templateParams: !1036, variables: !156165)
i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69460E
  tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.857, i64 0, metadata !156543, metadata !189756), !dbg !766304
!766304 = !DILocation(line: 113, scope: !156537, inlinedAt: !766305)
!766306 = distinct !DILexicalBlock(scope: !766307, file: !156148, line: 129, column: 56)
!159806 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156148, file: !156148, line: 129, type: !159807, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1036, variables: !159809)
LLVM ERROR: Broken function found, compilation aborted!
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax] Error 1

rust commit tested: 294ef5b

In trying to follow sanxiyn's comment above, changing the above RUSTFLAGS to:
RUSTFLAGS="--verbose -C llvm-args=-verify-each"
fails after a while(I'm probably doing it wrong):

...
time: 0.820; rss: 327MB lint checking
time: 0.000; rss: 327MB resolving dependency formats
rustc: Unknown command line argument '-verify-each'.  Try: 'rustc -help'
rustc: Did you mean '-verify-scev'?
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax] Error 1

No idea where to stick that -verify-each at the moment, so that make would work with it. Any ideas?

EDIT1: It actually went past it with -verify-scev I'm guessing due to llvm version that rust compilation uses, instead of my installed version(3.7.0) which does have the -verify-each
But the original reported issue still happens (with -verify-scev): !dbg attachment points at wrong subprogram for function

EDIT2: I'm still getting the error that maybe I meant to use -verify-scev when trying with my own llvm and my own (local)rust, after a make clean and removal of directory rust/x86_64-unknown-linux-gnu/ then a configure then make

$ make clean
...
$ rm -rf x86_64-unknown-linux-gnu
...
$ ./configure --prefix=/home/zazdxscf/build/1nonpkgs/rust/usr/local --disable-rpath --enable-manage-submodules --disable-clang --enable-ccache --enable-dist-host-only --disable-valgrind --disable-helgrind --disable-valgrind-rpass --python=/usr/bin/python2 --enable-optimize --enable-optimize-cxx --enable-optimize-llvm --enable-debug --enable-debuginfo --enable-debug-assertions --enable-debuginfo-tests --enable-llvm-assertions --enable-debug-jemalloc --disable-local-rust --enable-local-rust --local-rust-root=/usr --llvm-root=/usr
...
$ make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 'RUSTFLAGS=--verbose -C llvm-args=-verify-each'
...
time: 0.073; rss: 172MB stability checking
time: 0.000; rss: 172MB unused lib feature checking
src/libcore/lib.rs:82:12: 82:29 warning: unused or unknown feature, #[warn(unused_features)] on by default
src/libcore/lib.rs:82 #![feature(unwind_attributes)]
                                 ^~~~~~~~~~~~~~~~~
src/libcore/panicking.rs:65:9: 65:18 warning: unused attribute, #[warn(unused_attributes)] on by default
src/libcore/panicking.rs:65         #[unwind]
                                    ^~~~~~~~~
time: 1.496; rss: 172MB lint checking
time: 0.000; rss: 172MB resolving dependency formats
rustc: Unknown command line argument '-verify-each'.  Try: 'rustc -help'
rustc: Did you mean '-verify-scev'?
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.core' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.core] Error 1

Does rustc maybe have a whitelisted llvm-args that's allowing to passthru ? Or how does it fail to use my llvm(3.7.0) which clearly does have that option ?
EDIT3: One more thing, I accidentally had --disable-local-rust with --enable-local-rust of which only the former took place but when I removed it:

time: 0.006; rss: 49MB  gated macro checking
time: 0.000; rss: 49MB  crate injection
src/libcore/lib.rs:1:1: 1:1 error: can't find crate for `std`
src/libcore/lib.rs:1 // Copyright 2014 The Rust Project Developers. See the COPYRIGHT
                     ^
error: aborting due to previous error
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.core' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.core] Error 101

but maybe I got the --local-rust-root="/usr" wrong, I've no idea which is right since it fails the same with /usr/local . I'll stick with supplied rustc then.

EDIT4: ok, the reason -verify-each fails is because it's passed to llc instead of to opt (commands) and for some reason llc doesn't have it but opt does.

EDIT5: tested with git llvm (3.8.0) and the error still happens:

$ make -j1 -- all NO_REBUILD=1 TIME_PASSES=1 TIME_LLVM_PASSES=1 'RUSTFLAGS=--verbose -Z verbose -Z print-link-args -Z debug-llvm -Z          unstable-options -C llvm-args="-verify-debug-info -verify-dom-info -verify-loop-info -verify-regalloc -verify-region-info -verify-scev"'
...
rustc: x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libsyntax
...
time: 25.511; rss: 928MB    translation
  time: 9.280; rss: 631MB   llvm function passes
  time: 200.468; rss: 740MB llvm module passes
!dbg attachment points at wrong subprogram for function
!156127 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156111, file: !156111, line: 38, type: !42675, isLocal: true, isDefinition: true, scopeLine: 38, flags: DIFlagPrototyped, isOptimized: true, function: i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69395E, templateParams: !1032, variables: !156128)
i64 (%closure.1236*, %"ast::Expr"*)* @_ZN3ext6expand11expand_expr13closure.69395E
  tail call void @llvm.dbg.value(metadata %"ext::expand::IdentRenamer"* %rename_fld.i.857, i64 0, metadata !156506, metadata !189686), !dbg !765837
!765837 = !DILocation(line: 113, scope: !156500, inlinedAt: !765838)
!765839 = distinct !DILexicalBlock(scope: !765840, file: !156111, line: 129, column: 56)
!159769 = !DISubprogram(name: "fnfn", linkageName: "fnfn", scope: !156111, file: !156111, line: 129, type: !159770, isLocal: true, isDefinition: true, scopeLine: 129, flags: DIFlagPrototyped, isOptimized: true, templateParams: !1032, variables: !159772)
LLVM ERROR: Broken function found, compilation aborted!
/home/zazdxscf/build/1nonpkgs/rust/rust/mk/target.mk:164: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax' failed
make: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.syntax] Error 1

real    16m51.405s
user    15m38.200s
sys 0m49.577s

latest rust commit tested 1ad1b7d
Maybe I'll get back to rust some other time.

[nikomatsakis]

triage: P-medium