Orbit release build of glium takes a very long time in codegen passes

[jonas-schievink]

Steps to reproduce:

• clone glium and check out glium/glium@11a6b7b (most recent commit at the time of writing)
• try a -Zorbit release build with the current nightly (2016-06-07):
  • For me, this involved env RUSTFLAGS="-Zorbit" rustup run nightly cargo rustc --release -- -Ztime-passes
• Observe compilation progress (very slowly but steadily, in old Rust tradition) until llvm module passes, which itself takes about 24 seconds on my machine, and then get stuck somewhere in LLVM codegen

cc @eddyb (I heard you like nothing more than MIR trans and LLVM bugs)

[jonas-schievink]

Output of perf after a while:

 51,06%  rustc    librustc_llvm-8102e29f.so             [.] llvm::SpillPlacement::iterate                                                                            
   9,49%  rustc    librustc_llvm-8102e29f.so             [.] _ZN4llvm14BlockFrequencypLES0_@plt                                                                       
   9,28%  rustc    librustc_llvm-8102e29f.so             [.] llvm::BlockFrequency::operator+=                                                                         
   4,89%  rustc    librustc_llvm-8102e29f.so             [.] llvm::BlockFrequency::operator+                                                                          
   4,42%  rustc    librustc_llvm-8102e29f.so             [.] _ZNK4llvm14BlockFrequencyplES0_@plt                                                                      
   1,49%  rustc    librustc_llvm-8102e29f.so             [.] llvm::AssumptionCache::scanFunction    

[arielb1]

Could you -C llvm-args=-print-before-all?

[jonas-schievink]

@arielb1 That somehow managed to lock up rustc (using 0% cpu) before getting to codegen (or even optimizations). Fun!

[jonas-schievink]

Here's the last part of it:

https://gist.github.com/jonas-schievink/c2599dd9f9687c907773ddb05f381430

[jonas-schievink]

So it seems like there are multiple issues at play here. Let me try to sum up what @eddyb and me managed to dig up a few days ago:

• gl-rs generates Rust code for loading OpenGL function pointers, and the generated code isn't quite optimal:
  • in fact, it generates 1624 lines of code for initializing the Gl struct, each of which performs 2 function calls
  • this can be seen in target/debug/build/glium-*/out/gl_bindings.rs when building manually
• MIR trans generates a lot more LLVM values than old trans
  • in fact, the bad function results in 13001 values for old trans, but in 64978 values (!) for MIR trans (all unoptimized)
  • this is still fine, since the unoptimized MIR build works
  • after optimizations, old trans uses 53594 values, while MIR trans uses 61714 - MIR trans uses less than before, so this point might be moot (⬅️this might also indicate that the LLVM inliner should take the caller's size into account when deciding whether to inline or not)
• LLVM's greedy register allocator seems to have some bad performance characteristics
  • in fact, the build spends over 5 minutes in there until I cancelled it (see the gist linked below)
  • specifying a different allocator by using llc --regalloc=X makes the code compile successfully

I'm really not sure why this all works out without optimizations but fails with them, but it seems like the greedy register allocator is partially at fault here. Of course it's the greedy allocator, it is used by default for an optimized build, while the "fast" allocator is used for debug builds.

For details, refer to this gist.

[eddyb]

cc @alexcrichton Any chance this was already found and fixed upstream?

[alexcrichton]

Hm doesn't look familiar to me, so it may or may not have been (sorry not much help on this one).

If codegen is generating so much code though it seems like this is something that should arguably be fixed in upstream at the same time the MIR regression is fixed...

[eddyb]

Found something https://llvm.org/bugs/show_bug.cgi?id=19217 - and an older one https://llvm.org/bugs/show_bug.cgi?id=11811.
None of these bugs seem to have had much progress on them.
The cleanup one would suggest a potential reason for why MIR trans displays the problem.

[arielb1]

I blame the LLVM inliner. This can be fixed by moving metaloadfn to a #[inline(never)] function:

                #[inline(never)]
                fn do_metaloadfn(loadfn: &mut FnMut(&str) -> *const __gl_imports::raw::c_void,
                                 symbol: &str,
                                 symbols: &[&str])
                                 -> *const __gl_imports::raw::c_void
                {
                    let mut ptr = loadfn(symbol);
                    if ptr.is_null() {
                        for &sym in symbols {
                            ptr = loadfn(sym);
                            if !ptr.is_null() { break; }
                        }
                    }
                    ptr
                }
                let mut metaloadfn = |symbol: &str, symbols: &[&str]| {
                    do_metaloadfn(&mut loadfn, symbol, symbols)
                };

That still leaves MIR ~20% slower, so there may be still some issue, but if at least finishes.