compiler-builtins: Int trait functions are not inlined on wasm

[CryZe]

I'm currently doing some benchmarks and I've seen that __multi3 is taking a really long time to calculate on wasm. Turns out that none of the helper functions it uses are inlined at all, so it does a ton of unnecessary calls and misses a lot of potential optimizations:

(func $compiler_builtins::int::mul::__multi3::h134a3ac9dcea74e0 (type $t39) (param $p0 i32) (param $p1 i64) (param $p2 i64) (param $p3 i64) (param $p4 i64)
    (local $l5 i32) (local $l6 i64) (local $l7 i64) (local $l8 i64)
    global.get $g0
    i32.const 16
    i32.sub
    local.tee $l5
    global.set $g0
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 4294967295
    i64.and
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 4294967295
    i64.and
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    local.set $l6
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 32
    i64.shr_u
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 4294967295
    i64.and
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    local.get $l6
    i64.const 32
    i64.shr_u
    i64.add
    local.tee $l7
    i64.const 32
    i64.shr_u
    call $<i64_as_compiler_builtins::int::Int>::from_unsigned::h7efafe9ef60e0310
    local.set $l8
    local.get $l5
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 32
    i64.shr_u
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 4294967295
    i64.and
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    local.get $l7
    i64.const 4294967295
    i64.and
    i64.add
    local.tee $l7
    i64.const 32
    i64.shl
    local.get $l6
    i64.const 4294967295
    i64.and
    i64.or
    local.get $l8
    local.get $l7
    i64.const 32
    i64.shr_u
    call $<i64_as_compiler_builtins::int::Int>::from_unsigned::h7efafe9ef60e0310
    i64.add
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 32
    i64.shr_u
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    i64.const 32
    i64.shr_u
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    call $<i64_as_compiler_builtins::int::Int>::from_unsigned::h7efafe9ef60e0310
    i64.add
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::high::hc703bd80007672ce
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    call $<i64_as_compiler_builtins::int::Int>::from_unsigned::h7efafe9ef60e0310
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    call $<i64_as_compiler_builtins::int::Int>::wrapping_add::h9472fc521f362e13
    local.get $p1
    local.get $p2
    call $<i128_as_compiler_builtins::int::LargeInt>::low::h654cf3febd0eecf8
    call $<i64_as_compiler_builtins::int::Int>::from_unsigned::h7efafe9ef60e0310
    local.get $p3
    local.get $p4
    call $<i128_as_compiler_builtins::int::LargeInt>::high::hc703bd80007672ce
    call $<i64_as_compiler_builtins::int::Int>::wrapping_mul::h558368f41dd93d09
    call $<i64_as_compiler_builtins::int::Int>::wrapping_add::h9472fc521f362e13
    call $<i128_as_compiler_builtins::int::LargeInt>::from_parts::h8ea13ebf67bbf227
    local.get $l5
    i64.load
    local.set $p3
    local.get $p0
    local.get $l5
    i32.const 8
    i32.add
    i64.load
    i64.store offset=8
    local.get $p0
    local.get $p3
    i64.store
    local.get $l5
    i32.const 16
    i32.add
    global.set $g0)

The same is probably true for the Float trait.

[CryZe]

It seems like rust-lang/compiler-builtins#349 initially had a lot more inline annotations but removed them later on. That may have regressed the performance on wasm, but I'm not entirely sure.

This makes me wonder if compiler-builtins is even compiled with LTO at all as it supposedly is meant to be?!

[CryZe]

Looks like this is indeed a regression that started with Rust 1.44, which switched from compiler-builtins 0.1.25 to 0.1.27, which is where this PR is part of. These are the instructions in 1.43:

(func $__multi3 (type $t40) (param $p0 i32) (param $p1 i64) (param $p2 i64) (param $p3 i64) (param $p4 i64)
    (local $l5 i64) (local $l6 i64)
    local.get $p0
    local.get $p3
    i64.const 32
    i64.shr_u
    local.tee $l5
    local.get $p1
    i64.const 32
    i64.shr_u
    local.tee $l6
    i64.mul
    local.get $p3
    local.get $p2
    i64.mul
    i64.add
    local.get $p4
    local.get $p1
    i64.mul
    i64.add
    local.get $p3
    i64.const 4294967295
    i64.and
    local.tee $p3
    local.get $p1
    i64.const 4294967295
    i64.and
    local.tee $p1
    i64.mul
    local.tee $p4
    i64.const 32
    i64.shr_u
    local.get $p3
    local.get $l6
    i64.mul
    i64.add
    local.tee $p3
    i64.const 32
    i64.shr_u
    i64.add
    local.get $p3
    i64.const 4294967295
    i64.and
    local.get $l5
    local.get $p1
    i64.mul
    i64.add
    local.tee $p3
    i64.const 32
    i64.shr_u
    i64.add
    i64.store offset=8
    local.get $p0
    local.get $p3
    i64.const 32
    i64.shl
    local.get $p4
    i64.const 4294967295
    i64.and
    i64.or
    i64.store)

cc @tmiasko

[alexcrichton]

There's been a few various changes around codegen units handling and compiler-builtins recently, and it looks like this is caused by compiling this crate with -C codegen-units=1. The compiler ignores this setting for compiler-builtins but elsewhere in the compiler disables ThinLTO with only one CGU.

[alexcrichton]

Ok I've transferred this to the rust-lang/rust repository since this isn't a bug with compiler-builtins itself I believe but rather how we build compiler-builtins.

I believe this is an accidental regression from #70846 because only part of the compiler knows that compiler-builtins is built with more than one CGU, the rest of the compiler thinks it's one CGU (respecting CLI flags). I'll be opening a PR shortly to fix this.