`m68k-unknown-linux-gnu`: can't compile functions with certain return types

[AnnikaCodes]

I think the m68k support for Rust is really interesting, and I've been trying to make a project with it. However, I'm unable to build the core library, which makes it really hard to actually use Rust.

You can replicate this by running cargo build -Zbuild-std=core --target m68k-unknown-linux-gnu.

I expected it to build successfully.

Instead, I get an LLVM error:

LLVM ERROR: unable to allocate function return #2
error: could not compile `core`
warning: build failed, waiting for other jobs to finish...
error: build failed

I think this has something to do with functions that return u64 and u128?

Meta

rustc --version --verbose:

rustc 1.57.0-nightly (f03eb6bef 2021-10-02)
binary: rustc
commit-hash: f03eb6bef8ced8a243858b819e013b9caf83d757
commit-date: 2021-10-02
host: x86_64-apple-darwin
release: 1.57.0-nightly
LLVM version: 13.0.0

Backtrace

   Compiling core v0.0.0 (/Users/annika/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core)
   Compiling rustc-std-workspace-core v1.99.0 (/Users/annika/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/rustc-std-workspace-core)
error: could not compile `core`

Caused by:
  process didn't exit successfully: `rustc --crate-name core --edition=2018 /Users/annika/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=803798926cf71182 -C extra-filename=-803798926cf71182 --out-dir /Users/annika/randbats-winrates/target/m68k-unknown-linux-gnu/debug/deps --target m68k-unknown-linux-gnu -Z force-unstable-if-unmarked -L dependency=/Users/annika/randbats-winrates/target/m68k-unknown-linux-gnu/debug/deps -L dependency=/Users/annika/randbats-winrates/target/debug/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
warning: build failed, waiting for other jobs to finish...
error: build failed

[AnnikaCodes]

OK, I did some more testing and found that it crashes with the following return types:

• u128
• i128
• Tuples including at least one u64, i64, or f64
• Option<T> where T = u64, i64, or f64

Here's a minimal example:

#![no_std]
#![feature(no_core, lang_items)]
#![no_core]

#[lang = "sized"]
pub trait Sized {}

fn llvm_test() -> (u64, bool) {
    (24234324, false)
}

[glaubitz]

The m68k backend is in an early stage, so some issues are to be expected.

If it's a LLVM issue, please file a bug in the LLVM upstream tracker against the M68k backend.

CC @ricky26 @mshockwave @jrtc27

[AnnikaCodes]

I will check if the bug is present in upstream LLVM and file a bug if indicated, thanks!

[jrtc27]

Those all look like cases where the ABI returns values indirectly (the usual thing on 32-bit architectures, where 64-bit values are returned in register pairs, but larger things indirectly), so the frontend should be adding an implicit sret argument to the function and representing it as returning void, with the backends always assuming that part of the ABI has been done correctly. What IR do you get for your example?

[AnnikaCodes]

This is the IR I get:

*** IR Dump Before Module Verifier (verify) ***
; Function Attrs: nonlazybind uwtable
define { i64, i8 } @llvm_test() unnamed_addr #0 !dbg !6 {
start:
  %0 = alloca { i64, i8 }, align 4
  %1 = bitcast { i64, i8 }* %0 to i64*, !dbg !18
  store i64 24234324, i64* %1, align 4, !dbg !18
  %2 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 1, !dbg !18
  store i8 0, i8* %2, align 4, !dbg !18
  %3 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 0, !dbg !19
  %4 = load i64, i64* %3, align 4, !dbg !19
  %5 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 1, !dbg !19
  %6 = load i8, i8* %5, align 4, !dbg !19, !range !20
  %7 = trunc i8 %6 to i1, !dbg !19
  %8 = zext i1 %7 to i8, !dbg !19
  %9 = insertvalue { i64, i8 } undef, i64 %4, 0, !dbg !19
  %10 = insertvalue { i64, i8 } %9, i8 %8, 1, !dbg !19
  ret { i64, i8 } %10, !dbg !19
}

[jrtc27]

Yep, rustc frontend bug, should be:

; Function Attrs: nonlazybind uwtable
define void @llvm_test({ i64, i8 }* sret %result) unnamed_addr #0 !dbg !6 {
start:
  %0 = alloca { i64, i8 }, align 4
  %1 = bitcast { i64, i8 }* %0 to i64*, !dbg !18
  store i64 24234324, i64* %1, align 4, !dbg !18
  %2 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 1, !dbg !18
  store i8 0, i8* %2, align 4, !dbg !18
  %3 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 0, !dbg !19
  %4 = load i64, i64* %3, align 4, !dbg !19
  %5 = getelementptr inbounds { i64, i8 }, { i64, i8 }* %0, i32 0, i32 1, !dbg !19
  %6 = load i8, i8* %5, align 4, !dbg !19, !range !20
  %7 = trunc i8 %6 to i1, !dbg !19
  %8 = zext i1 %7 to i8, !dbg !19
  %9 = insertvalue { i64, i8 } undef, i64 %4, 0, !dbg !19
  %10 = insertvalue { i64, i8 } %9, i8 %8, 1, !dbg !19
  store { i64, i8 } %10, { i64, i8 }* %result
  ret void
}

[AnnikaCodes]

Alright, thanks for the example of correct IR; I'll see if I can figure out how to fix this in rustc.

[AnnikaCodes]

OK, I've been working on fixes for this, but I haven't figured out a way to test anything since rustc won't build because of another LLVM error, which seems to be related to the left shift operator when used on 64bit integers:

LLVM ERROR: Cannot select: 0x7fb0c9876678: i32,i32 = shl_parts 0x7fb0c98788e0, Constant:i32<0>, 0x7fb0c9876540
  0x7fb0c98788e0: i32,ch = CopyFromReg 0x7fb0c621bf58, Register:i32 %0
    0x7fb0c98769b8: i32 = Register %0
  0x7fb0c9877170: i32 = Constant<0>
  0x7fb0c9876540: i32 = add nuw nsw 0x7fb0c9878670, Constant:i32<21>
    0x7fb0c9878670: i32 = AssertZext 0x7fb0c9876af0, ValueType:ch:i8
      0x7fb0c9876af0: i32,ch = CopyFromReg 0x7fb0c621bf58, Register:i32 %2
        0x7fb0c9876a88: i32 = Register %2
    0x7fb0c9878a80: i32 = Constant<21>
In function: _ZN17compiler_builtins5float4conv11__floatsidf17hb63047be181dca88E

This is happening in the __floatsidf function in compiler-builtins, which calls int_to_float which does indeed do left shifting.

[glaubitz]

@AnnikaCodes Thanks a lot for investigating this and thanks a lot to @jrtc27 for figuring out so quickly what the problem is!

[AnnikaCodes]

The left-shift on 64-bit ints crash is an LLVM bug (it can be reproduced in C); I've reported it to the LLVM project.

[glaubitz]

Bug has been fixed in LLVM llvm/llvm-project#51461.