Skip to content

Undefined Behavior in safe code that unwinds out of extern "C" function #63943

Closed
@gnzlbg

Description

@gnzlbg

UPDATE: this optimization is perfectly correct. It only breaks code that already has undefined behavior. This is however an issue since until #52652 is fixed we want to keep such code working.

ORIGINAL TITLE: mis-compilation of noreturn extern "C" definitions that unwind on stable and nightly


Originally reported here: #63909 (comment) , which might contain a fix, but that has not been verified yet.

The default behavior of the Rust language on nightly Rust was to abort when a panic tries to escape from functions using certain ABIs intended for FFI, like "C". #62603 changed this behavior on nightly Rust to match the stable Rust behavior, which let the function unwind, while still applying the nounwind attribute to these functions. When these functions return Never, they are also noreturn, and this results in mis-compilations on stable and nightly Rust. MWE:

extern "C" fn bar() -> ! { panic!("nounwind noreturn fn unwinds") }
extern "C" fn baz() -> i32 {
    if let Ok(_) = std::panic::catch_unwind(|| bar() ) {
        unsafe { std::hint::unreachable_unchecked() }  // makes IR nicer
    }
    42
}
fn main() { std::process::exit((baz() != 42) as i32); }

cargo run --release returns success, but RUSTFLAGS="-C lto=fat" cargo run --release returns failure.

I think that since #63909 removes the nounwind attribute, it should end up fixing this bug, but we should probably add a test for this somewhere.


The problem #62603 intended to solve is to allow Rust->C->Rust FFI where a Rust callback called from C can unwind through C back into Rust. This example can be adapted to this application, where the miscompilation persists:

// foo.cpp
using fn_type = void(*)();
[[ noreturn ]] extern "C" void foo(fn_type x);
[[ noreturn ]] extern "C" void foo(fn_type x)  { x(); /* unreachable: */ throw 0; }
// main.rs
extern "C" { fn foo(x: extern "C" fn() -> !) -> !; }
extern "C" fn bar() -> ! { panic!("nounwind noreturn fn unwinds") }
extern "C" fn baz() -> i32 {
    if let Ok(_) = std::panic::catch_unwind(|| unsafe { foo(bar) }) {
        unsafe { std::hint::unreachable_unchecked() }
    }
    42
}
fn main() {
   std::process::exit((baz() != 42) as i32);
}

AFAICT this miscompilation has always existed for Rust->C-Rust FFI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-FFIArea: Foreign function interface (FFI)C-bugCategory: This is a bug.I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessT-langRelevant to the language team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions