Tracking Issue for `abi_custom`

[folkertdev]

The feature gate for the issue is #![feature(abi_custom)].

The goal of the "custom" ABI is to define functions that use a custom calling convention that rust does not natively know about.

Background

One of the uses of naked functions is to implement custom calling conventions. We have some code in compiler-builtins like this:

// NOTE This function and the ones below are implemented using assembly because they are using a
// custom calling convention which can't be implemented using a normal Rust function.
#[unsafe(naked)]
pub unsafe extern "C" fn __aeabi_uidivmod() {
    core::arch::naked_asm!(
        "push {{lr}}",
        "sub sp, sp, #4",
        "mov r2, sp",
        "bl {trampoline}",
        "ldr r1, [sp]",
        "add sp, sp, #4",
        "pop {{pc}}",
        trampoline = sym crate::arm::__udivmodsi4
    );
}

The ABI needs to be specified, so extern "C" is used. However, this is misleading as the function does not actually use the C calling convention.

Correct ABI would be considered part of the preconditions for this function and it would only be callable inside an unsafe block, but Rust has no way to call the function correctly so it seems like we should prevent this.

Design

The "custom" ABI can only be used with #[unsafe(naked)] function declarations, and functions with this ABI cannot be called with a standard rust call expression.

/// # Safety
///
/// This function implements a custom calling convention that requires the
/// following inputs:
/// * `r8` contains a pointer
/// * `r9` contains a length
/// The pointer in `r8` must be valid for reads up to `r9` bytes.
///
/// `r8` and `r9` are clobbered but no other registers are.
#[unsafe(naked)]
pub unsafe extern "custom" fn foo() {
    core::arch::naked_asm!(
        /* ... */
    );
}

It is also possible to link to an extern "custom" function:

// SAFETY: `bar` is provided by `libbar.a` which we link.
unsafe extern "custom" {
    fn bar();
}

The only way to execute an extern "custom" is to use inline assembly for the call:

fn call_foo(buf: &[u8]) {
    // SAFETY: I didn't read the docs
    unsafe {
        foo();
        //~^ ERROR: `foo` has an unspecified ABI and cannot be called directly
        bar();
        //~^ ERROR: `bar` has an unspecified ABI and cannot be called directly
    }

    // SAFETY: call `foo` with its specified ABI, account for r8 & r9 clobbers
    unsafe {
        core::arch::asm!(
            "mov r8 {ptr}",
            "mov r9 {len}",
            "call {foo}",
            ptr = in(reg) buf.as_ptr(),
            len = in(reg) buf.len(),
            out("r8") _,
            out("r9") _,
            foo = sym foo,
        )
    }
}

An extern "custom¨ function cannot have any parameters or a return type. The types would be meaningless, and can easily go out of date. The function's documentation should describe how inline assembly can call the function: how arguments are passed and how a value is returned.

An extern "custom" functions is always unsafe, mostly as a nudge for writing a # Safety comment on their definition:

• extern "custom" function declarations must use the unsafe keyword
• in an unsafe extern "custom" { /* ... */ } block, the safe keyword cannot be used

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Discussion comments will get marked as off-topic or deleted.
Repeated discussions on the tracking issue may lead to the tracking issue getting locked.

Steps

• Implementation add extern "custom" functions #140770
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

None

History

• Lang proposal: extern "unspecified" for naked functions with arbitrary ABI #140566
• add extern "custom" functions #140770

@rustbot label +T-lang +A-inline-assembly