Skip to content

Recursion using extern "C" causes miscompilation #114312

Closed
@AngelicosPhosphoros

Description

@AngelicosPhosphoros

I tried this code:

pub enum Expr {
    Lit(isize),
    Sum(isize, isize),
    Sub(isize, isize),
}

pub fn eval_slow(expr: Expr) -> isize {
    use Expr::*;
    extern "C" fn eval_inner(expr: Expr)->isize{
        match expr {
            Lit(x) => x,
            Sum(x, y) => x + y,
            Sub(x, y) => eval_inner(Sum(x, -y)),
        }
    }
    eval_inner(expr)
}

I expected to see this happen: compiled function correctly handles Sub(x, y)

Instead, this happened: compiler generates code with infinite loop.

See asm (from -Copt-level=3):

example::eval_slow:
        mov     rax, qword ptr [rdi]
        test    rax, rax
        je      .LBB0_3
        cmp     eax, 1
        jne     .LBB0_2
        mov     rax, qword ptr [rdi + 16]
        add     rax, qword ptr [rdi + 8]
        ret
.LBB0_2:                            ;  <--------- Infinite loop header
        jmp     .LBB0_2          ;  <--------- Unconditional jump to previous instruction
.LBB0_3:
        mov     rax, qword ptr [rdi + 8]
        ret

Meta

rustc --version --verbose:

rustc 1.71.0 (8ede3aae2 2023-07-12)
binary: rustc
commit-hash: 8ede3aae28fe6e4d52b38157d7bfe0d3bceef225
commit-date: 2023-07-12
host: x86_64-unknown-linux-gnu
release: 1.71.0
LLVM version: 16.0.5
Compiler returned: 0

Metadata

Metadata

Assignees

Labels

A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.C-bugCategory: This is a bug.I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessP-criticalCritical priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-stablePerformance or correctness regression from one stable version to another.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions