Skip to content

OSX compilation with debuginfo isn't deterministic #47086

Closed
@alexcrichton

Description

@alexcrichton

This is an issue extracted from #47066 (comment) which is caused by an issue that any OSX compilation with debuginfo ends up being nondeterministic. Specifically (currently known at least) the source of nondeterminism is that an mtime for an object file winds up in the final binary.

It turns out this isn't really our fault (unfortunately that makes it harder to fix!). This can be reproduced with just C and a linker:

# Compile an object file with a symbol in it
$ echo 'void foo() {}' > foo.c
$ cc -g foo.c -o foo.o -c

# Link that object to a shared library, take a look at the output
$ cc foo.o -m64 -dynamiclib -o libfoo.dylib -Wl,-dylib
$ md5 libfoo.dylib
MD5 (libfoo.dylib) = e60e735b7c919c19259daddd04a625c8

# update the timestamp on the object file
$ sleep 1
$ touch foo.o

# now link the same way we did above
$ cc foo.o -m64 -dynamiclib -o libfoo.dylib -Wl,-dylib
$ md5 libfoo.dylib
MD5 (libfoo.dylib) = 9754a78562696bbe5912efd9fc892a83

Here we're using the exact same object file (with two timestamps) and we're seeing different linked artifacts.

This is a source of bugs in programs that expect rustc to be deterministic (aka #47066 as was originally stated) and is something that we as rustc should probably fix.

Unfortunately I don't really know of a fix for this myself. I'd be tempted to take a big hammer to the problem and deterministically set all mtime fields for objects going into the linker to a known fixed value, but that unfortunately doesn't fix the determinism for C code (whose objects we don't control) and also is probably too big of a hammer (if a build system uses the mtime of the object to control rebuilds it'd get mixed up).

We could also use something like goblin and reach in to the specific field and remove the actual data. I found it in a symbol section with the N_OSO type (described in various documents online too apparently). We may be able to postprocess all output artifacts on OSX to maybe just zero out these fields unconditionally (or set them to something like May 15, 2015), although I'm not actually sure if this would be easy to do.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-debuginfoArea: Debugging information in compiled programs (DWARF, PDB, etc.)A-reproducibilityArea: Reproducible / deterministic buildsC-bugCategory: This is a bug.O-macosOperating system: macOST-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions