Skip to content

Accepted convention for incurring dependencies on external files in macro expansion hurts reproducibility #75887

Open
@pikajude

Description

@pikajude

According to this thread https://internals.rust-lang.org/t/pre-rfc-add-a-builtin-macro-to-indicate-build-dependency-to-file/9242/2 and a few other sources I found online, the common way to force a module with a proc-macro to be rebuilt based on external changes (for example, an HTML template) is to generate an expansion that uses include_bytes!() with the path to the file.

This function in librustc_span adds the string content of every macro input to the resulting rmeta file, including macros that are then re-expanded, like this one in the askama-derive crate. That macro expansion becomes include_bytes!("/absolute/path/to/source.html");, then the actual byte literal that contains the file contents.

I am using the Nix language/build system here and the cargo2nix tool, which causes every crate to be built in a temporary directory (to maximize isolation from the rest of the system). If the tempdir names differ between two builds of a crate that otherwise logically yields identical code, the metadata hashes will differ. (Though I haven't tested it myself, I can imagine this also being a problem with a tool like sccache if two devs with different filepaths to some git checkout try to use the same build artifacts).

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-macrosArea: All kinds of macros (custom derive, macro_rules!, proc macros, ..)A-reproducibilityArea: Reproducible / deterministic buildsO-NixOSOperating system: NixOS, https://nixos.org/T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions