Description
Proposal
I would like to propose an unstable flag for rustc
tentatively named -Zsplit-metadata
. It would instruct the compiler to avoid storing metadata in rlib
and dylib
artifacts. Instead, it would only store a small metadata stub, and the full metadata would be kept only in the .rmeta
file. When using such a compiled artifact as a dependency, the user would have to provide both the rlib
/dylib
and a corresponding .rmeta
file (passed through --extern
), which would contain the full metadata contents.
The main motivation for this flag is to reduce compiled artifact disk usage, and potentially also to improve compilation performance slightly when used with Cargo on systems with slow I/O/FS, by writing less data to disk (in exchange for reading a little bit more data, as the compiler would now have to read .rmeta
files when linking the final leaf artifact).
Currently, when Cargo compiles a crate, by default it uses --emit=metadata
for intermediate dependencies, which tells them to emit a .rmeta
file, in addition to the main .rlib
file. This allows Cargo to use pipelined compilation, which improves compilation speed. However, it also means that after the compilation, both the .rmeta
and .rlib
files are stored on the disk, and the crate metadata stored inside of them is thus duplicated. And since the metadata can be quite large, this can waste non-trivial amounts of disk space.
Preliminary results that show how much size could be saved by using this approach can be found here.
This idea has been floated around since a long time ago (rust-lang/rust#23366, rust-lang/rust#29511, rust-lang/rust#57076).
@bjorn3 has created an initial implementation, and under his guidance I cleaned it up a little bit and created an MVP in rust-lang/rust#137535.
If this gets accepted, the plan for landing this would be approximately this:
- Land
-Zsplit-metadata
- Teach Cargo to use it (e.g. behind an unstable Cargo flag)
- Dogfood
-Zsplit-metadata
for the compiler itself in bootstrap - Stabilize the flag
Mentors or Reviewers
Process
The main points of the Major Change Process are as follows:
- File an issue describing the proposal.
- A compiler team member or contributor who is knowledgeable in the area can second by writing
@rustbot second
.- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
-C flag
, then full team check-off is required. - Compiler team members can initiate a check-off via
@rfcbot fcp merge
on either the MCP or the PR.
- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
- Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.
You can read more about Major Change Proposals on forge.