Description
I recently tried to compile rust by using the ./x.py script. I noticed that the first thing that happens is a full clone of all the git submodules (see https://github.com/rust-lang/rust/blob/master/src/bootstrap/bootstrap.py#L679-L687).
Cloning the whole history of all the submodules is generally of little interest to compile the current version of rust, but it has a non-negligible memory footprint. Today, the largest submodules are:
src/llvm-projectwith 696.10 MiB of historysrc/doc/rust-by-examplewith 136.06 MiB of history
Downloading so much data can be detrimental for several reasons:
- This increases the latency of the build, especially for people with a slow downloading bandwidth.
- The remote git server takes some time to compress the objects before sending them, also increasing the latency for everyone.
- The history takes space on disk.
I've tried to tweak the lines in the above bootstrap.py script to limit the depth of the submodule update. Unfortunately, using --depth=1 doesn't work, because the submodule commit referenced here may not be the latest in the submodule repo, and apparently git cannot clone at a specific commit.
However, I found that --depth=20 (and --depth=1 for llvm-project) was working on today's state. Overall, this reduced the overhead of submodule updates quite significantly.
Of course, this isn't bullet-proof:
- Until git supports cloning a repo at a specific commit, the heuristic depth of 20 may not be enough in the future.
- I only tested this on a fresh clone of rust-lang/rust. Updating from an existing clone may not work.
Nonetheless, would it make sense to add an option (CLI flag to ./x.py for example) that allows to manually reduce the cloning depth?