Skip to content

improve CI cache docs #2247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 6, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions src/tests/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ Our CI workflow uses various caching mechanisms, mainly for two things:
### Docker images caching

The Docker images we use to run most of the Linux-based builders take a *long*
time to fully build. To speed up the build, we cache it using [Docker registry
time to fully build. To speed up the build, we cache them using [Docker registry
caching], with the intermediate artifacts being stored on [ghcr.io]. We also
push the built Docker images to ghcr, so that they can be reused by other tools
(rustup) or by developers running the Docker build locally (to speed up their
Expand All @@ -334,16 +334,32 @@ override the cache for the others. Instead, we store the images under different
tags, identifying them with a custom hash made from the contents of all the
Dockerfiles and related scripts.

The CI calculates a hash key, so that the cache of a Docker image is
invalidated if one of the following changes:

- Dockerfile
- Files copied into the Docker image in the Dockerfile
- The architecture of the GitHub runner (x86 or ARM)

[ghcr.io]: https://github.com/rust-lang-ci/rust/pkgs/container/rust-ci
[Docker registry caching]: https://docs.docker.com/build/cache/backends/registry/

### LLVM caching with sccache

We build some C/C++ stuff in various CI jobs, and we rely on [sccache] to cache
the intermediate LLVM artifacts. Sccache is a distributed ccache developed by
Mozilla, which can use an object storage bucket as the storage backend. In our
case, the artefacts are uploaded to an S3 bucket that we control
(`rust-lang-ci-sccache2`).
Mozilla, which can use an object storage bucket as the storage backend.

With sccache there's no need to calculate the hash key ourselves. Sccache
invalidates the cache automatically when it detects changes to relevant inputs,
such as the source code, the version of the compiler, and important environment
variables.
So we just pass the sccache wrapper on top of cargo and sccache does the rest.

We store the persistent artifacts on the S3 bucket `rust-lang-ci-sccache2`. So
when the CI runs, if sccache sees that LLVM is being compiled with the same C/C++
compiler and the LLVM source code is the same, sccache retrieves the individual
compiled translation units from S3.

[sccache]: https://github.com/mozilla/sccache

Expand Down