extend bootstrap related documentations

Signed-off-by: ozkanonur <[email protected]>

Author: onur-ozkan

src/building/bootstrapping.md

@@ -411,17 +411,17 @@ you can tell the bootstrap shim to print all env variables by adding `-vvv` to y
 
 This is an incomplete reference for the outputs generated by bootstrap:
 
-| Stage 0 Action                                            | Output                                       |
-|-----------------------------------------------------------|----------------------------------------------|
-| `beta` extracted                                          | `build/HOST/stage0`                          |
-| `stage0` builds `bootstrap`                               | `build/bootstrap`                            |
-| `stage0` builds `test`/`std`                              | `build/HOST/stage0-std/TARGET`               |
-| copy `stage0-std` (HOST only)                             | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
-| `stage0` builds `rustc` with `stage0-sysroot`             | `build/HOST/stage0-rustc/HOST`               |
-| copy `stage0-rustc` (except executable)                   | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
-| build `llvm`                                              | `build/HOST/llvm`                            |
-| `stage0` builds `codegen` with `stage0-sysroot`           | `build/HOST/stage0-codegen/HOST`             |
-| `stage0` builds `rustdoc`, `clippy`, `miri`, with `stage0-sysroot` | `build/HOST/stage0-tools/HOST`      |
+| Stage 0 Action                                                     | Output                                       |
+| ------------------------------------------------------------------ | -------------------------------------------- |
+| `beta` extracted                                                   | `build/HOST/stage0`                          |
+| `stage0` builds `bootstrap`                                        | `build/bootstrap`                            |
+| `stage0` builds `test`/`std`                                       | `build/HOST/stage0-std/TARGET`               |
+| copy `stage0-std` (HOST only)                                      | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
+| `stage0` builds `rustc` with `stage0-sysroot`                      | `build/HOST/stage0-rustc/HOST`               |
+| copy `stage0-rustc` (except executable)                            | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
+| build `llvm`                                                       | `build/HOST/llvm`                            |
+| `stage0` builds `codegen` with `stage0-sysroot`                    | `build/HOST/stage0-codegen/HOST`             |
+| `stage0` builds `rustdoc`, `clippy`, `miri`, with `stage0-sysroot` | `build/HOST/stage0-tools/HOST`               |
 
 `--stage=0` stops here.
 
@@ -448,3 +448,46 @@ This is an incomplete reference for the outputs generated by bootstrap:
 | copy `rustdoc`                                         | `build/HOST/stage2/bin`                                         |
 
 `--stage=2` stops here.
+
+### Clarification of build command's stdout
+
+In this part, we will investigate the build command's stdout in an action
+(similar, but more detailed and complete documentation compare to topic above).
+When you execute `x.py build --dry-run` command, the build output will be something
+like the following:
+
+```text
+Building stage0 library artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
+Copying stage0 library from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
+Building stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
+Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
+Assembling stage1 compiler (x86_64-unknown-linux-gnu)
+Building stage1 library artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
+Copying stage1 library from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
+Building stage1 tool rust-analyzer-proc-macro-srv (x86_64-unknown-linux-gnu)
+Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
+```
+
+#### Building stage0 {std,test,compiler} artifacts
+
+These steps use the provided (downloaded, usually) compiler to compile the
+local Rust source into libraries we can use.
+
+#### Copying stage0 {std,test,rustc}
+
+This copies the build output from Cargo into
+`build/$HOST/stage0-sysroot/lib/rustlib/$ARCH/lib`.
+[comment]: FIXME: this step's documentation should be expanded -- the information already here may be incorrect.
+
+#### Assembling stage1 compiler
+
+This copies the libraries we built in "building stage0 ... artifacts" into
+the stage1 compiler's lib directory. These are the host libraries that the
+compiler itself uses to run. These aren't actually used by artifacts the new
+compiler generates. This step also copies the rustc and rustdoc binaries we
+generated into `build/$HOST/stage/bin`.
+
+The stage1/bin/rustc is a fully functional compiler, but it doesn't yet have
+any libraries to link built binaries or libraries to. The next 3 steps will
+provide those libraries for it; they are mostly equivalent to constructing
+the stage1/bin compiler so we don't go through them individually.

src/building/how-to-build-and-run.md

@@ -54,6 +54,12 @@ clean` will not cause a rebuild of LLVM.
 
 ## Building the Compiler
 
+Building compiler is unfortunately not quite as simple as building other
+rust projects using `cargo build`. Instead, we use `x.py` script for building
+the compiler. The main reason behind it is the bootstrap stages where you use
+the output of one cargo invocation as the input compiler for another. Which
+is not possible with `cargo` itself.
+
 Note that building will require a relatively large amount of storage space.
 You may want to have upwards of 10 or 15 gigabytes available to build the compiler.
 

src/building/suggested.md

@@ -175,6 +175,35 @@ You can also use `--keep-stage 1` when running tests. Something like this:
 - Initial test run: `./x.py test tests/ui`
 - Subsequent test run: `./x.py test tests/ui --keep-stage 1`
 
+## Incremental builds
+
+Incremental compilation is enabled by default in almost all of the config profiles
+under the `src/bootstrap/defaults`.
+
+In case if it's not enabled, you can configure rustbuild to use incremental
+compilation with the `--incremental` flag:
+
+```sh
+$ ./x.py build --incremental
+```
+
+If you want to enable it once for all, you can apply the following configuration
+in your `config.toml`:
+
+```toml
+[rust]
+incremental=true
+```
+
+Incremental compilation will store compilation artifacts
+in `build/<host>/stage0-incremental`. Note that we only use incremental
+compilation for the stage0 -> stage1 compilation -- this is because
+the stage1 compiler is changing, and we don't try to cache and reuse
+incremental artifacts across different versions of the compiler.
+
+Note that if incremental compilation is not enabled, you will
+still be using the downloaded beta as your bootstrap.
+
 ## Fine-tuning optimizations
 
 Setting `optimize = false` makes the compiler too slow for tests. However, to

src/getting-started.md

@@ -114,6 +114,10 @@ serious development work. In particular, `./x.py build` and `./x.py test`
 provide many ways to compile or test a subset of the code, which can save a lot
 of time.
 
+Also, note that `x.py` supports all kinds of path suffixes for `compiler`, `library`,
+and `src/tools` directories. So, you can simply run `x.py test tidy` instead of
+`x.py test src/tools/tidy`. Or, `x.py build std` instead of `x.py build library/std`.
+
 [rust-analyzer]: ./building/suggested.html#configuring-rust-analyzer-for-rustc
 
 See the chapters on [building](./building/how-to-build-and-run.md),

src/tests/compiletest.md

@@ -4,6 +4,12 @@
 
 ## Introduction
 
+Most of the "meaty" steps that matter are backed by Cargo, which does indeed
+have its own parallelism and incremental management. Later steps, like
+tests, aren't incremental and simply run the entire suite currently.
+However, compiletest itself tries to avoid running tests when the artifacts
+that are involved (mainly the compiler) haven't changed.
+
 `compiletest` is the main test harness of the Rust test suite.
 It allows test authors to organize large numbers of tests
 (the Rust compiler has many thousands),