x.py's naming of stages is confusing

[nnethercote]

Rust has the standard multi-stage structure that all bootstrapping compilers have.

A newcomer who knows about compiler stages will be confident with this, until they run a command like ./x.py build --stage 1 and get output like this:

Building stage0 std artifacts 
    ...
Copying stage0 std from stage0
Building stage0 test artifacts 
    ...
Copying stage0 test from stage0
Building stage0 compiler artifacts 
    ...
Copying stage0 rustc from stage0
Building stage0 codegen artifacts 
    ...
Assembling stage1 compiler 
Building stage1 std artifacts 
    ...
Copying stage1 std from stage1
Building stage1 test artifacts 
    ...
Copying stage1 test from stage1
Building stage1 compiler artifacts 
    ...
Copying stage1 rustc from stage1
Building stage1 codegen artifacts 
    ...
Building rustdoc for stage1 
    ...

For a newcomer, this is completely bizarre.

• Why is it building stage 0? Isn't stage 0 the compiler you download?
• How does it assemble the stage 1 compiler before building the stage 1 artifacts?
• Why is it building two stages? Shouldn't it only build one stage?

The key to understanding this is that x.py uses a surprising naming convention:

• A "stage N artifact" is an artifact that is produced by the stage N compiler.
• The "stage (N+1) compiler" is assembled from "stage N artifacts".
• A --stage N flag means build with stage N.

Somebody had to explain this to me when I started working on Rust. I have since had to explain it to multiple newcomers. Even though I understand it now, I still find it very confusing, and I have to think carefully about it all.

Here's a naming convention that makes more sense to me:

• A "stage N artifact" is an artifact that is produced by the stage (N-1) compiler.
• The "stage N compiler" is assembled from "stage N artifacts".
• A --stage N flag means build stage N, using stage (N-1).

That way, a command like ./x.py build --stage 1 would produce output like this:

Building stage1 std artifacts 
    ...
Copying stage1 std from stage1
Building stage1 test artifacts 
    ...
Copying stage1 test from stage1
Building stage1 compiler artifacts 
    ...
Copying stage1 rustc from stage1
Building stage1 codegen artifacts 
    ...
Assembling stage1 compiler 

Is there any appetite for this change? I realize it would be invasive and people would have to update their aliases, build scripts, etc. But it might be worthwhile to avoid the ongoing confusion to both newcomers and experienced contributors. Deprecating the word "stage" in favour of "phase" could help with the transition.

[steveklabnik]

FWIW I also found this confusing for a while, and would support this chnage.

…

On Apr 10, 2019, at 7:23 PM, Nicholas Nethercote ***@***.***> wrote: Rust has the standard multi-stage structure that all bootstrapping compilers have. A newcomer who knows about compiler stages will be confident with this, until they run a command like ./x.py build --stage 1 and get output like this: Building stage0 std artifacts ... Copying stage0 std from stage0 Building stage0 test artifacts ... Copying stage0 test from stage0 Building stage0 compiler artifacts ... Copying stage0 rustc from stage0 Building stage0 codegen artifacts ... Assembling stage1 compiler Building stage1 std artifacts ... Copying stage1 std from stage1 Building stage1 test artifacts ... Copying stage1 test from stage1 Building stage1 compiler artifacts ... Copying stage1 rustc from stage1 Building stage1 codegen artifacts ... Building rustdoc for stage1 ... For a newcomer, this is completely bizarre. Why is it building stage 0? Isn't stage 0 the compiler you download? Why is it building two stages? Shouldn't it only build one stage? The key to understanding this is that x.py uses a surprising naming convention: A "stage N artifact" is an artifact that is produced by the stage N compiler. The "stage (N+1) compiler" is assembled from "stage N artifacts". A --stage N flag means build with stage N. Somebody had to explain this to me when I started working on Rust. I have since had to explain it to multiple newcomers. Even though I understand it now, I still find it very confusing, and I have to think carefully about it all. Here's a naming convention that makes more sense to me: A "stage N artifact" is an artifact that is produced by the stage (N-1) compiler. The "stage N compiler" is assembled from "stage N artifacts". A --stage N flag means build stage N, using stage (N-1). That way, a command like ./x.py build --stage 1 would produce output like this: Building stage1 std artifacts ... Copying stage1 std from stage1 Building stage1 test artifacts ... Copying stage1 test from stage1 Building stage1 compiler artifacts ... Copying stage1 rustc from stage1 Building stage1 codegen artifacts ... Assembling stage1 compiler Is there any appetite for this change? I realize it would be invasive and people would have to update their aliases, build scripts, etc. But it might be worthwhile to avoid the ongoing confusion to both newcomers and experienced contributors. Deprecating the word "stage" in favour of "phase" could help with the transition. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ehuss]

I think this is essentially #57963. I agree it is confusing (I even drew a little picture to help me understand it).

[cuviper]

@ehuss that diagram is excellent!

[cuviper]

cc @Mark-Simulacrum @alexcrichton -- related to the discussion we had at all-hands.

[Mark-Simulacrum]

My plan is to put in sometime this weekend to figure out and propose PRs for the changes discussed at the all hands (which should, to an extent, remove this issue by making dealing with --stage largely unnecessary). We may separately want to land a patch that changes stage "naming" though.

[Manishearth]

I basically have to re-figure this out any time I'm doing anything complex with rustc, and yeah, we should change the naming here.

A key issue here is that the rust compiler is actually a thin wrapper around underlying libraries, which are shipped with rust like the stdlib is. So which stage is which differs based on whether you care about the stdlib or the compiler. Kind of. I've complained about this before and generally seen agreement that this needs fixing but not as many good solutions

[spastorino]

@nnethercote I think your explanation and @ehuss diagram are perfect for rustc-guides 😄. I also would like to see this change happen, meanwhile can we document this on Rustc? are you comfortable if your explanation and diagrams are reused for rustc-guides?.
@igaray was interested in getting started with rustc-guides documentation maybe it's something he can use as a way to start.

[unbalancedparentheses]

Friday was the first time I compiled rust and it was pretty confusing for me too. I really like the diagram.

[nikomatsakis]

At minimum, we should move that content to the rustc-guide, yes. I am agnostic about the names. I'm wary that all names will be confusing unless there is a picture, and if there is a picture, any name may be fine.

[Mark-Simulacrum]

The current plan (discussed and mentioned at all hands) is to work towards not needing to say "--stage" unless you're hacking on rustbuild itself or doing something odd; I'm not sure how quickly we can make that happen, but hopefully quite soon.

[scottmcm]

@Mark-Simulacrum Does that apply to --keep-stage as well? That's the one I use most, since it's currently essentially if you're doing anything in core or std...

[Mark-Simulacrum]

Yes, keep-stage as well -- you can see some notes here: https://paper.dropbox.com/doc/Rustbuild-All-Hands-2019--AbYzEwK9y9KN~OKiOg2bMnWrAQ-MLKsxxj1SXORp4Qg8aGbQ

[nnethercote]

are you comfortable if your explanation and diagrams are reused for rustc-guides?.

Absolutely, please use the above words any way you like.