De-stabilize target spec JSON

[Mark-Simulacrum]

In a recent Zulip discussion, it was noted that we currently consider JSON target specifications unstable, in the sense that options are relatively freely added and removed.

Currently, the compiler will load target specifications from three places.

If a triple is passed without trailing ".json":

• a built-in list, specified as Rust structs
• $RUST_TARGET_PATH, split like PATH on the system we're running on, or, if not set, the current directory.
  • we then attempt to load $triple.json from the directories listed

If a triple is passed with .json, then we directly attempt to load that file path (relative or absolute).

This is the history as best as I can tell:

• Original --target support, including RUST_TARGET_PATH, was implemented in Implement flexible target specification #16156.
• Trailing .json leading to a file lookup was added by @phil-opp in Introduce a TargetTriple enum to support absolute target paths #49019.

I cannot currently find any explicit discussion around stability, but presumably that discussion did happen at some time, happy to receive links to it. I would like to propose that we de-stabilize the RUST_TARGET_PATH and file-path loading, making only built-in targets usable. To my knowledge, this essentially is already the status quo: you need to build core/std to actually use a target, and that's not possible on beta/stable without intentional stability holes like RUSTC_BOOTSTRAP.

[Mark-Simulacrum]

@rustbot prioritize

Let's try this out!

[rustbot]

Error: The feature prioritize is not enabled in this repository.
To enable it add its section in the triagebot.toml in the root of the repository.

Please let @rust-lang/release know if you're having trouble with this bot.

[therealprof]

There seems to be some concern (raised on #rust-embedded:matrix.org) that this might negatively impact some tier 3 ports unless there're some guarantees along the lines of rust-lang/rfcs#2803 to provide a miniscule bar of entry as internal target specification.

[Mark-Simulacrum]

Can you clarify how those tier 3 targets are currently working on stable?

[jonas-schievink]

They don't work on stable, there was just some confusion about what this means for eg. Rust forks using custom LLVM backends. I believe we've resolved all of that though.

[therealprof]

Can you clarify how those tier 3 targets are currently working on stable?

I've no idea whether and how/whether they could work on stable. All I know is that there are some target specifications around which might potentially work with a stable compiler (at least if they were included) and some people raised that concern that this would limit the ability to achieve support in the future.

[Mark-Simulacrum]

Ah, okay. I would love to know what those use cases are.

I think I missed it in my first post - was thinking about it though - but I meant to also say that if we do move ahead with this I would want to do so very slowly; we'd land some warning on stable telling people that if they are using these today, to comment here, and let that sit for several cycles before actually removing the ability from stable users. I hope that can help alleviate your concern that there might be unknown use cases that people are using today.

[phil-opp]

I think de-stabilizing target JSON files is a good idea, given that they are effectively unstable already and only sparsely documented. However, I'm a bit worried about the user-experience for people that want to continue working with custom target specifications on nightly. I think we should provide some sort of file-based opt-in (e.g. a feature flag or a .cargo/config configuration key) instead of requiring a -Z command line parameter.

This is especially important because cargo also contains special support for custom targets (see e.g. rust-lang/cargo#5228 and rust-lang/cargo#6778), so we might need to make that part of the cargo command line API unstable too. The problem is that there is currently no equivalent to RUSTFLAGS for cargo, so unstable command line features are only usable with an explicit -Z command line argument. For example, the -Zbuild-std functionality requires you to pass that flag to every invocation of cargo build or cargo run, otherwise the compilation fails. This would make the use of custom targets very cumbersome. Edit: I created a proposal for a build.cargoflags configuration key at rust-lang/cargo#8127, which would alleviate my concerns. Edit2: There is now a way to configure unstable flags through config files: rust-lang/cargo#8393.

[petrochenkov]

cc #71769 (a PR that breaks custom target specifications that specify CRT objects)

[cardoe]

Looks like this has some interaction with #38338.

[cardoe]

So @Mark-Simulacrum, if I'm working on targets for compiling code as bare metal Xen targets and I have those as JSON files is the suggestion that I have to build a custom rustc to get that support if the JSON files go away or get them added into the compiler?

[cardoe]

Also interacts with #32819.