build: `vendor = true` seems to not work anymore

[semarie]

When downloading a tarball (rustc-beta-src.tar.gz from https://static.rust-lang.org/dist/) and building it with config.toml file with vendor = true, the build fails early with:

Tue Aug  9 18:10:24 CEST 2022: starting rustbuild dist --jobs=4
detected default triple x86_64-unknown-openbsd from pre-installed rustc
Building rustbuild
running: /usr/local/bin/cargo build --manifest-path /data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/Cargo.toml --verbose --frozen
error: Unable to update registry `crates-io`

Caused by:
  attempting to make an HTTP request, but --frozen was specified
Traceback (most recent call last):
  File "/data/semarie/build-rust/build_dir/rustc-beta-src/x.py", line 48, in <module>
    bootstrap.main()
  File "/data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/bootstrap.py", line 926, in main
    bootstrap(help_triggered)
  File "/data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/bootstrap.py", line 903, in bootstrap
    build.build_bootstrap(args.color)
  File "/data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/bootstrap.py", line 774, in build_bootstrap
    run(args, env=env, verbose=self.verbose)
  File "/data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /usr/local/bin/cargo build --manifest-path /data/semarie/build-rust/build_dir/rustc-beta-src/src/bootstrap/Cargo.toml --verbose --frozen

I have this behaviour in beta since 2022-08-09, and with nightly since 2022-06-25.

My config.toml file is:

changelog-seen = 2

[build]
rustc = "/usr/local/bin/rustc"
cargo = "/usr/local/bin/cargo"
rustfmt = "/usr/local/bin/rustfmt"
python = "/usr/local/bin/python3"
gdb = "/usr/local/bin/egdb"
#docs = false
vendor = true
extended = true
verbose = 1

[install]
prefix = "/data/semarie/build-rust/install_dir/beta"

[dist]
src-tarball = false
missing-tools = true

[rust]
channel = "beta"
codegen-tests = false
verbose-tests = true

[target.x86_64-unknown-openbsd]
llvm-config = "/usr/local/bin/llvm-config"

[llvm]
static-libstdcpp = false
ninja = true

rustc and cargo binaries are 1.62.1-stable. The build is done on OpenBSD (but as it is failing at first cargo invocation, I think it is OS agnostic).

[bjorn3]

Which directory are you invoking ./x.py from? Is it the source directory or somewhere else? I think the .cargo/config.toml necessary for vendoring to work may be missed if you invoke it from another directory than the source directory.

cc @jyn514

[semarie]

x.py is invoked from outside the source directory. I will check that

[semarie]

ok, I confirm that invoking from inside the source directory solves the problem. now I need to use --config and --build-dir to keep my original behavior to build outside the source directory, but it isn't a big deal.

it might be something to mention in changelog file. I think it is common in downstream dsitribution to use a different build and source directories.

thanks for the help @bjorn3

[bjorn3]

I think we should support building outside the source directory again.

[jyn514]

#98526 fixed a bunch of these issues - @semarie can you try with a tarball from the latest nightly and see if it's fixed there?

[bjorn3]

That PR only changes the rust code of rustbuild, not the python code. This issue is while compiling rustbuild. The problem is that .cargo/config.toml is not in the current working directory when compiling rustbuild and as such ignored.

[semarie]

I confirm that the latest nightly doesn't build outside the directory.

also, please note that currently the problem is in both beta and nightly channels.

[jyn514]

The problem is that .cargo/config.toml is not in the current working directory

Ah, I see. That seems possible to fix, we can run the command from the source root. The working directory of the build itself shouldn't affect any of the runtime configuration.