x.py install doesn't install binaries.

[vext01]

CC @pietroalbini

On today's git (0cd459f) x.py install with the config below doesn't install any binaries.

Here's the config:

[build]
docs = false

[install]
prefix = "build/ykrustc-stage2-latest"
sysconfdir = "etc"

docs = false is needed to work around a different x.py bug where installing the docs hangs.

After x.py install the resulting install dir contains:

build/ykrustc-stage2-latest/
build/ykrustc-stage2-latest/lib
build/ykrustc-stage2-latest/lib/rustlib
build/ykrustc-stage2-latest/lib/rustlib/manifest-rust-docs
build/ykrustc-stage2-latest/lib/rustlib/install.log
build/ykrustc-stage2-latest/lib/rustlib/rust-installer-version
build/ykrustc-stage2-latest/lib/rustlib/uninstall.sh
build/ykrustc-stage2-latest/lib/rustlib/components
build/ykrustc-stage2-latest/share
build/ykrustc-stage2-latest/share/doc
build/ykrustc-stage2-latest/share/doc/rust
build/ykrustc-stage2-latest/share/doc/rust/html
build/ykrustc-stage2-latest/share/doc/rust/html/SourceCodePro-Regular.woff
build/ykrustc-stage2-latest/share/doc/rust/html/ayu1.51.0.css
build/ykrustc-stage2-latest/share/doc/rust/html/src
build/ykrustc-stage2-latest/share/doc/rust/html/src/std
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/prelude
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/prelude/mod.rs.html
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/prelude/v1.rs.html
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/macros.rs.html
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/sys
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/sys/unix
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/sys/unix/os.rs.html
build/ykrustc-stage2-latest/share/doc/rust/html/src/std/sys/unix/futex.rs.html
...
build/ykrustc-stage2-latest/share/doc/rust/html/std/collections/hash_set
build/ykrustc-stage2-latest/share/doc/rust/html/std/collections/linked_list
build/ykrustc-stage2-latest/share/doc/rust/html/std/collections/hash
build/ykrustc-stage2-latest/share/doc/rust/html/std/collections/hash/map

Also odd that some docs are installed?

Thanks!

[pietroalbini]

@vext01 tried doing a ./x.py install, and it seems that all the files are properly installed, and docs are not installed. Could you try removing the build, build/ykrustc-stage2-latest and etc directories?

[vext01]

I nuked my whole git clone and started afresh. Cloned today's git code (b5c496d), put the above config in place, ran x.py install:

$ x.py install
...
   Compiling tempfile v3.1.0
   Compiling regex v1.3.9
   Compiling rustdoc v0.0.0 (/home/vext01/source/rust/src/librustdoc)
   Compiling rustdoc-tool v0.0.0 (/home/vext01/source/rust/src/tools/rustdoc)
    Finished release [optimized] target(s) in 1m 32s
Dist rustc-1.51.0-dev-x86_64-unknown-linux-gnu
        finished in 40.785 seconds
Install rustc stage2 (Some(TargetSelection { triple: "x86_64-unknown-linux-gnu", file: None }))
install: creating uninstall script at /home/vext01/source/rust/build/tmp/empty_dir/build/ykrustc-stage2-latest/lib/rustlib/uninstall.sh
install: installing component 'rustc'

    rustc installed.

warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
note: this message was printed twice to make it more likely to be seen
Build completed successfully in 0:28:40
$ find build/ykrustc-stage2-latest/
build/ykrustc-stage2-latest/
$

So it seems it created an empty directory this time.

So do we suspect this is something local to me, if you see something else?

[vext01]

I also tried a different prefix outside of build and got the same behaviour.

This is odd...

[pietroalbini]

I found out why my local result and your local result differ, and the piece of code that change it, but I still need to figure out why my change triggered the bug.

If we look at the log you posted, the path in this line has an extra build/tmp/empty_dir in it:

install: creating uninstall script at /home/vext01/source/rust/build/tmp/empty_dir/build/ykrustc-stage2-latest/lib/rustlib/uninstall.sh

That means the installation script is running successfully, but it's installing stuff on the wrong prefix, which is then deleted at the end of this snippet:

rust/src/bootstrap/install.rs

Lines 67 to 83 in b5c496d

	let empty_dir = builder.out.join("tmp/empty_dir");
	t!(fs::create_dir_all(&empty_dir));
	let mut cmd = Command::new("sh");
	cmd.current_dir(&empty_dir)
	.arg(sanitize_sh(&tarball.decompressed_output().join("install.sh")))
	.arg(format!("--prefix={}", sanitize_sh(&prefix)))
	.arg(format!("--sysconfdir={}", sanitize_sh(&sysconfdir)))
	.arg(format!("--datadir={}", sanitize_sh(&datadir)))
	.arg(format!("--docdir={}", sanitize_sh(&docdir)))
	.arg(format!("--bindir={}", sanitize_sh(&bindir)))
	.arg(format!("--libdir={}", sanitize_sh(&libdir)))
	.arg(format!("--mandir={}", sanitize_sh(&mandir)))
	.arg("--disable-ldconfig");
	builder.run(&mut cmd);
	t!(fs::remove_dir_all(&empty_dir));

I was not reproducing the error myself because I used absolute paths in my config.toml. I'll spend some time right now figuring out how my PRs triggered this bug.

[pietroalbini]

Found the cause of the regression! #80240 moved the path canonicalization code, causing the prefix path to be absolute while all the rest of the paths are relative. Before that PR all paths were instead absolute.

Working on a PR now.

[pietroalbini]

Opened a PR with a fix: #80797