Rustup proxies not longer working with symlink resolution since 1.28.0

[syphar]

Verification

• I searched for recent similar issues at https://github.com/rust-lang/rustup/issues?q=is%3Aissue+is%3Aopen%2Cclosed and found no duplicates.
• I am on the latest version of Rustup according to https://github.com/rust-lang/rustup/tags and am still able to reproduce my issue.

Problem

coming from this report: rust-lang/rustwide#94 , currently docs.rs builds are broken because of this issue.

There seems to be something wrong with the symlinks that are now generated instead of hardlinks for the binaries (#4023).

We are using fs::canonicalize before passing it to the binaries:
https://github.com/rust-lang/rustwide/blob/744375a9050e0584124ea11adaeab41781fa8a50/src/utils.rs#L111-L112

Steps

What information would you need to be able to track this down?

we can reproduce it when just trying to run any docs.rs build,

the two errors I saw where both just showing that the call does to rustup install instead of cargo install (or other calls to cargo are done on rustup)

Possible Solution(s)

No response

Notes

No response

Rustup version

1.28.0

Installed toolchains

it happens with just one toolchain installed, tested with `stable-aarch64-apple-darwin` or `stable-x86_64-unknown-linux-gnu`

OS version

Ubuntu 18.04.2 LTS on the server, or newer ubuntu in docker images

[syphar]

cc @skius @Skgland

[djc]

Would be nice to get a bit closer to the root cause of this issue, do you think you can do that? Do you have a reproduction scenario that doesn't involve rustwide/docs.rs?

[ChrisDenton]

Quick workarounds I can think of:

• Maybe std::path::absolute would help?
• Or else just canonicalise the parent path than rejoin the file name?

[syphar]

Would be nice to get a bit closer to the root cause of this issue, do you think you can do that? Do you have a reproduction scenario that doesn't involve rustwide/docs.rs?

not yet, below is something that works with rustwide only.

I don't know rustwide internals / the build process very well, so a more low level example will take more time.

I'll probably first try to find a way to downgrade rustup for rustwide.

the rustwide / rustup example

The rustwide docs-builder example lets us see the error directly.

It can be made a little faster by choosing a smaller docker base image:

[...]
    // Create a new workspace in .workspaces/docs-builder
    let workspace =
        WorkspaceBuilder::new(Path::new(".workspaces/docs-builder"), "rustwide-examples")
            // this is new
            .sandbox_image(SandboxImage::remote(
                "ghcr.io/rust-lang/crates-build-env/linux-micro",
            )?)
            .init()?;
[...]

[syphar]

Quick workarounds I can think of:

* Maybe [`std::path::absolute`](https://doc.rust-lang.org/stable/std/path/fn.absolute.html) would help?

* Or else just `canonicalise` the parent path than rejoin the file name?

You mean in our rustwide code? Or rustup?

if rustwide, I first need to figure out why we actually canonicalize the paths before calling, that needs digging into old crater code before rustwide was extracted

[ChrisDenton]

On the rustwide side.

I don't think canonicalizing the filename can ever work with symlinked proxies because rustup shims rely on the link name to know where to direct the command to whereas canonicalizing a symlink removes that information by design. Canonicalizing the parent directory should always be fine though.

On the rustup side we could remove symlink support and only hard link (we've had this symlink behaviour for a long time but hard linking used to take precedence). Or else there might be something we can do to fix whatever problem necessitated canonicalizing.

[syphar]

Or else there might be something we can do to fix whatever problem necessitated canonicalizing.

I'm still trying to figure that out. It was added in rust-lang/crater@2c3e40c, but no specific explaination as to why.

Canonicalizing the parent directory should always be fine though.

I will see if I can figure out something here. ( hoping it's the only issue )

[Skgland]

I don't think canonicalizing the filename can ever work with symlinked proxies because rustup shims rely on the link name to know where to direct the command to whereas canonicalizing a symlink removes that information by design. Canonicalizing the parent directory should always be fine though.

Explicitly setting arg0 could work but that's only available on Unix and not on Windows.
Not canonicalizing is probably easier.

[ChrisDenton]

Oh of course. It can be done on Windows too but we haven't added it to the standard library yet. We really should. And possibly make a cross-platform arg0 function. But I realise that doesn't help in this moment.

[syphar]

so, path::absolute doesn't work. My current hunch is that this is related to how rustwide maps the toolchain into the docker container, and it somehow breaks when we try to use symlinks

[syphar]

Canonicalizing the parent directory should always be fine though.

this also doesn't work. Not sure why yet

[ChrisDenton]

My current hunch is that this is related to how rustwide maps the toolchain into the docker container, and it somehow breaks when we try to use symlinks

Ah, could this be #4222? Which is definitely something we should fix.

[syphar]

My current hunch is that this is related to how rustwide maps the toolchain into the docker container, and it somehow breaks when we try to use symlinks

Ah, could this be #4222? Which is definitely something we should fix.

could be. From what I remember, the toolchain installation is done outside of docker on the host, and then the directory is mapped into the docker container.