`x.py` submodule handling breaks when tracking upstream as `upstream`

[CAD97]

git --version --build-options

git version 2.37.1.windows.1
cpu: x86_64
built from commit: 323a69709944b193bb5cee81ff09fe9a4a686df5
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon

Example:

D:\tmp〉git clone git@github.com:rust-lang/rust.git -o upstream

Cloning into 'rust'...
remote: Enumerating objects: 1974535, done.
remote: Counting objects: 100% (185/185), done.
remote: Compressing objects: 100% (153/153), done.
remote: Total 1974535 (delta 83), reused 62 (delta 32), pack-reused 1974350
Receiving objects: 100% (1974535/1974535), 913.08 MiB | 42.72 MiB/s, done.
Resolving deltas: 100% (1547149/1547149), done.
Updating files: 100% (35926/35926), done.

D:\tmp〉cd rust

〉master:rust〉python ./x.py setup

downloading https://static.rust-lang.org/dist/2022-08-09/rust-std-beta-x86_64-pc-windows-msvc.tar.xz
################################################################################################################ 100.0%
extracting D:\tmp\rust\build\cache\2022-08-09\rust-std-beta-x86_64-pc-windows-msvc.tar.xz
downloading https://static.rust-lang.org/dist/2022-08-09/rustc-beta-x86_64-pc-windows-msvc.tar.xz
################################################################################################################ 100.0%
extracting D:\tmp\rust\build\cache\2022-08-09\rustc-beta-x86_64-pc-windows-msvc.tar.xz
downloading https://static.rust-lang.org/dist/2022-08-09/cargo-beta-x86_64-pc-windows-msvc.tar.xz
################################################################################################################ 100.0%
extracting D:\tmp\rust\build\cache\2022-08-09\cargo-beta-x86_64-pc-windows-msvc.tar.xz
Building rustbuild
   Compiling winapi v0.3.9
   Compiling proc-macro2 v1.0.39
   Compiling cfg-if v1.0.0
   Compiling unicode-ident v1.0.0
   Compiling version_check v0.9.4
   Compiling typenum v1.15.0
   Compiling memchr v2.5.0
   Compiling syn v1.0.95
   Compiling serde_derive v1.0.137
   Compiling cc v1.0.73
   Compiling serde v1.0.137
   Compiling windows_x86_64_msvc v0.36.1
   Compiling libc v0.2.126
   Compiling pkg-config v0.3.25
   Compiling log v0.4.17
   Compiling crossbeam-utils v0.8.8
   Compiling regex-syntax v0.6.26
   Compiling fnv v1.0.7
   Compiling serde_json v1.0.81
   Compiling once_cell v1.12.0
   Compiling lazy_static v1.4.0
   Compiling bootstrap v0.0.0 (D:\tmp\rust\src\bootstrap)
   Compiling itoa v1.0.2
   Compiling ryu v1.0.10
   Compiling unicode-width v0.1.9
   Compiling cpufeatures v0.2.2
   Compiling hex v0.4.3
   Compiling getopts v0.2.21
   Compiling generic-array v0.14.5
   Compiling thread_local v1.1.4
   Compiling windows-sys v0.36.1
   Compiling cmake v0.1.48
   Compiling quote v1.0.18
   Compiling aho-corasick v0.7.18
   Compiling bstr v0.2.17
   Compiling lzma-sys v0.1.17
   Compiling fd-lock v3.0.6
   Compiling crypto-common v0.1.3
   Compiling block-buffer v0.10.2
   Compiling digest v0.10.3
   Compiling regex v1.5.6
   Compiling sha2 v0.10.2
   Compiling globset v0.4.8
   Compiling winapi-util v0.1.5
   Compiling filetime v0.2.16
   Compiling opener v0.5.0
   Compiling tar v0.4.38
   Compiling same-file v1.0.6
   Compiling walkdir v2.3.2
   Compiling ignore v0.4.18
   Compiling toml v0.5.9
   Compiling xz2 v0.1.6
    Finished dev [unoptimized] target(s) in 22.27s
Welcome to the Rust project! What do you want to do with x.py?
a) library: Contribute to the standard library
b) compiler: Contribute to the compiler itself
c) codegen: Contribute to the compiler, and also modify LLVM or codegen
d) tools: Contribute to tools which depend on the compiler, but do not modify it directly (e.g. rustdoc, clippy, miri)
e) user: Install Rust from source
Please choose one (a/b/c/d/e): b
Updating submodule src/tools/rust-installer
Submodule 'src/rust-installer' (https://github.com/rust-lang/rust-installer.git) registered for path 'src/tools/rust-installer'
Cloning into 'D:/tmp/rust/src/tools/rust-installer'...
remote: Enumerating objects: 53, done.
remote: Counting objects: 100% (53/53), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 53 (delta 4), reused 36 (delta 0), pack-reused 0
Receiving objects: 100% (53/53), 38.05 KiB | 1.09 MiB/s, done.
Resolving deltas: 100% (4/4), done.
Submodule path 'src/tools/rust-installer': checked out '300b5ec61ef38855a07e6bb4955a37aa1c414c00'
Updating submodule src/tools/cargo
Submodule 'src/tools/cargo' (https://github.com/rust-lang/cargo.git) registered for path 'src/tools/cargo'
Cloning into 'D:/tmp/rust/src/tools/cargo'...
remote: Enumerating objects: 1546, done.
remote: Counting objects: 100% (1546/1546), done.
remote: Compressing objects: 100% (1097/1097), done.
remote: Total 1546 (delta 396), reused 872 (delta 297), pack-reused 0
Receiving objects: 100% (1546/1546), 2.33 MiB | 11.31 MiB/s, done.
Resolving deltas: 100% (396/396), done.

(continued)

remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 6da726708a4406f31f996d813790818dce837161. Direct fetching of that commit failed.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 6da726708a4406f31f996d813790818dce837161. Direct fetching of that commit failed.
Build completed unsuccessfully in 0:00:47

〉master:rust〉git show-ref HEAD

1ea4efd0656599f824e2567a5b7a95454f701c03 refs/remotes/upstream/HEAD

@rustbot label +C-bug +A-bootstrap

[CAD97]

Additional info: switching to a new branch that is not tracking upstream succeeds, so it is the upstream tracking that's messing something up:

terminal session

〉master:rust〉git switch -c new-branch
Switched to a new branch 'new-branch'
〉new-branch:rust〉python ./x.py setup
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.07s
Welcome to the Rust project! What do you want to do with x.py?
a) library: Contribute to the standard library
b) compiler: Contribute to the compiler itself
c) codegen: Contribute to the compiler, and also modify LLVM or codegen
d) tools: Contribute to tools which depend on the compiler, but do not modify it directly (e.g. rustdoc, clippy, miri)
e) user: Install Rust from source
Please choose one (a/b/c/d/e): b
Updating submodule src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 47, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 24 (delta 19), reused 5 (delta 0), pack-reused 0
Unpacking objects: 100% (24/24), 2.77 KiB | 78.00 KiB/s, done.
From https://github.com/rust-lang/cargo
 * branch            6da726708a4406f31f996d813790818dce837161 -> FETCH_HEAD
Submodule path 'src/tools/cargo': checked out '6da726708a4406f31f996d813790818dce837161'
Updating submodule src/tools/miri
Submodule 'src/tools/miri' (https://github.com/rust-lang/miri.git) registered for path 'src/tools/miri'
Cloning into 'D:/tmp/rust/src/tools/miri'...
remote: Enumerating objects: 1278, done.
remote: Counting objects: 100% (1278/1278), done.
remote: Compressing objects: 100% (893/893), done.
remote: Total 1278 (delta 455), reused 653 (delta 332), pack-reused 0
Receiving objects: 100% (1278/1278), 608.55 KiB | 4.95 MiB/s, done.
Resolving deltas: 100% (455/455), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 179, done.
remote: Counting objects: 100% (179/179), done.
remote: Compressing objects: 100% (90/90), done.
remote: Total 106 (delta 69), reused 27 (delta 13), pack-reused 0Receiving objects:  95% (101/106)
Receiving objects: 100% (106/106), 69.45 KiB | 13.89 MiB/s, done.
Resolving deltas: 100% (69/69), completed with 46 local objects.
From https://github.com/rust-lang/miri
 * branch            ab88e64b152d3704c35db96dbbc6efaaed67773f -> FETCH_HEAD
Submodule path 'src/tools/miri': checked out 'ab88e64b152d3704c35db96dbbc6efaaed67773f'
Updating submodule library/backtrace
Submodule 'library/backtrace' (https://github.com/rust-lang/backtrace-rs.git) registered for path 'library/backtrace'
Cloning into 'D:/tmp/rust/library/backtrace'...
remote: Enumerating objects: 136, done.
remote: Counting objects: 100% (136/136), done.
remote: Compressing objects: 100% (102/102), done.
remote: Total 136 (delta 9), reused 75 (delta 4), pack-reused 0
Receiving objects: 100% (136/136), 93.77 KiB | 2.47 MiB/s, done.
Resolving deltas: 100% (9/9), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), 706 bytes | 117.00 KiB/s, done.
From https://github.com/rust-lang/backtrace-rs
 * branch            4e5a3f72929f152752d5659e95bb15c8f6b41eff -> FETCH_HEAD
Submodule path 'library/backtrace': checked out '4e5a3f72929f152752d5659e95bb15c8f6b41eff'
Updating submodule library/stdarch
Submodule 'library/stdarch' (https://github.com/rust-lang/stdarch.git) registered for path 'library/stdarch'
Cloning into 'D:/tmp/rust/library/stdarch'...
remote: Enumerating objects: 317, done.
remote: Counting objects: 100% (317/317), done.
remote: Compressing objects: 100% (268/268), done.
remote: Total 317 (delta 50), reused 110 (delta 12), pack-reused 0
Receiving objects: 100% (317/317), 1.39 MiB | 4.14 MiB/s, done.
Resolving deltas: 100% (50/50), done.
Submodule path 'library/stdarch': checked out '42df7394d38bc7b945116ea3ad8a7cbcd1db50a9'
Submodule 'crates/intrinsic-test/acle' (https://github.com/ARM-software/acle.git) registered for path 'library/stdarch/crates/intrinsic-test/acle'
Cloning into 'D:/tmp/rust/library/stdarch/crates/intrinsic-test/acle'...
remote: Enumerating objects: 87, done.
remote: Counting objects: 100% (87/87), done.
remote: Compressing objects: 100% (83/83), done.
remote: Total 87 (delta 18), reused 31 (delta 1), pack-reused 0
Receiving objects: 100% (87/87), 616.53 KiB | 4.28 MiB/s, done.
Resolving deltas: 100% (18/18), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
remote: Enumerating objects: 67, done.
remote: Counting objects: 100% (67/67), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 40 (delta 25), reused 5 (delta 0), pack-reused 0
Unpacking objects: 100% (40/40), 255.02 KiB | 1008.00 KiB/s, done.
From https://github.com/ARM-software/acle
 * branch            5626f85f469f419db16f20b1614863aeb377c22b -> FETCH_HEAD
Submodule path 'library/stdarch/crates/intrinsic-test/acle': checked out '5626f85f469f419db16f20b1614863aeb377c22b'
`x.py` will now use the configuration at D:\tmp\rust/src/bootstrap/defaults/config.compiler.toml

`stage1` toolchain already linked; not attempting to link `stage1` toolchain

Rust's CI will automatically fail if it doesn't pass `tidy`, the internal tool for ensuring code quality.
If you'd like, x.py can install a git hook for you that will automatically run `tidy --bless` before
pushing your code to ensure your code is up to par. If you decide later that this behavior is
undesirable, simply delete the `pre-push` file from .git/hooks.
Would you like to install the git hook?: [y/N]
Ok, skipping installation!

To get started, try one of the following commands:
- `x.py check`
- `x.py build`
- `x.py test`
For more suggestions, see https://rustc-dev-guide.rust-lang.org/building/suggested.html
Build completed successfully in 0:00:17

[Mark-Simulacrum]

My current working theory is that this is a git bug (maybe we can work around it though).

$ git submodule update --init --recursive --depth=1 --progress -- src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 6da726708a4406f31f996d813790818dce837161. Direct fetching of that commit failed.

I think this is because git has created the submodule with a remote of origin, but probably due to the -o upstream passed for the rust checkout, it's trying to fetch/pull from the upstream remote inside the submodule too. That won't work; the upstream remote doesn't exist there, as the submodule was initially cloned (via that same git submodule update) with an "origin" remote.

I'm not sure what we should be doing to avoid this -- my guess is a proper fix would be in git itself... it might be that we can try to workaround it somehow, but I'm not seeing an obvious hook for us to attach to.

[jyn514]

Can we report this bug upstream to git? I think they use a mailing list?

[jyn514]

I was not able to reproduce this locally :/ I tried

git clone git@github.com:rust-lang/rust.git git-bug -o upstream
cd git-bug/
x setup compiler

and it checked out the submodules successfully. I'm using

; git --version --build-options
git version 2.25.1
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8

[WaffleLapkin]

For me this happens only sometimes. Most of the time when I checkout an old branch.

I think I've got a reproduction (by checking out an old commit and pushing a branch to my fork), probably has unnecessary steps, but it seems to be "working" for me.

; git clone git@github.com:rust-lang/rust.git git-bug
; cd git-bug
; git remote add fork git@github.com:WaffleLapkin/rust.git # replace with your own fork
; x setup compiler
; x c compiler # (optionally) cancel after all submodules are fetched
; git checkout HEAD~5000
; git switch -c new_test_branch_so_maybe_the_bug_will_show_up
; git commit --allow-empty -m "it's a commit, nothing more, nothing less"
; git push -u fork
; x c compiler
Updating only changed submodules
Updating submodule src/tools/rust-installer
fatal: 'fork' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: Fetched in submodule path 'src/tools/rust-installer', but it did not contain d66f476b4d5e7fdf1ec215c9ac16c923dc292324. Direct fetching of that commit failed.
Traceback (most recent call last):
  File "/home/waffle/projects/repos/git-bug/x.py", line 11, in <module>
    bootstrap.main()
  File "/home/waffle/projects/repos/git-bug/src/bootstrap/bootstrap.py", line 1009, in main
    bootstrap(help_triggered)
  File "/home/waffle/projects/repos/git-bug/src/bootstrap/bootstrap.py", line 969, in bootstrap
    build.update_submodules()
  File "/home/waffle/projects/repos/git-bug/src/bootstrap/bootstrap.py", line 845, in update_submodules
    self.update_submodule(module[0], module[1], recorded_submodules)
  File "/home/waffle/projects/repos/git-bug/src/bootstrap/bootstrap.py", line 797, in update_submodule
    run(update_args, cwd=self.rust_root, verbose=self.verbose, exception=True)
  File "/home/waffle/projects/repos/git-bug/src/bootstrap/bootstrap.py", line 143, in run
    raise RuntimeError(err)
RuntimeError: failed to run: git submodule update --init --recursive --progress src/tools/rust-installer

git version:

; git --version --build-options
git version 2.37.3
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh

[jyn514]

To make @WaffleLapkin's steps reproducible - HEAD is currently 750bd1a

[jyn514]

Oh lol, I just ran into this today. A bit more info:

PS C:\Users\Joshua Nelson\src\rust> git --version
git version 2.37.0.windows.1
PS C:\Users\Joshua Nelson\src\rust\src\tools\cargo> git remote -v
origin  https://github.com/rust-lang/cargo.git (fetch)
origin  https://github.com/rust-lang/cargo.git (push)
PS C:\Users\Joshua Nelson\src\rust> git remote -v
origin  https://github.com/rust-lang/rust (fetch)
origin  git@github.com:jyn514/rust.git (push)
personal        git@github.com:jyn514/rust.git (fetch)
personal        git@github.com:jyn514/rust.git (push)
PS C:\Users\Joshua Nelson\src\rust> $env:GIT_TRACE=1
PS C:\Users\Joshua Nelson\src\rust> git submodule update --init --recursive --depth=1 --progress -- src/tools/cargo
13:41:27.150152 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/bin
13:41:27.152153 git.c:749               trace: exec: git-submodule update --init --recursive --depth=1 --progress -- src/tools/cargo
13:41:27.152153 run-command.c:654       trace: run_command: git-submodule update --init --recursive --depth=1 --progress -- src/tools/cargo
13:41:27.306151 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.460297 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.463299 git.c:749               trace: exec: git-sh-i18n--envsubst --variables 'usage: $dashless $USAGE'
13:41:27.463299 run-command.c:654       trace: run_command: git-sh-i18n--envsubst --variables 'usage: $dashless $USAGE'
13:41:27.475297 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.504297 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.507297 git.c:749               trace: exec: git-sh-i18n--envsubst 'usage: $dashless $USAGE'
13:41:27.507297 run-command.c:654       trace: run_command: git-sh-i18n--envsubst 'usage: $dashless $USAGE'
13:41:27.518298 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.621299 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.622298 git.c:460               trace: built-in: git rev-parse --git-dir
13:41:27.677299 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.678298 git.c:460               trace: built-in: git rev-parse --git-path objects
13:41:27.753300 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.754301 git.c:460               trace: built-in: git rev-parse --show-prefix
13:41:27.781299 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.782306 git.c:460               trace: built-in: git rev-parse --show-toplevel
13:41:27.881298 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.883299 git.c:460               trace: built-in: git submodule--helper update --progress --recursive --init --depth=1 -- src/tools/cargo
13:41:27.892298 run-command.c:1553      run_processes_parallel: preparing to run up to 4 tasks
13:41:27.893298 run-command.c:1591      run_processes_parallel: done
13:41:27.899298 run-command.c:654       trace: run_command: cd src/tools/cargo; unset GIT_PREFIX; GIT_DIR=.git git rev-list -n 1 ba607b23db8398723d659249d9abf5536bc322e5 --not --all
13:41:27.964298 run-command.c:654       trace: run_command: cd src/tools/cargo; unset GIT_PREFIX; GIT_DIR=.git git fetch --depth=1
13:41:27.976298 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.978298 git.c:460               trace: built-in: git fetch --depth=1
13:41:27.982298 run-command.c:654       trace: run_command: git remote-https origin https://github.com/rust-lang/cargo.git
13:41:27.995299 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:27.996299 git.c:749               trace: exec: git-remote-https origin https://github.com/rust-lang/cargo.git
13:41:27.996299 run-command.c:654       trace: run_command: git-remote-https origin https://github.com/rust-lang/cargo.git
13:41:28.012300 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
13:41:28.458422 run-command.c:654       trace: run_command: git --shallow-file 'C:/Users/Joshua Nelson/src/rust/.git/modules/src/tools/cargo/shallow.lock' unpack-objects --pack_header=2,0
13:41:28.472425 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:28.476426 git.c:460               trace: built-in: git unpack-objects --pack_header=2,0
13:41:28.489425 run-command.c:654       trace: run_command: git --shallow-file 'C:/Users/Joshua Nelson/src/rust/.git/modules/src/tools/cargo/shallow.lock' rev-list --objects --stdin --quiet --alternate-refs
13:41:28.502424 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:28.505423 git.c:460               trace: built-in: git rev-list --objects --stdin --quiet --alternate-refs
13:41:28.531424 run-command.c:1553      run_processes_parallel: preparing to run up to 4 tasks
13:41:28.531424 run-command.c:1591      run_processes_parallel: done
13:41:28.531424 run-command.c:654       trace: run_command: git maintenance run --auto --no-quiet
13:41:28.544424 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:28.546423 git.c:460               trace: built-in: git maintenance run --auto --no-quiet
13:41:28.552423 run-command.c:654       trace: run_command: cd src/tools/cargo; unset GIT_PREFIX; GIT_DIR=.git git rev-list -n 1 ba607b23db8398723d659249d9abf5536bc322e5 --not --all
13:41:28.616423 run-command.c:654       trace: run_command: cd src/tools/cargo; unset GIT_PREFIX; GIT_DIR=.git git fetch --depth=1 personal ba607b23db8398723d659249d9abf5536bc322e5
13:41:28.629426 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:28.633427 git.c:460               trace: built-in: git fetch --depth=1 personal ba607b23db8398723d659249d9abf5536bc322e5
13:41:28.641426 run-command.c:654       trace: run_command: unset GIT_DIR GIT_PREFIX; GIT_PROTOCOL=version=2 'git-upload-pack '\''personal'\'''        
13:41:28.690425 exec-cmd.c:237          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
13:41:28.692425 git.c:460               trace: built-in: git upload-pack personal
fatal: 'personal' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain ba607b23db8398723d659249d9abf5536bc322e5. Direct fetching of that commit failed.

[jyn514]

Additional info: switching to a new branch that is not tracking upstream succeeds, so it is the upstream tracking that's messing something up

On the bright side, this means it's easy to work around the bug.

$ git submodule update --init --recursive --depth=1 src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: 'personal' does not appear to be a git repository
$ git -c branch.docs.remote=origin submodule update --init --recursive --depth=1 src/tools/cargo
From https://github.com/rust-lang/cargo
 * branch            f6e737b1e3386adb89333bf06a01f68a91ac5306 -> FETCH_HEAD
Submodule path 'src/tools/cargo': checked out 'f6e737b1e3386adb89333bf06a01f68a91ac5306'

[jyn514]

ah hmm, after doing that I can no longer reproduce the bug ... I think this might only happen on the very first submodule we try to clone? git is acting so strangely ... this happens even in a fresh worktree, it's only when it's a fresh clone it hits this problem, which makes no sense to me since submodules aren't shared between worktrees.

Anyway, here's a minimal reproduction that doesn't depend on exactly which commit you're using, or x.py setup:

git clone https://github.com/rust-lang/rust rust-not-a-worktree
cd rust-not-a-worktree
git remote add personal git@github.com:jyn514/rust
git checkout -b example-branch
git push -u personal example-branch --no-verify
x test tidy

I'll put up a PR shortly.