Description
Tasks:
- Refactor so target architectures are always immediately clear (#1479).
- Build
aarch64-apple-darwin
and make release archives for it (#1479). - Make release archives with universal binaries (#1486).
- Build
aarch64-pc-windows-msvc
and make release archives for it (#1479). - Research if any analogous situation may apply on Linux-based systems (#1478 (comment)).
Current behavior 😯
The release workflow builds binaries for x86_64-apple-darwin
but not for aarch64-apple-darwin
.
An ARM64 (AArch64) macOS system is capable of running x86-64 macOS binaries by emulation, and will do so automatically, so it is easy to run the x86-64 builds on an ARM64 Mac. But in view of how performance is a key benefit of gitoxide
, users would most often not prefer to do this, since the instruction sets are not compatible, so emulation imposes a significant overhead. Most importantly, users would most often not want to inadvertently run an x86-64 build on ARM64.
Unfortunately, this is likely to happen when using cargo binstall
:
- When using it directly, it prompts for confirmation, but users may miss the different architecture in the target name.
- When invoking it indirectly, as through
cargo install-update
, it does not prompt for confirmation.
It is really the indirect case where I believe users likely to miss that this is installing an x86-64 binary on their ARM64 system. This is because:
- It may be one of many upgrades as when passing
-a
/--all
. - This happens, without any warning, even when upgrading from an ARM64 build.
This can be fixed for future releases by adding aarch64-apple-darwin
jobs to the release workflow. That has further benefits, since even users who are not being inadvertently misled into using the wrong build will have another way, besides cargo quickinstall
, to install same-architecture binaries.
cargo quickinstall
does succeed at obtaining a native aarch64-apple-darwin
build, since there is such a build among the cargo quickinstall
releases. Current versions of cargo binstall
are capable of installing those. But that does not happen here with cargo binstall
, since cargo binstall
apparently prefers the gitoxide
releases even though their architecture is not as good of a match.
Unless something gets in the way, I will try to do this in my next PR that modifies the workflow, and I may fix by itself or with limited other changes rather than combining it with fixes for #1477.
An analogous situation exists for Windows
The same problem happens on an ARM64 Windows system, which also supports x86-64 executables by emulation. On such a system, one would ordinarily wish to run a aarch64-pc-windows-msvc
build, but cargo binstall
provides a x86_64-pc-windows-msvc
build instead. I expect that this is less often encountered in practice so far, in part because I don't think ARM64 Windows systems are are common yet as ARM64 macOS systems.
Fortunately, it should be feasible to add both at the same time. Unlike on the Ubuntu runner where the tools to perform cross-compilation without cross
would have to be installed explicitly (such as via sudo apt install ...
) and further steps might be needed to install some libraries or to enable pkg-config
to find them, on the macOS and Windows runners both targets have the Rust-nonspecific tools for targeting both architectures already.
Linux-based systems
Something like this may happen for Linux-based systems. I haven't investigated that at this time, because I wouldn't be immediately fixing it for those systems, as #1477 should probably be fixed before adding new Linux targets.
Other systems
At this time, we don't make any binary builds for targets outside macOS, Windows, and Linux-based systems. So users of such systems are not likely to be affected.
Expected behavior 🤔
Since macOS is increasingly run on ARM64 processors, one may expect aarch64-apple-darwin
binaries to be available. But the reason I wanted to open this issue and ensure this is not forgotten even if I have trouble adding the target is two other expectations that I think users hold more strongly and implicitly:
- Getting a binary that is slower because it is for a different architecture that requires emulation should not happen without clear indication.
- Switching the architecture of the binary should not happen automatically when upgrading.
- The recommended binary installation method in the readme (which is
cargo binstall
) should deliver a build of the correct architecture. - If
cargo binstall
is otherwise capable of delivering a build of the correct architecture by using thecargo-quickinstall
releases, then nothing about the binariesgitoxide
provides as its own releases should prevent that from happening.
To an extent the reason these expectations are not satisfied is due to aspects of the design of some other projects that could perhaps also be improved. For example, cargo install-update
could be changed to warn or even prompt when it would upgrade to a different architecture, or cargo binstall
could check for better architecture matches in the cargo-quickinstall
releases even if an installable match has already been found.
However, for gitoxide
, it should be feasible and maybe even easy to fix this by also shipping ARM64 macOS releases.
Regarding Windows: In this section, I didn't add anything about expectations for Windows ARM64 targets, since as noted above, I don't think the expectations about that are as strong, though I do believe we should provide such builds.
Regarding Linux-based systems: Users may be less likely to assume suitable builds are available, because most feature builds are not currently available (also something that #1477 will work toward).
Git behavior
I think this is mostly not applicable, because as far as I know, the upstream Git project does not build official binaries. However:
Downstream git
builds for macOS
macOS does have git
(or maybe the Apple developer tools install it, I'm not sure). That binary carries both x86-64 and ARM64 code:
~> git version
git version 2.39.3 (Apple Git-146)
~> type git
git is /usr/bin/git
~> file (command -v git)
/usr/bin/git: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/git (for architecture x86_64): Mach-O 64-bit executable x86_64
/usr/bin/git (for architecture arm64e): Mach-O 64-bit executable arm64e
(The ARM64 code, at least on that particular macOS 14.5 system, is arm64e
rather than arm64
, but I think that does not have significant performance implications, and that, even though there are many different ARM architectures, arm64
vs. arm64e
is an ABI difference rather than being different architectures.)
Although this is a relevant comparison, I mainly present the above to show that the file
command on macOS is capable of revealing when a binary contains code of both architectures, so it's clear that its output, as shown below, really does demonstrate the issue.
Git for Windows
Git for Windows does not yet provide ARM64 builds, so the x86-64 version of Git for Windows really is the best available target for an ARM64 Windows system (unless one is willing to use experimental builds, which in any case I don't think are provided as binaries).
This is one of the current benefits of gitoxide
and one of the reasons I think it's useful to make it as easy as possible to install ARM64 binaries on Windows. While gitoxide
has performance benefits for some of its functionality that overlaps with that of git
, such as for cloning, the performance benefit is far greater on an ARM64 Windows system where Git for Windows requires emulation but gitoxide
does not. (Of course, Git for Windows native ARM64 builds for Windows will be coming; this is not likely to be a long-standing difference.)
Steps to reproduce 🕹
I used a macOS 14.5 system running on an ARM64 (AArch64) processor, and nothing was set up in a way that would itself lead to an x86-64 build being seen as preferred:
~> sw_vers -productVersion
14.5
~> uname -m
arm64
~> rustup toolchain list
stable-aarch64-apple-darwin (default)
~> rustup target list --installed
aarch64-apple-darwin
Installing with cargo binstall
gave x86-64, not ARM64, binaries:
~> cargo binstall gitoxide
INFO resolve: Resolving package: 'gitoxide'
WARN The package gitoxide v0.37.0 (x86_64-apple-darwin) has been downloaded from github.com
INFO This will install the following binaries:
INFO - ein (ein -> /Users/ek/.cargo/bin/ein)
INFO - gix (gix -> /Users/ek/.cargo/bin/gix)
Do you wish to continue? yes/[no]
? yes
INFO Installing binaries...
INFO Done in 14.699131666s
~> file (command -v gix ein)
/Users/ek/.cargo/bin/gix: Mach-O 64-bit executable x86_64
/Users/ek/.cargo/bin/ein: Mach-O 64-bit executable x86_64
(This was in fish
. To do this in zsh
, bash
, or another Bourne-style shell, I would use a $
before the opening parenthesis of the command substitution.)
I uninstalled, then installed the previous version from source, then upgraded with cargo install-upgrade
, showing that the result, due to its use of cargo-binstall
, was to upgrade to a x86-64 build:
~> cargo uninstall gitoxide
Removing /Users/ek/.cargo/bin/ein
Removing /Users/ek/.cargo/bin/gix
~> cargo install [email protected] --locked --quiet
~> cargo install-update --all
Polling registry 'https://index.crates.io/'.....
Package Installed Latest Needs update
gitoxide v0.36.0 v0.37.0 Yes
cargo-binstall v1.8.0 v1.8.0 No
cargo-nextest v0.9.72 v0.9.72 No
cargo-quickinstall v0.2.10 v0.2.10 No
cargo-update v13.4.0 v13.4.0 No
Updating gitoxide
INFO resolve: Resolving package: 'gitoxide'
WARN The package gitoxide v0.37.0 (x86_64-apple-darwin) has been downloaded from github.com
INFO This will install the following binaries:
INFO - ein (ein -> /Users/ek/.cargo/bin/ein)
INFO - gix (gix -> /Users/ek/.cargo/bin/gix)
INFO Installing binaries...
INFO Done in 3.101160291s
Updated 1 package.
Overall updated 1 package: gitoxide.
~> file (command -v gix ein)
/Users/ek/.cargo/bin/gix: Mach-O 64-bit executable x86_64
/Users/ek/.cargo/bin/ein: Mach-O 64-bit executable x86_64
I uninstalled again, and installed the current version from source for contrast, showing that gives ARM64:
~> cargo uninstall gitoxide
Removing /Users/ek/.cargo/bin/ein
Removing /Users/ek/.cargo/bin/gix
~> cargo install gitoxide --quiet
~> file (command -v gix ein)
/Users/ek/.cargo/bin/gix: Mach-O 64-bit executable arm64
/Users/ek/.cargo/bin/ein: Mach-O 64-bit executable arm64
And again, showing that cargo quickinstall
gives ARM64 as well:
~> cargo quickinstall gitoxide
Calling `cargo-binstall` to do the install
INFO resolve: Resolving package: 'gitoxide'
INFO resolve: Verified signature for package 'gitoxide-0.37.0-aarch64-apple-darwin': timestamp:1722141643 file:gitoxide-0.37.0-aarch64-apple-darwin.tar.gz hashed
WARN The package gitoxide v0.37.0 (aarch64-apple-darwin) has been downloaded from third-party source QuickInstall
INFO This will install the following binaries:
INFO - ein (ein -> /Users/ek/.cargo/bin/ein)
INFO - gix (gix -> /Users/ek/.cargo/bin/gix)
INFO Installing binaries...
INFO Done in 2.783621666s
~> file (command -v gix ein)
/Users/ek/.cargo/bin/gix: Mach-O 64-bit executable arm64
/Users/ek/.cargo/bin/ein: Mach-O 64-bit executable arm64
Windows: On an ARM64 Windows 11 system with rustup
installed via https://win.rustup.rs/aarch64 (the Rust site and the rustup site don't yet link to it, but it's official), and with cargo-binstall
installed with this technique, installing gitoxide
using cargo binstall
installs the x86-64 version:
PS C:\Users\parnassus> cargo uninstall gitoxide
Removing C:\Users\parnassus\.cargo\bin\ein.exe
Removing C:\Users\parnassus\.cargo\bin\gix.exe
PS C:\Users\parnassus> cargo binstall gitoxide
INFO resolve: Resolving package: 'gitoxide'
WARN The package gitoxide v0.37.0 (x86_64-pc-windows-msvc) has been downloaded from github.com
INFO This will install the following binaries:
INFO - ein.exe (ein.exe -> C:\Users\parnassus\.cargo\bin\ein.exe)
INFO - gix.exe (gix.exe -> C:\Users\parnassus\.cargo\bin\gix.exe)
Do you wish to continue? yes/[no]
? yes
INFO Installing binaries...
INFO Done in 13.2124373s
PS C:\Users\parnassus> &'C:\Program Files\Git\usr\bin\file.exe' (gcm gix).path
C:\Users\parnassus\.cargo\bin\gix.exe: PE32+ executable (console) x86-64, for MS Windows, 5 sections
PS C:\Users\parnassus> &'C:\Program Files\Git\usr\bin\file.exe' (gcm ein).path
C:\Users\parnassus\.cargo\bin\ein.exe: PE32+ executable (console) x86-64, for MS Windows, 5 sections
PS C:\Users\parnassus> cargo uninstall gitoxide
Removing C:\Users\parnassus\.cargo\bin\ein.exe
Removing C:\Users\parnassus\.cargo\bin\gix.exe