Distro-overridable default Unix linker

[etcwilde]

DRAFT NOTES AND QUESTIONS:
I didn't modify the Darwin or Windows paths here. I think it would be a bit confusing if it didn't. That said, I'm a little bit confused about what this "toolchain" means. Is it referring to the tools used when the driver is running on that platform, or when the driver is compiling code for that platform? e.g. if I managed to legally obtain the Windows headers on Linux, would the driver try to use link.exe to fail, or would it try to use.. I guess gold?

---

Being able to specify various defaults that the driver should assume is necessary for building specialized distributions of the toolchain. This mostly affects the non-Darwin Unix environment, where folks may be using a whole smorgasbord of different linkers in their toolchain. The original experience made it impossible to generally build the driver to default to a specific linker without editing source, which isn't great.

Swift has the challenge of not allowing values assigned to macro definitions though, and neither does SwiftPM. Unless you have some fun with the clang importer, of course :). So we introduce a new DriverDefaults module that really just contains strings that were set as defaults. It seems that that clang importer was unable to import macros directly if they involved more than the literal, so I was unable to use SWIFT_DEFAULT_LINKER directly since I want to quote it, but instead have to assign it to an actual value.

To configure the default linker while building with SwiftPM, swift build -Xcc -DSWIFT_DEFAULT_LINKER=lld, and you will end up with a Swift driver that will default to using lld on Linux.

[compnerd]

Yes, the default behavior on windows would be to use the system linker (link.exe), but we can use lld instead on other platforms.

The current implementation on Windows uses a ToolchainInfo.plist to add the default flags (-use-ld=lld) at least the SPM layer. We should be able to handle that in swift-driver as well (I believe that there is a TODO in the driver code for this already).

I think that I would prefer to avoid this path on Windows in light of the above.

[etcwilde]

Yes, the default behavior on windows would be to use the system linker (link.exe), but we can use lld instead on other platforms.

So does that mean the Windows driver is using link.exe to build ELF files for Linux today?

The current implementation on Windows uses a ToolchainInfo.plist to add the default flags (-use-ld=lld) at least the SPM layer. We should be able to handle that in swift-driver as well (I believe that there is a TODO in the driver code for this already).

Where do we put that plist? Is it installed next to the driver, or do we need a way to configure it? Or do we embed it in the binary itself and unpack it? I'm not entirely enthusiastic about the idea of a /usr/bin/ToolchainInfo.plist containing all of the driver defaults. (at least it should be /usr/bin/SwiftToolchainInfo.plist or something a little less ambiguous).

[artemcm]

I'm a little bit confused about what this "toolchain" means. Is it referring to the tools used when the driver is running on that platform, or when the driver is compiling code for that platform?

It means the tools used when the driver is running on that platform. Toolchain type is selected based on the specified target triple:
https://github.com/apple/swift-driver/blob/775df464251a39af22166ab44b140afff6a80819/Sources/SwiftDriver/Driver/Driver.swift#L3101

[artemcm]

We must also implement the building of this module in our CMake build infrastructure.

[etcwilde]

heh, I thought I put that comment in my draft notes. Yeah, I put up the draft to get feedback on the approach at the moment. :)

[artemcm]

Overall I am happy with this approach to defining a build-time constant and for selecting the linker!

[compnerd]

Yes, the default behavior on windows would be to use the system linker (link.exe), but we can use lld instead on other platforms.

So does that mean the Windows driver is using link.exe to build ELF files for Linux today?

As @artemcm already mentioned, it is platform specific (it is selected by that host that we are building for).

The current implementation on Windows uses a ToolchainInfo.plist to add the default flags (-use-ld=lld) at least the SPM layer. We should be able to handle that in swift-driver as well (I believe that there is a TODO in the driver code for this already).

Where do we put that plist? Is it installed next to the driver, or do we need a way to configure it? Or do we embed it in the binary itself and unpack it? I'm not entirely enthusiastic about the idea of a /usr/bin/ToolchainInfo.plist containing all of the driver defaults. (at least it should be /usr/bin/SwiftToolchainInfo.plist or something a little less ambiguous).

We place that into the platform SDK:
https://github.com/apple/swift/blob/main/utils/build-windows-toolchain.bat#L777-L778

[etcwilde]

I don't think the platform SDK should dictate what linker you use. I could build a toolchain that runs on macOS that contains lld, and then pair it with a Linux SDK. I know that I have lld, but the SDK may dictate that I use gold, which would be incorrect for this use-case, but might be fine when using that SDK on a Linux box. It's certainly related, but there isn't just a single linker that would work with that SDK.

[compnerd]

The SDK can have content that dictates the linker - I think that the SDK does need to dictate the linker for the platform.

[etcwilde]

Rejecting in favor of just using the clang-linker to decide (along with all of its logic for build-time configuration, configuration files, flags, etc...)