Revert "build: bootstrap SwiftPM with forked and not host clang"

[al45tair]

Reverts #64629

[al45tair]

@swift-ci Please smoke test

[finagolfin]

What did this pull break? It appears to work fine on the SPM CI.

[atrick]

Thanks

[atrick]

What did this pull break? It appears to work fine on the SPM CI.

Hopefully @al45tair can shed a little light. Our build jobs have ben failing with missing yams symbols. I don't know why the public SPM job passed while other build bots and our local builds are failing:

ld: warning: ignoring file lib/libCYaml.a, building for macOS-arm64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture arm64:
  "_yaml_document_end_event_initialize", referenced from:
      _$s4Yams7EmitterC9serialize4nodeyAA4NodeO_tKF in Emitter.swift.o
...

[finagolfin]

The error given appears to be related to linking and makes no sense, as both say arm64:
ld: warning: ignoring file lib/libCYaml.a, building for macOS-arm64 but attempting to link with file built for macOS-arm64. My guess is that this pull is not the reason, since other builds work, but that some related linking issue is being triggered somehow.

I think this is the wrong way to fix it, in any case, as this Swift-forked clang is what should be used, and was for years.

[al45tair]

My guess is that there’s some issue with the forked clang that makes the SwiftPM build fail when run from the build script. That needs to be addressed before we can switch over, because breaking it blocks other people.

I realise it’s annoying having things reverted, and I appreciate what you’re trying to do here, but I think in this case reverting until the build issue has been resolved is absolutely the right way forward. Sorry.

[finagolfin]

I realise it’s annoying having things reverted, and I appreciate what you’re trying to do here, but I think in this case reverting until the build issue has been resolved is absolutely the right way forward. Sorry.

Sure, I was just skeptical this was the cause of the build issue, as using this clang worked for years. If reverting this has fixed your SPM build, I have no problem with this revert.

Now the question is, where is the failing build log, so we can figure out the problem? Because I don't see any CI failures with this issue.

[al45tair]

I'm looking into this at the moment to try to figure out why it's happening (I can reproduce it locally). I'll let you know when I have a fix and we can try the original change again.

[al45tair]

OK, since we're discussing it in the revert already I'll keep the discussion here. The issue is that CMake is entirely too clever for its own good.

When it's trying to locate the ranlib tool to use, it looks to see what compiler you're using, and if that compiler happens to be Clang, it will try to use llvm-ranlib instead of whatever else it might have found. Now, CMake also knows that if it sees Apple Clang, it mustn't use llvm-ranlib but should use the one from the toolchain instead. However the Clang we've just built in the Swift build doesn't announce itself as Apple Clang; it announces itself as just ordinary Clang. So CMake tries to use llvm-ranlib, which then causes the linker to fail with a weird error message.

[al45tair]

Hence, when you changed things to use the just-built compiler, we ended up trying to use the wrong ranlib tool, resulting in a static archive that wouldn't link.

[finagolfin]

OK, makes sense, was the macOS CMake also recently updated, like on linux with #67018, so that's why it's only failing now? Let me know how you think this should be fixed on macOS, as this pull works everywhere else.

[al45tair]

I'm talking to @etcwilde about it at the moment. In an ideal world we'd be using libtool anyway on Darwin, so this might be a reason to fix that at the same time in CMake upstream. We'll keep you informed and circle back round to this changeset once we have things sorted out.

[al45tair]

OK. I think that once swiftlang/swift-package-manager#6721 has landed, we will probably be good to have another go at your change.