[CMake 3.25+]: Swift 5.8: Fails to compile bootstrap1 modules

[jaredjones]

Description
Ubuntu 23.04 + Swift 5.8: Fails to compile bootstrap1 modules due to possible module map issues with CXX interop

When compiling Swift from source with --bootstrapping=bootstrapping, compilation fails during generation of modules. Specifically on the Basic module.

Steps to reproduce

1. Download Ubuntu 23.04, which is currently in beta but also in feature freeze so its ABI stable at this point.
2. Install Swift dependencies as listed in the documentation
3. Construct a swift source root by running:
   ./swift/utils/update-checkout --clone --tag swift-5.8-RELEASE
4. Invoke the build-script as follows:

export SCCACHE_CACHE_SIZE="50G" && ./swift/utils/build-script -R --swift-install-components="autolink-driver;compiler;clang-resource-dir-symlink;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc" --llvm-install-components="llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file" --lldb --build-lld=1 --build-ninja=1 --llbuild=1 --libcxx=1 --swiftpm=1 --foundation=1 --libdispatch=1 --indexstore-db=1 --libicu=1 --install-swiftpm=1 --build-swift-static-stdlib --build-swift-static-sdk-overlay --install-llvm=1 --install-swift=1 --install-foundation --install-libdispatch --swiftdocc=1 -j=8 --sccache=1 --skip-test-linux=1 --skip-build-benchmarks --skip-early-swift-driver --skip-early-swiftsyntax --bootstrapping=bootstrapping --install-destdir=/home/jjones/swift-install

Expected behavior
Bootstrap compilation should succeed, however a failure occurs when swiftc is invoked and has to deal with cxx-interop.

Build Failure

[1216/1439][ 84%][321.279s] Building swift module Basic
FAILED: bootstrapping1/SwiftCompilerSources/Basic.o /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping1/SwiftCompilerSources/Basic.o
cd /home/jjones/swift-source-root/swift/SwiftCompilerSources && /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping0/bin/swiftc -c -o /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping1/SwiftCompilerSources/Basic.o -I /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping0/bin/../lib -I /usr/lib -target x86_64-unknown-linux-gnu -module-name Basic -emit-module -emit-module-path /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping1/SwiftCompilerSources/Basic.swiftmodule -parse-as-library /home/jjones/swift-source-root/swift/SwiftCompilerSources/Sources/Basic/SourceLoc.swift /home/jjones/swift-source-root/swift/SwiftCompilerSources/Sources/Basic/Utils.swift -wmo -Xfrontend -validate-tbd-against-ir=none -Xfrontend -enable-experimental-cxx-interop -Xcc -UIBOutlet -Xcc -UIBAction -Xcc -UIBInspectable -Xfrontend -disable-implicit-string-processing-module-import -O -cross-module-optimization -Xcc -I -Xcc /home/jjones/swift-source-root/llvm-project/llvm/include -Xcc -I -Xcc /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/llvm-linux-x86_64/include -Xcc -I -Xcc /home/jjones/swift-source-root/llvm-project/clang/include -Xcc -I -Xcc /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/llvm-linux-x86_64/tools/clang/include -Xcc -I -Xcc /home/jjones/swift-source-root/swift/include -Xcc -I -Xcc /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/SwiftCompilerSources/../include -I /home/jjones/swift-source-root/build/Ninja-ReleaseAssert/swift-linux-x86_64/bootstrapping1/SwiftCompilerSources
<unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found
/home/jjones/swift-source-root/swift/include/swift/AST/ClangModuleLoader.h:19:10: note: while building module 'Clang_AST' imported from /home/jjones/swift-source-root/swift/include/swift/AST/ClangModuleLoader.h:19:
#include "clang/AST/DeclTemplate.h"
         ^
/home/jjones/swift-source-root/llvm-project/clang/include/clang/AST/APValue.h:16:10: note: while building module 'Clang_Basic' imported from /home/jjones/swift-source-root/llvm-project/clang/include/clang/AST/APValue.h:16:
#include "clang/Basic/LLVM.h"
         ^
/home/jjones/swift-source-root/llvm-project/clang/include/clang/Basic/Sanitizers.h:20:10: note: while building module 'LLVM_Transforms' imported from /home/jjones/swift-source-root/llvm-project/clang/include/clang/Basic/Sanitizers.h:20:
#include "llvm/Transforms/Instrumentation/AddressSanitizerOptions.h"
         ^
/home/jjones/swift-source-root/llvm-project/llvm/include/llvm/Transforms/AggressiveInstCombine/AggressiveInstCombine.h:20:10: note: while building module 'LLVM_intrinsic_gen' imported from /home/jjones/swift-source-root/llvm-project/llvm/include/llvm/Transforms/AggressiveInstCombine/AggressiveInstCombine.h:20:
#include "llvm/IR/PassManager.h"
         ^
<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "IR/Argument.h"
         ^
/home/jjones/swift-source-root/llvm-project/llvm/include/llvm/IR/Argument.h:17:10: note: in file included from /home/jjones/swift-source-root/llvm-project/llvm/include/llvm/IR/Argument.h:17:
#include "llvm/IR/Attributes.h"
         ^
/home/jjones/swift-source-root/llvm-project/llvm/include/llvm/IR/Attributes.h:31:10: note: in file included from /home/jjones/swift-source-root/llvm-project/llvm/include/llvm/IR/Attributes.h:31:
#include <set>
         ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/set:60:10: note: in file included from /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/set:60:
#include <bits/stl_tree.h>
         ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_tree.h:2008:5: error: missing '#include "gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_pair.h"'; 'pair' must be declared before it is used
    pair<typename _Rb_tree<_Key, _Val, _KeyOfValue,
    ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_pair.h:185:12: note: declaration here is not visible
    struct pair

Environment

• Swift compiler: swift-5.8-RELEASE
• OS: Ubuntu 23.04

The full build-log is attached.
swift_build_failure.txt

[finagolfin]

I see similar issues on Ubuntu 20.04, but can you first make sure that all package dependencies are installed? I know you listed it in your steps, but if you missed something, it may cause this.

[jaredjones]

@buttaface Thanks for the assistance, I've verified all of that and have attempted much more.
Since this was an upgrade from the previous version of Ubuntu, I went through and removed all of the old C++ headers and reinstalled them. When I removed them, I manually verified that nothing was left in the directory.
So I'm completely positive that the c++ includes directory is not invalid.
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/ -> /usr/include/c++/12

It's my understanding that the C++ standard library Ubuntu ships with does not include a modulemap. Instead, the swift team includes it at the path: stdlib/public/Cxx/std/libstdcxx.modulemap.

I've also attempted modifications to the modulemap in an attempt to resolve this. I also made changes to the CMakeLists.txt in SwiftCompilerSources to ensure that swiftc is invoked with c++17.

The logs to me look look like the modulemap isn't being included at all. However, I went into the loader and added an assert that crashes swiftc if the modulemap cannot be loaded. That assert isn't being hit, so the modulemap is indeed being loaded.

[finagolfin]

Having dealt with these modulemap issues for years, my guess is that the organization of the C++ stdlib headers changed with 23.04 and the current ordering for 20.04/22.04 doesn't work with the latest Ubuntu, as the bits/ headers are included automatically, not explicitly.

Of course, that doesn't explain why I see similar errors on 20.04 too, so maybe there's some implicit dependency we're missing.

[finagolfin]

@futurejones, I see you just released builds for Ubuntu 23.04, any idea how to fix these build issues we're seeing on Ubuntu?

[futurejones]

@buttaface I had 2 issues when first trying to build swift on ubuntu 23.04

1. When trying to build for x86_64 on an arm64 server using docker --platform linux/amd64 the build failed -

qemu: uncaught target signal 4 (Illegal instruction) - core dumped
<unknown>:0: error: unable to execute command: Illegal instruction
<unknown>:0: error: compile command failed due to signal 4 (use -v to see invocation)
[1223/1463][ 83%][2189.922s] Compiling /home/build-user/build/buildbot_linux/swift-linux-x86_64/bootstrapping0/stdlib/public/core/LINUX/x86_64/Swift.o
FAILED: bootstrapping0/stdlib/public/core/LINUX/x86_64/Swift.o 

2. When building native on x86_64 machine builds were failing with lots of set but not used errors like error: variable 'spins' set but not used [-Werror,-Wunused-but-set-variable].
   This is caused by changes in Cmake version 3.25.1.
   To fix this I downgraded to version 3.19.6 which is the default swift build version.

I am building with the following comand without any problems -
--preset buildbot_linux,no_test

[finagolfin]

Thanks @futurejones, maybe that's it, @jaredjones, as I'm building with one of the latest CMake releases, 3.26.1. I will try downgrading to what the linux CI is using and see if that makes a difference, you should try that too.

[finagolfin]

Yep, that's it, the problem goes away after downgrading to CMake 3.19.6. I'll look into what flags recent CMake is adding that causes these build issues.

[jaredjones]

I can also confirm that downgrading cmake has resolved this issue.

[finagolfin]

Checked the failing command, it is identical between the two CMake versions. This implies some other build input is being generated differently based on the CMake version, will dig deeper.

[finagolfin]

Narrowed it down further: built LLVM with CMake 3.26.1 and then built the Swift compiler first with CMake 3.26.1, then wiped it and built with 3.19.6 and now could reproduce the problem with both, so it looks like the issue is in some LLVM file generated by CMake.

I think this may be what caused my similar problems on Android in #60272 too, will look into it more.

[stephank]

For what it's worth, I'm also seeing this exact error packaging Swift 5.8 in NixOS. CMake 3.24.3 works, while 3.25.3 doesn't. (I'm not sure if it was already established this was a CMake >=3.25 issue. I wanted to narrow the range.)

I tried to diff the result of the configure step for LLVM between the two CMake versions, but nothing really stands out. (But I'm no expert.)