Clang segfault when using C++ modules and a precompiled header

Hello,
when compiling a project that uses 4 module partitions and a precompiled header with #include <iostream> , I get a segfault.

I've originally encountered this problem on MacOS with llvm clang-18.1.8 . I've also been able to reproduce it in an Ubuntu docker container (on the same machine) with: Ubuntu clang version 18.1.3 (1ubuntu1) Target: aarch64-unknown-linux-gnu.

The machine this happens on is Apple Silicon M2 Pro (mac mini). Host architecture is aarch64.

Reproduction instructions using docker:

1. docker run --rm -it -v /local/path/to/repro/project:/src ubuntu bash
2. apt-get update && apt-get install -y cmake clang clang-tools ninja-build
3. cd /src
4. PATH="/usr/lib/llvm-18/bin:$PATH" CXX=clang cmake -S . -B build -G Ninja
5. cmake --build build

Should get the following output:

FAILED: CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm 
/usr/lib/llvm-18/bin/clang   -std=gnu++23 -Winvalid-pch -Xclang -include-pch -Xclang /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx.pch -Xclang -include -Xclang /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx -MD -MT CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -MF CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o.d @CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o.modmap -o CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -c /src/Mod.cppm
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /usr/lib/llvm-18/bin/clang -cc1 -triple aarch64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name Mod.cppm -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=non-leaf -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +v8a -target-feature +fp-armv8 -target-feature +neon -target-abi aapcs -debugger-tuning=gdb -fdebug-compilation-dir=/src/build -fcoverage-compilation-dir=/src/build -resource-dir /usr/lib/llvm-18/lib/clang/18 -Winvalid-pch -std=gnu++23 -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fmodule-file=mod:part1=CMakeFiles/pch-cppm-problem-repro.dir/mod-part1.pcm -fmodule-file=mod:part2=CMakeFiles/pch-cppm-problem-repro.dir/mod-part2.pcm -fmodule-file=mod:part3=CMakeFiles/pch-cppm-problem-repro.dir/mod-part3.pcm -fmodule-file=mod:part4=CMakeFiles/pch-cppm-problem-repro.dir/mod-part4.pcm -fskip-odr-check-in-gmf -include-pch /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx.pch -include /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx -target-feature +outline-atomics -target-feature -fmv -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -x pcm CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm
 #0 0x0000ffff7698d398 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9d398)
 #1 0x0000ffff7698b5a8 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9b5a8)
 #2 0x0000ffff7698dafc (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9dafc)
 #3 0x0000ffff80cdf790 (linux-vdso.so.1+0x790)
 #4 0x0000ffff7f5465d0 clang::ASTReader::getLocalModuleFile(clang::serialization::ModuleFile&, unsigned int) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x27065d0)
 #5 0x0000ffff7f5c6e90 (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2786e90)
 #6 0x0000ffff7f5c5bc0 clang::ASTReader::loadDeclUpdateRecords(clang::ASTReader::PendingUpdateRecord&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2785bc0)
 #7 0x0000ffff7f575f44 clang::ASTReader::finishPendingActions() (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2735f44)
 #8 0x0000ffff7f578ed8 clang::ASTReader::FinishedDeserializing() (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2738ed8)
 #9 0x0000ffff7f55f79c clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, clang::serialization::ModuleFile**) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x271f79c)
#10 0x0000ffff7f6f9250 clang::ASTUnit::LoadFromASTFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, clang::PCHContainerReader const&, clang::ASTUnit::WhatToLoad, llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine>, clang::FileSystemOptions const&, std::shared_ptr<clang::HeaderSearchOptions>, bool, clang::CaptureDiagsKind, bool, bool, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x28b9250)
#11 0x0000ffff7f77cc7c clang::FrontendAction::BeginSourceFile(clang::CompilerInstance&, clang::FrontendInputFile const&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x293cc7c)
#12 0x0000ffff7f710580 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x28d0580)
#13 0x0000ffff7f7f8e4c clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x29b8e4c)
#14 0x0000aaaaacaf21c4 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/usr/lib/llvm-18/bin/clang+0x121c4)
#15 0x0000aaaaacaefbac (/usr/lib/llvm-18/bin/clang+0xfbac)
#16 0x0000aaaaacaeed00 clang_main(int, char**, llvm::ToolContext const&) (/usr/lib/llvm-18/bin/clang+0xed00)
#17 0x0000aaaaacafae40 main (/usr/lib/llvm-18/bin/clang+0x1ae40)
#18 0x0000ffff757d84c4 (/lib/aarch64-linux-gnu/libc.so.6+0x284c4)
#19 0x0000ffff757d8598 __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x28598)
#20 0x0000aaaaacaec970 _start (/usr/lib/llvm-18/bin/clang+0xc970)
clang: error: unable to execute command: Segmentation fault
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-18/bin
clang: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/Mod-9d95fb.cppm
clang: note: diagnostic msg: /tmp/Mod-9d95fb.sh
clang: note: diagnostic msg: 

********************
ninja: build stopped: subcommand failed. ```

pch-cppm-problem-repro.zip

I've also just confirmed this on x86_64. Same repro instructions (inside docker). Compiler: Ubuntu clang version 18.1.3 (1ubuntu1)
Target: x86_64-pc-linux-gnu

Just so that people don't have to download a .zip file to see the sources, I'm also trying to include the files themselves directly:
CMakeLists.txt:

cmake_minimum_required(VERSION 3.28)
project(PCH-CPPM-problem-repro VERSION 0.0.0 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 23)
add_executable(pch-cppm-problem-repro main.cpp)
set_target_properties(pch-cppm-problem-repro PROPERTIES CXX_SCAN_FOR_MODULES 1)
target_precompile_headers(pch-cppm-problem-repro PRIVATE stdafx.h)
target_sources(pch-cppm-problem-repro PRIVATE FILE_SET CXX_MODULES FILES
    Mod.cppm Part1.cppm Part2.cppm Part3.cppm Part4.cppm
)

main.cpp:

import mod;

int main() {
}

Mod.cppm:

export module mod;
export import :part1;
export import :part2;
export import :part3;
export import :part4;

Part1.cppm:

export module mod:part1;

Part2.cppm:

export module mod:part2;

Part3.cppm:

export module mod:part3;

Part4.cppm:

export module mod:part4;

stdafx.h:

#pragma once
#include <iostream>

Having 4 partitions exported in Mod.cppm seems key, removing one of them and segfault stops happening.
Also not including iostream in the PCH fixes the segfault. Those 2 things seem significant.
Putting vector instead of iostream also removes the segfault.

And funny thing is, it has problems both with the Ubuntu iostream file and the MacOS one as well.

[llvmbot]

@llvm/issue-subscribers-clang-modules

Author: None (ShaderKeeper)

Hello, when compiling a project that uses 4 module partitions and a precompiled header with ```#include <iostream>``` , I get a segfault.

I've originally encountered this problem on MacOS with llvm clang-18.1.8 . I've also been able to reproduce it in an Ubuntu docker container (on the same machine) with: Ubuntu clang version 18.1.3 (1ubuntu1) Target: aarch64-unknown-linux-gnu.

The machine this happens on is Apple Silicon M2 Pro (mac mini). Host architecture is aarch64.

Reproduction instructions using docker:

1. docker run --rm -it -v /local/path/to/repro/project:/src ubuntu bash
2. apt-get update && apt-get install -y cmake clang clang-tools ninja-build
3. cd /src
4. PATH="/usr/lib/llvm-18/bin:$PATH" CXX=clang cmake -S . -B build -G Ninja
5. cmake --build build

Should get the following output:

FAILED: CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm 
/usr/lib/llvm-18/bin/clang   -std=gnu++23 -Winvalid-pch -Xclang -include-pch -Xclang /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx.pch -Xclang -include -Xclang /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx -MD -MT CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -MF CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o.d @<!-- -->CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o.modmap -o CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -c /src/Mod.cppm
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.	Program arguments: /usr/lib/llvm-18/bin/clang -cc1 -triple aarch64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name Mod.cppm -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=non-leaf -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu generic -target-feature +v8a -target-feature +fp-armv8 -target-feature +neon -target-abi aapcs -debugger-tuning=gdb -fdebug-compilation-dir=/src/build -fcoverage-compilation-dir=/src/build -resource-dir /usr/lib/llvm-18/lib/clang/18 -Winvalid-pch -std=gnu++23 -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fmodule-file=mod:part1=CMakeFiles/pch-cppm-problem-repro.dir/mod-part1.pcm -fmodule-file=mod:part2=CMakeFiles/pch-cppm-problem-repro.dir/mod-part2.pcm -fmodule-file=mod:part3=CMakeFiles/pch-cppm-problem-repro.dir/mod-part3.pcm -fmodule-file=mod:part4=CMakeFiles/pch-cppm-problem-repro.dir/mod-part4.pcm -fskip-odr-check-in-gmf -include-pch /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx.pch -include /src/build/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch.hxx -target-feature +outline-atomics -target-feature -fmv -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o -x pcm CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm
 #<!-- -->0 0x0000ffff7698d398 llvm::sys::PrintStackTrace(llvm::raw_ostream&amp;, int) (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9d398)
 #<!-- -->1 0x0000ffff7698b5a8 llvm::sys::RunSignalHandlers() (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9b5a8)
 #<!-- -->2 0x0000ffff7698dafc (/usr/lib/llvm-18/bin/../lib/libLLVM.so.18.1+0xd9dafc)
 #<!-- -->3 0x0000ffff80cdf790 (linux-vdso.so.1+0x790)
 #<!-- -->4 0x0000ffff7f5465d0 clang::ASTReader::getLocalModuleFile(clang::serialization::ModuleFile&amp;, unsigned int) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x27065d0)
 #<!-- -->5 0x0000ffff7f5c6e90 (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2786e90)
 #<!-- -->6 0x0000ffff7f5c5bc0 clang::ASTReader::loadDeclUpdateRecords(clang::ASTReader::PendingUpdateRecord&amp;) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2785bc0)
 #<!-- -->7 0x0000ffff7f575f44 clang::ASTReader::finishPendingActions() (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2735f44)
 #<!-- -->8 0x0000ffff7f578ed8 clang::ASTReader::FinishedDeserializing() (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x2738ed8)
 #<!-- -->9 0x0000ffff7f55f79c clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, clang::serialization::ModuleFile**) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x271f79c)
#<!-- -->10 0x0000ffff7f6f9250 clang::ASTUnit::LoadFromASTFile(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt;&gt; const&amp;, clang::PCHContainerReader const&amp;, clang::ASTUnit::WhatToLoad, llvm::IntrusiveRefCntPtr&lt;clang::DiagnosticsEngine&gt;, clang::FileSystemOptions const&amp;, std::shared_ptr&lt;clang::HeaderSearchOptions&gt;, bool, clang::CaptureDiagsKind, bool, bool, llvm::IntrusiveRefCntPtr&lt;llvm::vfs::FileSystem&gt;) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x28b9250)
#<!-- -->11 0x0000ffff7f77cc7c clang::FrontendAction::BeginSourceFile(clang::CompilerInstance&amp;, clang::FrontendInputFile const&amp;) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x293cc7c)
#<!-- -->12 0x0000ffff7f710580 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&amp;) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x28d0580)
#<!-- -->13 0x0000ffff7f7f8e4c clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/usr/lib/llvm-18/bin/../lib/libclang-cpp.so.18.1+0x29b8e4c)
#<!-- -->14 0x0000aaaaacaf21c4 cc1_main(llvm::ArrayRef&lt;char const*&gt;, char const*, void*) (/usr/lib/llvm-18/bin/clang+0x121c4)
#<!-- -->15 0x0000aaaaacaefbac (/usr/lib/llvm-18/bin/clang+0xfbac)
#<!-- -->16 0x0000aaaaacaeed00 clang_main(int, char**, llvm::ToolContext const&amp;) (/usr/lib/llvm-18/bin/clang+0xed00)
#<!-- -->17 0x0000aaaaacafae40 main (/usr/lib/llvm-18/bin/clang+0x1ae40)
#<!-- -->18 0x0000ffff757d84c4 (/lib/aarch64-linux-gnu/libc.so.6+0x284c4)
#<!-- -->19 0x0000ffff757d8598 __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x28598)
#<!-- -->20 0x0000aaaaacaec970 _start (/usr/lib/llvm-18/bin/clang+0xc970)
clang: error: unable to execute command: Segmentation fault
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-18/bin
clang: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/Mod-9d95fb.cppm
clang: note: diagnostic msg: /tmp/Mod-9d95fb.sh
clang: note: diagnostic msg: 

********************
ninja: build stopped: subcommand failed. ```


</details>

Link for reproduction in compiler explorer: https://godbolt.org/z/cMhdaTszb
Compiler: x86-64 clang assertions trunk

I'm trying to look at this. Notable, this happens during processing of the first (and only) input file which is a .pcm file. Even trying to dump info about this .pcm file with -module-file-info segfaults clang.

Command line:
clang -g -std=gnu++23 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk -fcolor-diagnostics -Winvalid-pch -Xarch_arm64 -include/Users/shader/CLionProjects/llvm-project/pch-cppm-problem-repro/cmake-build-debug/CMakeFiles/pch-cppm-problem-repro.dir/cmake_pch_arm64.hxx @CMakeFiles/pch-cppm-problem-repro.dir/Mod.cppm.o.modmap -x pcm -module-file-info CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm

Offending .pcm file attached
mod.pcm.zip
(from a different compilation pass): module-file-info-mod-pcm.txt

When in Mod.cppm I remove the last export import :part4; suddenly the build succeeds. The mod.pcm file looks like this in that situation:
mod.pcm-successful.zip
module-file-info-mod-successful-pcm.txt

I have no idea if this actually fixes it, but this makes the build finish successfully in the test project:

Subject: [PATCH] Try to prevent segfault (#105994)
---
Index: clang/lib/Serialization/ASTReader.cpp
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/clang/lib/Serialization/ASTReader.cpp b/clang/lib/Serialization/ASTReader.cpp
--- a/clang/lib/Serialization/ASTReader.cpp	(revision 33f3ebc86e7d3afcb65c551feba5bbc2421b42ed)
+++ b/clang/lib/Serialization/ASTReader.cpp	(revision 2d7041961e94ed641bb372f1281f433dd5b72262)
@@ -9187,6 +9187,9 @@
     auto I = GlobalSubmoduleMap.find(getGlobalSubmoduleID(M, ID >> 1));
     return I == GlobalSubmoduleMap.end() ? nullptr : I->second;
   } else {
+    if(ID == (std::numeric_limits<decltype(ID)>::max()-1)) {
+      return nullptr;
+    }
     // It's a prefix (preamble, PCH, ...). Look it up by index.
     unsigned IndexFromEnd = ID >> 1;
     assert(IndexFromEnd && "got reference to unknown module file");

Try_to_prevent_segfault_#105994.patch

-module-file-info for mod.pcm after applying this patch gets outputted successfully. It would probably be good to somehow check if this doesn't still contain unwanted garbage
module-file-info-mod-successful-pcm.txt

Well, all 4 partitions are exported. But previously, the PCH was mentioned here. Now it isn't. But maybe that's desirable? Because a PCH is not a module partition (I would think)

Information for module file 'CMakeFiles/pch-cppm-problem-repro.dir/mod.pcm':
  Module format: raw
  ====== C++20 Module structure ======
  Interface Unit 'mod' is the Primary Module at index #5
   Exports:
    Partition Interface 'mod:part1' is at index #1
    Partition Interface 'mod:part2' is at index #2
    Partition Interface 'mod:part3' is at index #3
    Partition Interface 'mod:part4' is at index #4