Open
Description
We are seeing some issues with dsymutil on arm32 with your Linaro CI bots when we moved to Ubuntu 24 containters:
LLVM :: tools/dsymutil/ARM/DWARFLinkerParallel/accel-imported-declarations.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-fwd-declaration2.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-fwd-declaration3.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-member-functions.cpp
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-nested-types2.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-predictable-output2.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-string.test
LLVM :: tools/dsymutil/X86/DWARFLinkerParallel/odr-uniquing.cpp
We first thought it was related to the system libraries since we didn't see it on Ubuntu 22 containers. Investigating one of the failures, odr-member-functions.cpp, it seems that it is something related to parallelization:
$ gdb --args /home/adhemerval.zanella/projects/llvm/llvm-build-32/bin/dsymutil --linker=parallel -f -oso-prepend-path=/home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions -y /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../dummy-debug-map.map -o /home/adhemerval.zanella/projects/llvm/llvm-build-32/test/tools/dsymutil/X86/DWARFLinkerParallel/Output/odr-member-functions.cpp.tmp1.out
[...]
Thread 5 "dsymutil" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf5e403e0 (LWP 924523)]
rllvm::dwarf_linker::parallel::CompileUnit::allocateTypeDie (this=<optimized out>, TypeDescriptor=TypeDescriptor@entry=0x0, TypeDIEGenerator=...,
DieTag=DieTag@entry=llvm::dwarf::DW_TAG_subprogram, IsDeclaration=<optimized out>, IsParentDeclaration=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1425
1425 DIE *DefinitionDie = TypeDescriptor->Die;
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) n
Program not restarted.
(gdb) bt
#0 llvm::dwarf_linker::parallel::CompileUnit::allocateTypeDie (this=<optimized out>, TypeDescriptor=TypeDescriptor@entry=0x0, TypeDIEGenerator=...,
DieTag=DieTag@entry=llvm::dwarf::DW_TAG_subprogram, IsDeclaration=<optimized out>, IsParentDeclaration=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1425
#1 0x02445d18 in llvm::dwarf_linker::parallel::CompileUnit::createTypeDIEandCloneAttributes (this=0xc6800620, InputDieEntry=InputDieEntry@entry=0xc6900898,
TypeDIEGenerator=..., ClonedParentTypeDIE=<optimized out>, ArtificialTypeUnit=0xe5500600)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1490
#2 0x02444c54 in llvm::dwarf_linker::parallel::CompileUnit::cloneDIE (this=0xc6800620, InputDieEntry=InputDieEntry@entry=0xc6900898, ClonedParentTypeDIE=0xc6a05058,
OutOffset=46, FuncAddressAdjustment=std::optional [no contained value], VarAddressAdjustment=std::optional [no contained value], Allocator=..., ArtificialTypeUnit=0xe5500600)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1316
#3 0x02444d7c in llvm::dwarf_linker::parallel::CompileUnit::cloneDIE (this=0xc6800620, InputDieEntry=InputDieEntry@entry=0xc6900880, ClonedParentTypeDIE=0xc6a05058,
OutOffset=<optimized out>, FuncAddressAdjustment=std::optional [no contained value], VarAddressAdjustment=std::optional [no contained value], Allocator=...,
ArtificialTypeUnit=0xe5500600) at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1337
#4 0x02444d7c in llvm::dwarf_linker::parallel::CompileUnit::cloneDIE (this=this@entry=0xc6800620, InputDieEntry=<optimized out>, ClonedParentTypeDIE=0xc9e04190,
OutOffset=<optimized out>, FuncAddressAdjustment=std::optional [no contained value], VarAddressAdjustment=std::optional [no contained value], Allocator=...,
ArtificialTypeUnit=0xe5500600) at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1337
#5 0x024448f8 in llvm::dwarf_linker::parallel::CompileUnit::cloneAndEmit (this=0xc6800620, TargetTriple=std::optional = {...}, ArtificialTypeUnit=0xe5500600)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1245
#6 0x024332ec in llvm::dwarf_linker::parallel::DWARFLinkerImpl::LinkContext::linkSingleCompileUnit(llvm::dwarf_linker::parallel::CompileUnit&, llvm::dwarf_linker::parallel::TypeUnit*, llvm::dwarf_linker::parallel::CompileUnit::Stage)::$_0::operator()() const (this=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:672
#7 llvm::function_ref<llvm::Expected<bool> ()>::callback_fn<llvm::dwarf_linker::parallel::DWARFLinkerImpl::LinkContext::linkSingleCompileUnit(llvm::dwarf_linker::parallel::CompileUnit&, llvm::dwarf_linker::parallel::TypeUnit*, llvm::dwarf_linker::parallel::CompileUnit::Stage)::$_0>(int) (callable=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:46
#8 llvm::function_ref<llvm::Expected<bool> ()>::operator()() const (this=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69
#9 llvm::dwarf_linker::finiteLoop(llvm::function_ref<llvm::Expected<bool> ()>, unsigned int) (MaxCounter=100000, Iteration=...)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/DWARFLinker/Utils.h:30
#10 llvm::dwarf_linker::parallel::DWARFLinkerImpl::LinkContext::linkSingleCompileUnit (this=0x74002a8, CU=..., ArtificialTypeUnit=<optimized out>,
DoUntilStage=llvm::dwarf_linker::parallel::CompileUnit::Stage::Cleaned)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:597
#11 0x026289bc in llvm::function_ref<void (unsigned int)>::operator()(unsigned int) const (this=<optimized out>, params=0)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69
#12 llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1::operator()() const (this=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:232
#13 std::__invoke_impl<void, llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1&>(std::__invoke_other, llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1&) (__f=...) at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/invoke.h:61
#14 std::__invoke_r<void, llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1&>(llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1&) (__fn=...) at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/invoke.h:111
#15 std::_Function_handler<void (), llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1>::_M_invoke(std::_Any_data const&) (__functor=...)
at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/std_function.h:290
#16 0x026287c4 in std::function<void ()>::operator()() const (this=0x7400cf8) at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/std_function.h:591
#17 llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0::operator()() const (this=0x7400cf8)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:199
#18 std::__invoke_impl<void, llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0&>(std::__invoke_other, llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0&)
(__f=...) at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/invoke.h:61
#19 std::__invoke_r<void, llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0&>(llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0&) (__fn=...)
at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/invoke.h:111
#20 std::_Function_handler<void (), llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) (__functor=...)
at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/std_function.h:290
#21 0x026282d4 in std::function<void ()>::operator()() const (this=0xf5e3fd18) at /usr/bin/../lib/gcc/arm-linux-gnueabihf/14/../../../../include/c++/14/bits/std_function.h:591
#22 llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work (
this=0x5295f40 <llvm::parallel::detail::(anonymous namespace)::Executor::getDefaultExecutor()::Exec>, S=..., ThreadID=<optimized out>)
at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:122
#23 0xf7ca4f16 in ?? () from /lib/arm-linux-gnueabihf/libstdc++.so.6
#24 0xf7b24f46 in start_thread (arg=0x660b829c) at pthread_create.c:447
#25 0xf7b78960 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:74 from /lib/arm-linux-gnueabihf/libc.so.6
And valgrind also shows a similar issue:
$ valgrind /home/adhemerval.zanella/projects/llvm/llvm-build-32/bin/dsymutil --linker=parallel -f -oso-prepend-path=/home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions -y /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../dummy-debug-map.map -o /home/adhemerval.zanella/projects/llvm/llvm-build-32/test/tools/dsymutil/X86/DWARFLinkerParallel/Output/odr-member-functions.cpp.tmp1.out
==924618== Memcheck, a memory error detector
==924618== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==924618== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==924618== Command: /home/adhemerval.zanella/projects/llvm/llvm-build-32/bin/dsymutil --linker=parallel -f -oso-prepend-path=/home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions -y /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../dummy-debug-map.map -o /home/adhemerval.zanella/projects/llvm/llvm-build-32/test/tools/dsymutil/X86/DWARFLinkerParallel/Output/odr-member-functions.cpp.tmp1.out
==924618==
warning: Unable to open /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o No such file or directory
warning: /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o: No such file or directory
note: while processing /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o
==924618== Thread 16:
==924618== Invalid read of size 4
==924618== at 0x1B83728: load (atomic_base.h:833)
==924618== by 0x1B83728: operator llvm::DIE * (atomic_base.h:707)
==924618== by 0x1B83728: operator llvm::DIE * (atomic:436)
==924618== by 0x1B83728: llvm::dwarf_linker::parallel::CompileUnit::allocateTypeDie(llvm::dwarf_linker::parallel::TypeEntryBody*, llvm::dwarf_linker::parallel::DIEGenerator&, llvm::dwarf::Tag, bool, bool) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1425)
==924618== by 0x1B82D17: llvm::dwarf_linker::parallel::CompileUnit::createTypeDIEandCloneAttributes(llvm::DWARFDebugInfoEntry const*, llvm::dwarf_linker::parallel::DIEGenerator&, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1490)
==924618== by 0x1B81C53: llvm::dwarf_linker::parallel::CompileUnit::cloneDIE(llvm::DWARFDebugInfoEntry const*, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, unsigned long long, std::optional<long long>, std::optional<long long>, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096u, 4096u, 128u>&, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1316)
==924618== by 0x1B81D7B: llvm::dwarf_linker::parallel::CompileUnit::cloneDIE(llvm::DWARFDebugInfoEntry const*, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, unsigned long long, std::optional<long long>, std::optional<long long>, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096u, 4096u, 128u>&, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1337)
==924618== by 0x1B818F7: llvm::dwarf_linker::parallel::CompileUnit::cloneAndEmit(std::optional<std::reference_wrapper<llvm::Triple const> >, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1245)
==924618== by 0x1B702EB: operator() (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:672)
==924618== by 0x1B702EB: callback_fn<(lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:597:30)> (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:46)
==924618== by 0x1B702EB: operator() (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69)
==924618== by 0x1B702EB: finiteLoop (llvm-project-git/llvm/include/llvm/DWARFLinker/Utils.h:30)
==924618== by 0x1B702EB: llvm::dwarf_linker::parallel::DWARFLinkerImpl::LinkContext::linkSingleCompileUnit(llvm::dwarf_linker::parallel::CompileUnit&, llvm::dwarf_linker::parallel::TypeUnit*, llvm::dwarf_linker::parallel::CompileUnit::Stage) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:597)
==924618== by 0x1D659BB: operator() (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69)
==924618== by 0x1D659BB: operator() (llvm-project-git/llvm/lib/Support/Parallel.cpp:232)
==924618== by 0x1D659BB: __invoke_impl<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:230:16) &> (invoke.h:61)
==924618== by 0x1D659BB: __invoke_r<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:230:16) &> (invoke.h:111)
==924618== by 0x1D659BB: std::_Function_handler<void (), llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1>::_M_invoke(std::_Any_data const&) (std_function.h:290)
==924618== by 0x1D657C3: operator() (std_function.h:591)
==924618== by 0x1D657C3: operator() (llvm-project-git/llvm/lib/Support/Parallel.cpp:199)
==924618== by 0x1D657C3: __invoke_impl<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:198:49) &> (invoke.h:61)
==924618== by 0x1D657C3: __invoke_r<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:198:49) &> (invoke.h:111)
==924618== by 0x1D657C3: std::_Function_handler<void (), llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) (std_function.h:290)
==924618== by 0x1D652D3: operator() (std_function.h:591)
==924618== by 0x1D652D3: llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work(llvm::ThreadPoolStrategy, unsigned int) (llvm-project-git/llvm/lib/Support/Parallel.cpp:122)
==924618== by 0x5346F15: ??? (in /usr/lib/arm-linux-gnueabihf/libstdc++.so.6.0.33)
==924618== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==924618==
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
==924618==
==924618== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==924618== Access not within mapped region at address 0x0
==924618== at 0x1B83728: load (atomic_base.h:833)
==924618== by 0x1B83728: operator llvm::DIE * (atomic_base.h:707)
==924618== by 0x1B83728: operator llvm::DIE * (atomic:436)
==924618== by 0x1B83728: llvm::dwarf_linker::parallel::CompileUnit::allocateTypeDie(llvm::dwarf_linker::parallel::TypeEntryBody*, llvm::dwarf_linker::parallel::DIEGenerator&, llvm::dwarf::Tag, bool, bool) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1425)
==924618== by 0x1B82D17: llvm::dwarf_linker::parallel::CompileUnit::createTypeDIEandCloneAttributes(llvm::DWARFDebugInfoEntry const*, llvm::dwarf_linker::parallel::DIEGenerator&, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1490)
==924618== by 0x1B81C53: llvm::dwarf_linker::parallel::CompileUnit::cloneDIE(llvm::DWARFDebugInfoEntry const*, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, unsigned long long, std::optional<long long>, std::optional<long long>, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096u, 4096u, 128u>&, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1316)
==924618== by 0x1B81D7B: llvm::dwarf_linker::parallel::CompileUnit::cloneDIE(llvm::DWARFDebugInfoEntry const*, llvm::StringMapEntry<std::atomic<llvm::dwarf_linker::parallel::TypeEntryBody*> >*, unsigned long long, std::optional<long long>, std::optional<long long>, llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096u, 4096u, 128u>&, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1337)
==924618== by 0x1B818F7: llvm::dwarf_linker::parallel::CompileUnit::cloneAndEmit(std::optional<std::reference_wrapper<llvm::Triple const> >, llvm::dwarf_linker::parallel::TypeUnit*) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp:1245)
==924618== by 0x1B702EB: operator() (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:672)
==924618== by 0x1B702EB: callback_fn<(lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:597:30)> (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:46)
==924618== by 0x1B702EB: operator() (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69)
==924618== by 0x1B702EB: finiteLoop (llvm-project-git/llvm/include/llvm/DWARFLinker/Utils.h:30)
==924618== by 0x1B702EB: llvm::dwarf_linker::parallel::DWARFLinkerImpl::LinkContext::linkSingleCompileUnit(llvm::dwarf_linker::parallel::CompileUnit&, llvm::dwarf_linker::parallel::TypeUnit*, llvm::dwarf_linker::parallel::CompileUnit::Stage) (llvm-project-git/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp:597)
==924618== by 0x1D659BB: operator() (llvm-project-git/llvm/include/llvm/ADT/STLFunctionalExtras.h:69)
==924618== by 0x1D659BB: operator() (llvm-project-git/llvm/lib/Support/Parallel.cpp:232)
==924618== by 0x1D659BB: __invoke_impl<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:230:16) &> (invoke.h:61)
==924618== by 0x1D659BB: __invoke_r<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:230:16) &> (invoke.h:111)
==924618== by 0x1D659BB: std::_Function_handler<void (), llvm::parallelFor(unsigned int, unsigned int, llvm::function_ref<void (unsigned int)>)::$_1>::_M_invoke(std::_Any_data const&) (std_function.h:290)
==924618== by 0x1D657C3: operator() (std_function.h:591)
==924618== by 0x1D657C3: operator() (llvm-project-git/llvm/lib/Support/Parallel.cpp:199)
==924618== by 0x1D657C3: __invoke_impl<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:198:49) &> (invoke.h:61)
==924618== by 0x1D657C3: __invoke_r<void, (lambda at /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/Support/Parallel.cpp:198:49) &> (invoke.h:111)
==924618== by 0x1D657C3: std::_Function_handler<void (), llvm::parallel::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) (std_function.h:290)
==924618== by 0x1D652D3: operator() (std_function.h:591)
==924618== by 0x1D652D3: llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work(llvm::ThreadPoolStrategy, unsigned int) (llvm-project-git/llvm/lib/Support/Parallel.cpp:122)
==924618== by 0x5346F15: ??? (in /usr/lib/arm-linux-gnueabihf/libstdc++.so.6.0.33)
==924618== If you believe this happened as a result of a stack
==924618== overflow in your program's main thread (unlikely but
==924618== possible), you can try to increase the size of the
==924618== main thread stack using the --main-stacksize= flag.
==924618== The main thread stack size used in this run was 8388608.
I also check if increasing the default thread stack might help, but even with 16MB I still see issues. The failures are intermittent, which leads me to believe it is some kind of data race issue.
As an experiment, I tried to also check on aarch64 with -fsanitize=thread enable and running the same test does some something wrong:
$ bin/dsymutil --linker=parallel -f -oso-prepend-path=/home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions -y /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../dummy-debug-map.map -o /home/adhemerval.zanella/projects/llvm/llvm-build/test/tools/dsymutil/X86/DWARFLinkerParallel/Output/odr-member-functions.cpp.tmp1.out
warning: Unable to open /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o No such file or directory
warning: /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o: No such file or directory
note: while processing /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/../../Inputs/odr-member-functions/4.o
==================
WARNING: ThreadSanitizer: data race (pid=3559441)
Read of size 1 at 0xffff7fd028e8 by thread T4:
#0 memcpy <null> (dsymutil+0x34d71c) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#1 void llvm::SmallVectorTemplateBase<llvm::DWARFExpression::Operation::Encoding, true>::uninitialized_copy<llvm::DWARFExpression::Operation::Encoding const, llvm::DWARFExpression::Operation::Encoding>(llvm::DWARFExpression::Operation::Encoding const*, llvm::DWARFExpression::Operation::Encoding const*, llvm::DWARFExpression::Operation::Encoding*, std::enable_if<std::is_same<std::remove_const<llvm::DWARFExpression::Operation::Encoding const>::type, llvm::DWARFExpression::Operation::Encoding>::value, void>::type*) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/SmallVector.h:520:7 (dsymutil+0x190cf88) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#2 llvm::SmallVectorImpl<llvm::DWARFExpression::Operation::Encoding>::operator=(llvm::SmallVectorImpl<llvm::DWARFExpression::Operation::Encoding> const&) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/SmallVector.h:1043:3 (dsymutil+0x190cf88)
#3 llvm::SmallVector<llvm::DWARFExpression::Operation::Encoding, 40u>::SmallVector(llvm::SmallVector<llvm::DWARFExpression::Operation::Encoding, 40u> const&) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/ADT/SmallVector.h:1238:27 (dsymutil+0x190cf88)
#4 llvm::DWARFExpression::Operation::Description::Description(llvm::DWARFExpression::Operation::Description const&) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/include/llvm/DebugInfo/DWARF/DWARFExpression.h:66:12 (dsymutil+0x190cf88)
#5 llvm::getDescImpl(llvm::ArrayRef<llvm::DWARFExpression::Operation::Description>, unsigned int) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:118:10 (dsymutil+0x190cf88)
#6 llvm::getOpDesc(unsigned int) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:123:10 (dsymutil+0x190cf88)
#7 llvm::DWARFExpression::Operation::extract(llvm::DataExtractor, unsigned char, unsigned long, std::optional<llvm::dwarf::DwarfFormat>) /home/adhemerval.zanella/projects/llvm/llvm-project-git/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:149:10 (dsymutil+0x190cf88)
Previous write of size 8 at 0xffff7fd028e8 by thread T3:
#0 operator new(unsigned long) <null> (dsymutil+0x3c5d54) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#1 __gnu_cxx::new_allocator<llvm::DWARFExpression::Operation::Description>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27 (dsymutil+0x1910a78) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#2 std::allocator_traits<std::allocator<llvm::DWARFExpression::Operation::Description> >::allocate(std::allocator<llvm::DWARFExpression::Operation::Description>&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20 (dsymutil+0x1910a78)
#3 std::_Vector_base<llvm::DWARFExpression::Operation::Description, std::allocator<llvm::DWARFExpression::Operation::Description> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20 (dsymutil+0x1910a78)
#4 std::vector<llvm::DWARFExpression::Operation::Description, std::allocator<llvm::DWARFExpression::Operation::Description> >::_M_default_append(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:635:34 (dsymutil+0x1910a78)
Location is heap block of size 18360 at 0xffff7fd00000 allocated by thread T3:
#0 operator new(unsigned long) <null> (dsymutil+0x3c5d54) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#1 __gnu_cxx::new_allocator<llvm::DWARFExpression::Operation::Description>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/ext/new_allocator.h:127:27 (dsymutil+0x1910a78) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#2 std::allocator_traits<std::allocator<llvm::DWARFExpression::Operation::Description> >::allocate(std::allocator<llvm::DWARFExpression::Operation::Description>&, unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:464:20 (dsymutil+0x1910a78)
#3 std::_Vector_base<llvm::DWARFExpression::Operation::Description, std::allocator<llvm::DWARFExpression::Operation::Description> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:346:20 (dsymutil+0x1910a78)
#4 std::vector<llvm::DWARFExpression::Operation::Description, std::allocator<llvm::DWARFExpression::Operation::Description> >::_M_default_append(unsigned long) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/vector.tcc:635:34 (dsymutil+0x1910a78)
Thread T4 (tid=3559446, running) created by thread T1 at:
#0 pthread_create <null> (dsymutil+0x3421c4) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd3320) (BuildId: a012b2bb77110e84b266cd7425b50e57427abb02)
Thread T3 (tid=3559445, running) created by thread T1 at:
#0 pthread_create <null> (dsymutil+0x3421c4) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1)
#1 std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) <null> (libstdc++.so.6+0xd3320) (BuildId: a012b2bb77110e84b266cd7425b50e57427abb02)
SUMMARY: ThreadSanitizer: data race (/home/adhemerval.zanella/projects/llvm/llvm-build/bin/dsymutil+0x34d71c) (BuildId: dff61953132f7f1ecaa8c12a46c5287e076c74c1) in __interceptor_memcpy
==================
ThreadSanitizer: reported 1 warnings