LLDB can't find function containing a PC from musl libc.so

[rprichard]

Bugzilla Link	51466
Version	unspecified
OS	Linux
CC	@JDevlieghere

Extended Description

On Debian, I compile hello world using "musl-gcc", which links dynamically against musl's libc.

$ cat hello.c
#include <stdio.h>

int main() {
printf("hi\n");
return 0;
}

$ musl-gcc hello.c -g

$ file a.out
a.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped

The /lib/ld-musl-x86_64.so.1 interpreter is really a symlink to libc.so. (With musl, the loader and libc.so are the same file.)

$ realpath /lib/ld-musl-x86_64.so.1
/usr/lib/x86_64-linux-musl/libc.so

$ /x/clang12/bin/lldb a.out
(lldb) target create "a.out"
Current executable set to '/x/mess/a.out' (x86_64).
(lldb) b main
Breakpoint 1: where = a.out`main + 4 at hello.c:4:3, address = 0x0000000000001159
(lldb) run
Process 627144 launched: '/x/mess/a.out' (x86_64)
Process 627144 stopped

• thread #​1, name = 'a.out', stop reason = breakpoint 1.1
  frame #​0: 0x0000555555555159 a.outmain at hello.c:4:3 1 #include <stdio.h> 2 3 int main() { -> 4 printf("hi\n"); 5 return 0; 6 } (lldb) disas -n exit libc.soexit:
  0x7ffff7f5e090 <+0>: pushq %rbp
  0x7ffff7f5e091 <+1>: movl %edi, %ebp
  0x7ffff7f5e093 <+3>: callq 0x7ffff7f683e0
  0x7ffff7f5e098 <+8>: callq 0x7ffff7fbfea0
  0x7ffff7f5e09d <+13>: xorl %eax, %eax
  0x7ffff7f5e09f <+15>: callq 0x7ffff7fa4aa0
  0x7ffff7f5e0a4 <+20>: movl %ebp, %edi
  0x7ffff7f5e0a6 <+22>: callq 0x7ffff7f68210
  (lldb) disas -a 0x7ffff7f5e090
  error: Could not find function bounds for address 0x7ffff7f5e090

(lldb) image show-unwind -n exit
UNWIND PLANS for libc.so`exit (start addr 0x7ffff7f5e090)

Asynchronous (not restricted to call-sites) UnwindPlan is 'assembly insn profiling'
...

(lldb) image show-unwind -a 0x7ffff7f5e090
error: no unwind data found that matches '0x7ffff7f5e090'.

---

I think the problem is that "duplicate ld.so instance" special case in DynamicLoaderPOSIXDYLD.cpp:

  ModuleSP module_sp =
      LoadModuleAtAddress(I->file_spec, I->link_addr, I->base_addr, true);
  if (module_sp.get()) {
    if (module_sp->GetObjectFile()->GetBaseAddress().GetLoadAddress(
            &m_process->GetTarget()) == m_interpreter_base &&
        module_sp != m_interpreter_module.lock()) {
      // If this is a duplicate instance of ld.so, unload it.  We may end up
      // with it if we load it via a different path than before (symlink
      // vs real path).
      // TODO: remove this once we either fix library matching or avoid
      // loading the interpreter when setting the rendezvous breakpoint.
      UnloadSections(module_sp);
      loaded_modules.Remove(module_sp);
      continue;
    }

    loaded_modules.AppendIfNeeded(module_sp);
    new_modules.Append(module_sp);
  }

Adding the duplicate musl module should replace the initial interpreter module in SectionLoadList. Unloading the duplicate will remove the musl libc from the SectionLoadList without restoring the original.

Here's some log output that might help:

rprichard@cashew:/x/mess$ /x/clang12/bin/lldb a.out
(lldb) target create "a.out"
Current executable set to '/x/mess/a.out' (x86_64).
(lldb) log enable -v lldb dyld
(lldb) b main
Breakpoint 1: where = a.out`main + 4 at hello.c:4:3, address = 0x0000000000001159
(lldb) run
DynamicLoaderDarwin::UseDYLDSPI: Use old DynamicLoader plugin
DynamicLoaderDarwin::UseDYLDSPI: Use old DynamicLoader plugin
DYLDRendezvous::UpdateExecutablePath exe module executable path set: '/x/mess/a.out'
DynamicLoaderPOSIXDYLD::DidLaunch()
(section = 0x00000000016E1460 (/x/mess/a.out.PT_LOAD[0]), load_addr = 0x555555554000) module = 0x000000000178DE80
(section = 0x0000000001799370 (/x/mess/a.out.PT_LOAD[1]), load_addr = 0x555555555000) module = 0x000000000178DE80
(section = 0x000000000179A370 (/x/mess/a.out.PT_LOAD[2]), load_addr = 0x555555556000) module = 0x000000000178DE80
(section = 0x000000000179A5B0 (/x/mess/a.out.PT_LOAD[3]), load_addr = 0x555555557e18) module = 0x000000000178DE80
DynamicLoaderPOSIXDYLD::DidLaunch about to call ProbeEntry()
Rendezvous structure is not set up yet. Trying to locate rendezvous breakpoint in the interpreter by symbol name.
(section = 0x0000000001803A80 (/usr/lib/x86_64-linux-musl/libc.so.PT_LOAD[0]), load_addr = 0x7ffff7f49000) module = 0x0000000001826080
(section = 0x0000000001803C00 (/usr/lib/x86_64-linux-musl/libc.so.PT_LOAD[1]), load_addr = 0x7ffff7f5e000) module = 0x0000000001826080
(section = 0x0000000001826B20 (/usr/lib/x86_64-linux-musl/libc.so.PT_LOAD[2]), load_addr = 0x7ffff7fc3000) module = 0x0000000001826080
(section = 0x0000000001826BE0 (/usr/lib/x86_64-linux-musl/libc.so.PT_LOAD[3]), load_addr = 0x7ffff7ffab20) module = 0x0000000001826080
Successfully set rendezvous breakpoint at address 0x7ffff7fbd560 for pid 629373
(section = 0x0000000001663A40 ([vdso].PT_LOAD[0]), load_addr = 0x7ffff7f47000) module = 0x000000000184A4A0
Process 629373 launched: '/x/mess/a.out' (x86_64)
DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit called for pid 629373
DYLDRendezvous::Resolve address size: 8, padding 4
ResolveRendezvousAddress info_location = 0xffffffffffffffff
ResolveRendezvousAddress resolved via direct object file approach to 0x555555557f00
ResolveRendezvousAddress reading pointer (8 bytes) from 0x555555557f00
DYLDRendezvous::Resolve cursor = 0x7ffff7ffd880
(section = 0x00000000016E1460 (/x/mess/a.out.PT_LOAD[0]), load_addr = 0x555555554000) module = 0x000000000178DE80
(section = 0x0000000001799370 (/x/mess/a.out.PT_LOAD[1]), load_addr = 0x555555555000) module = 0x000000000178DE80
(section = 0x000000000179A370 (/x/mess/a.out.PT_LOAD[2]), load_addr = 0x555555556000) module = 0x000000000178DE80
(section = 0x000000000179A5B0 (/x/mess/a.out.PT_LOAD[3]), load_addr = 0x555555557e18) module = 0x000000000178DE80
(section = 0x00007FE778008E30 (/lib/ld-musl-x86_64.so.1.PT_LOAD[0]), load_addr = 0x7ffff7f49000) module = 0x00007FE77800B170
(section = 0x00007FE77800BFE0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[1]), load_addr = 0x7ffff7f5e000) module = 0x00007FE77800B170
(section = 0x00007FE77800C0A0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[2]), load_addr = 0x7ffff7fc3000) module = 0x00007FE77800B170
(section = 0x00007FE77800C160 (/lib/ld-musl-x86_64.so.1.PT_LOAD[3]), load_addr = 0x7ffff7ffab20) module = 0x00007FE77800B170
SectionLoadList::SetSectionUnloaded (section = 0x7fe778008e30 (/lib/ld-musl-x86_64.so.1.PT_LOAD[0]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800bfe0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[1]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800c0a0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[2]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800c160 (/lib/ld-musl-x86_64.so.1.PT_LOAD[3]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800dea0 (/lib/ld-musl-x86_64.so.1..gnu_debuglink))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800e000 (/lib/ld-musl-x86_64.so.1..shstrtab))
SectionLoadList::SetSectionUnloaded (section = 0x7fe778008e30 (/lib/ld-musl-x86_64.so.1.PT_LOAD[0]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800bfe0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[1]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800c0a0 (/lib/ld-musl-x86_64.so.1.PT_LOAD[2]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800c160 (/lib/ld-musl-x86_64.so.1.PT_LOAD[3]))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800dea0 (/lib/ld-musl-x86_64.so.1..gnu_debuglink))
SectionLoadList::SetSectionUnloaded (section = 0x7fe77800e000 (/lib/ld-musl-x86_64.so.1..shstrtab))
DynamicLoaderPOSIXDYLD::RendezvousBreakpointHit pid 629373 stop_when_images_change=false
Process 629373 stopped

• thread #​1, name = 'a.out', stop reason = breakpoint 1.1
  frame #​0: 0x0000555555555159 a.out`main at hello.c:4:3
  1 #include <stdio.h>
  2
  3 int main() {
  -> 4 printf("hi\n");
  5 return 0;
  6 }

It looks like 0x7ffff7f49000 (the first segment of musl libc.so / ld.so) is first loaded by LoadInterpreterModule, then loaded through the ordinary code path (for /lib/ld-musl-x86_64.so.1), then unloaded (twice?).

---

Aside: There is a Debian musl-dbgsym package that provides a /usr/lib/debug/.build-id/5f/4d26469932e5b7acdcf03ed224ff9b10ab15c0.debug file. If the Debian package is installed, then "image show-unwind -n exit" instead reports:

Asynchronous (not restricted to call-sites) UnwindPlan is 'DWARF CFI'
Synchronous (restricted to call-sites) UnwindPlan is 'DWARF CFI'

[DavidSpickett]

The situation here may have been changed by #69932, which fixed a similar issue on Armv7 Linux.

The problem there was the same, we would load an SO just to see if it was the same, but there's only one copy of the sections for it. Might have been the same issue here.