Use dl_iterate_phdr on Haiku

[bjorn3]

An implementation or dl_iterate_phdr was added to Haiku in haiku/haiku@908107a a little over a year after Haiku support got added to backtrace-rs.

[bjorn3]

cc @nielx

[workingjubilee]

We don't have an official Haiku maintainer enrolled it seems, so if either @waddlesplash or @trungnt2910 (or whoever else, basically) think this is good then I'm happy to ship it.

[waddlesplash]

Should be fine; internally dl_iterate_phdr is implemented using get_next_image_info anyway. You lose the "length" information since the phdr structs don't provide this, but that's the same as on other OSes.

Also note that dl_iterate_phdr is in libbsd.so, the "BSD-related APIs" library (as opposed to libroot.so, the C library, API primitives & syscalls, etc. where get_next_image_info lives.) I think Rust on Haiku always links in libbsd.so anyway, so that doesn't really matter in the end.

[waddlesplash]

Hmm, I just looked at the code some more, and it appears that when dl_iterate_phdr is used, "gimli" tries to read /proc/self/maps to determine size information? Haiku doesn't have /proc/ at all, so that won't work. There are APIs to get the mappings/"areas" information for the current process, but they're Haiku-specific. If this is needed, in the end it may just make sense to keep the Haiku-specific get_next_image_info which provides map size information directly instead of using dl_iterate_phdr (and perhaps add a comment indicating this.)

[nielx]

Agreed with @waddlesplash here.

[bjorn3]

Haiku doesn't have /proc/ at all, so that won't work.

Huh, why didn't the backtrace-rs tests fail on Haiku with this PR if that is the case?

[nielx]

Haiku doesn't have /proc/ at all, so that won't work.

Huh, why didn't the backtrace-rs tests fail on Haiku with this PR if that is the case?

In the PR you are disabling one of the tests. How did it fail?
What other testing have you done? The changes will affect finding symbols in libraries. Have you done a more elaborate test where you try to get a stack trace from a library?

[bjorn3]

In the PR you are disabling one of the tests. How did it fail?

This test reads the dynamic linker from the test executable and then invokes the dynamic linker as executable with the teste executable as argument. I got confirmation from @waddlesplash that this is fundamentally unsupported on Haiku.

The changes will affect finding symbols in libraries.

Ah, could well be that there are simply no tests for resolving symbols in shared libraries.

[workingjubilee]

There are but they're only run on some OS, I think?

Access to /proc/self/maps should not be hard-required, last I recall. It is merely checked because it is more accurate than certain resolution mechanisms if it is available.

[workingjubilee]

I will review the code again to confirm my understanding of our own library and then, assuming I am correct, merge this.

[waddlesplash]

There are but they're only run on some OS, I think?

Well, that should be checked on Haiku to make sure this doesn't regress.

Is it really so bad to just use the Haiku-specific API that provides this information directly?

[workingjubilee]

Well, I always prefer having more shared code to reason about, so that testing other OS also tests Haiku, you see? But if Haiku has a significant functionality gap here based on what APIs are used, such that it cannot reuse certain shared paths, then I'm happy to keep using Haiku-specific APIs. I just would kind of hope that if a function exists in its libc then calling it, uh, works?

[workingjubilee]

Oh, I see, there's two libc-ish objects, cool. That's fine, but we link in libbsd.so? Hmm. Is that desirable or not? Is libbsd.so necessary for other POSIX support, like pthreads? If so, then yes we absolutely require it forever and ever and we can just say we need both libbsd and libroot.

[waddlesplash]

I just would kind of hope that if a function exists in its libc then calling it, uh, works?

It works fine, but the dl_iterate_phdr structure provides no size information on any OS, Haiku included. You thus have to use platform-specific APIs to retrieve this on other OSes. Linux and FreeBSD support /proc for this, but Haiku does not (and neither does OpenBSD it appears). So if the sizing information is needed/wanted, then you will either have to (1) use the Haiku-specific area APIs, or (2) use the Haiku-specific shared image enumeration. So if there's Haiku-specific code being used either way, you might as well use the simpler (2).

Is libbsd.so necessary for other POSIX support, like pthreads?

No, it's not. Only "BSD-style" APIs that are not in POSIX (and which Haiku has found it useful to provide) are in libbsd.so (besides dl_iterate_phdr, there's also kqueue, now getloadavg, etc.)

[workingjubilee]

Right, okay, I had not finished my coffee before saying things. Currently reading https://www.haiku-os.org/legacy-docs/bebook/TheKernelKit_Images.html

Oh wow the image_info struct has a lot more than the comment says it does...

[workingjubilee]

I see!

If libroot.so is considered essential (it sure seems to be) and libbsd.so is considered an optional component of the OS, and Rust has no loss of functionality on the platform using libroot.so, then we would probably prefer to only link against libroot.so in actual fact. Less things to link and load is an obvious good, after all.

There's always a balance of "well, if things are greatly simplified by simply using the BSD way on Haiku, then use the BSD way", of course. But it sounds like the gains are pretty marginal, and we generally prefer to lean heavily on POSIX APIs when we are trying to maximize the cross-platformness of our code.

[waddlesplash]

Oh wow the image_info struct has a lot more than the comment says it does...

Yes, indeed. (Also note that Haiku provides the more-standard dladdr to look up symbol information at a given address.)

If libroot.so is considered essential (it sure seems to be)

Yes it is; the syscall hooks are all in libroot.so, so anything that invokes syscalls at all has to link to it.

and libbsd.so is considered an optional component of the OS

Well, it's shipped in the same base package as libroot.so, it's not "optional" in the sense that you can't ever have a (valid) Haiku install that doesn't have libbsd.so. I think even Haiku's own libnetwork.so (sockets, DNS, etc.) depends on libbsd.so internally. So there's a good chance it will get loaded regardless depending on what functionality is being used.

and we generally prefer to lean heavily on POSIX APIs when we are trying to maximize the cross-platformness of our code.

Right, makes sense; only there's no POSIX API to enumerate all mappings of a process, and I don't think there's a "generally accepted" BSD/GNU one either, i.e. that is the same on at least 2 BSDs if not also Linux libcs. (If I'm mistaken and there is one, then we could probably implement that for convenience.)

[workingjubilee]

Then it sounds like avoiding libbsd.so is also pretty marginal in gain! It's still always nice that if someone's going to link it in anyway, that it be the thing that actually needs it and not everyone else, so that things don't wind up creating an implicit dependency on something being in the address space that isn't actually required nor guaranteed, but yanno.

Right, makes sense; only there's no POSIX API to enumerate all mappings of a process, and I don't think there's a "generally accepted" BSD/GNU one either, i.e. that is the same on at least 2 BSDs if not also Linux libcs. (If I'm mistaken and there is one, then we could probably implement that for convenience.)

Yeah, I think dl_iterate_phdr is all we got... ignoring the emulation here and there of /proc/self/maps, which is often itself pretty jank and sometimes not as useful for the thing we want (precise virtual address space mapping information enriched with all metadata we care about). So let me examine things to understand better why we do what we do before I arrive at a final conclusion here, because I don't immediately recall how much information of the trace hinges on the things that only struct image_info can provide.

[waddlesplash]

(precise virtual address space mapping information enriched with all metadata we care about).

If you need/want more information here than struct image_info provides, you may want to take a look at Haiku's get_area_info API (plus area_for to get the area_id of some address. Do note that one ELF image will have multiple area_ids, corresponding to X/RW/etc. or other section divisions.)

[workingjubilee]

So, getting back to this, my main question:

It works fine, but the dl_iterate_phdr structure provides no size information on any OS, Haiku included.

It seems to me that p_memsz in Elf64_Phdr does in fact provide size information. However, I am aware of a nuance here, as not all the addresses we are getting perfectly correspond, because some things don't take into account that section starts may have to be page-aligned by dynamic loading. Likewise, some things get multiple mappings. So the exact location of something gets a bit arbitrary. Is that what you are thinking of?

It is certainly true that it is the ELF sections that we actually care about, and those correspond better to the mappings. This is partly because DWARF carves the world into sections, and we are trying to peer into various sections to get information about other sections.

[workingjubilee]

For note, backtrace-rs currently assumes (possibly incorrectly) that once we have obtained the right actual virtual address for the base of a LibrarySegment, that we can assume that p_memsz is an accurate report of the total size in memory... i.e. that every byte from start to finish is something that could matter to us (I won't say "readable" because we use more discerning logic than just that), and that any bytes beyond that are something else. Depending on how p_memsz is computed that may be... varying degrees of incorrect.

[workingjubilee]

Also, apologies if I seem increasingly puzzled about things that seem "obvious". Adopting maintainership of backtrace-rs has mostly been an exercise in unlearning assumptions about what is true. That is sometimes including whether the code is correct.

[waddlesplash]

So the exact location of something gets a bit arbitrary. Is that what you are thinking of?

I haven't dug into backtrace-rs in detail, just enough to see that it's reading the map information from /proc. So if the ELF information is sufficient, why does it do that? Or did nobody bother to implement using p_memsz instead of reading the map size? (Or is reading the map size just for safety for the rare cases where memsz is wrong/inaccurate somehow?)

[workingjubilee]

I haven't dug into backtrace-rs in detail, just enough to see that it's reading the map information from /proc. So if the ELF information is sufficient, why does it do that? Or did nobody bother to implement using p_memsz instead of reading the map size? (Or is reading the map size just for safety for the rare cases where memsz is wrong/inaccurate somehow?)

Using p_memsz was I believe the original solution.

The use of those MapsEntry structures was primarily introduced to handle the ld.so executable trick and any other ways of launching a process that could make using the name of the process to identify what object in memory holds its program... confused at best. In general the map information is slightly more precise about a number of things, since it reports actual memory ranges, so it wound up preferred. I don't believe it's required, however. Even on Linux, we should be able to obtain a trace without access to the procfs. Perhaps I should construct a test for that...