[SR-9347] XCTest and dispatch compiled with wrong linker settings [was: SwiftPM libBlocksRuntime.so.0 missing dependency]

[swift-ci]

Previous ID	SR-9347
Radar	rdar://problem/52194061
Original Reporter	djones6 (JIRA User)
Type	Bug

Environment

Ubuntu 16.04 (docker: ubuntu:16.04)

Additional Detail from JIRA

Votes	1
Component/s	libdispatch
Labels	Bug
Assignee	None
Priority	Medium

md5: c366bc20e06db396929e3f33a653ef84

Issue Description:

SwiftPM in swift development snapshots of 2018-07-21-a and later fails on a clean Ubuntu installation with:

/swift-DEVELOPMENT-SNAPSHOT-2018-07-21-a-ubuntu16.04/usr/bin/swift-package: error while loading shared libraries: libBlocksRuntime.so.0: cannot open shared object file: No such file or directory

It seems that installing libblocksruntime-dev resolves this, but this is not a dependency that was required for any previous versions of Swift.

It looks like this additional dependency was introduced in this PR: swiftlang/swift-package-manager#1631

Snapshots around 2018-10-19-a and later do seem to include /usr/lib/swift/linux/libBlocksRuntime.so - possibly as a result of #362 - but there is no libBlocksRuntime.so.0 symlink to this. Creating a symlink from libBlocksRuntime.so -> libBlocksRuntime.so.0 seems to make things work, but I'll freely admit that I don't really know what I'm doing.

[swift-ci]

Comment by Ian Partridge (JIRA)

@compnerd can you take a look?

[weissi]

so @kevints and I just had a look at what's going on here. Turns out the build system that builds SwiftPM is very wrong and the build systems that build XCTest & dispatch aren't quite right either (but do better). Let's have a quick looks at why this happens: In total, I can see the following binaries trying to link libBlocksRuntime:

Group 1, getting it wrong: using the wrong soname (libBlocksRuntime.so.0 instead of libBlocksRuntime.so):

• swift-build

• swift-run

• swift-test

Running readelf -d on them:

 0x0000000000000001 (NEEDED)             Shared library: [libBlocksRuntime.so.0]

the soname is wrong, should be libBlocksRuntime.so but is libBlocksRuntime.so.0 but the RUNPATH at least contains the needed entries $ORIGIN).

The effect of this is that it'll only ever "work" if you have the distribution's libBlocksRuntime installed and it'll be picked up. I put "work" in quotes because it should really not pick the distribution's libBlocksRuntime up.

Group 2, getting the soname right but the RUNPATH wrong:

• libXCTest.so

• libdispatch.so

• libswiftDispatch.so

• plutil

• swift-package
  and sadly all Swift software built by the toolchain 🙁

running {{readelf -d }} on them:

 0x0000000000000001 (NEEDED)             Shared library: [libBlocksRuntime.so]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux:/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/libdispatch-linux-x86_64/src]

We're missing $ORIGIN in the RUNPATH

The effect of this is that it'll work correctly if you do not have the distro's libBlocksRuntime installed but if you do, the distro's will be picked up preferably which is bad (but the binary will usually work because the two libraries appear to be mostly ABI compatible)

Group 3:

• libFoundation.so

gets both the soname as well as the RUNPATH right:

 0x0000000000000001 (NEEDED)             Shared library: [libBlocksRuntime.so]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/swift-linux-x86_64/lib/swift/linux:/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-16_04/build/buildbot_linux/libdispatch-linux-x86_64/src:$ORIGIN]

I say "right", this should definitely not contain the buildnode's paths but it's probably not a disaster.

[weissi]

@swift-ci create

[weissi]

@kevints is this a fair summary? You know more than I do about this stuff, so please fill in everything that's missing.

[weissi]

pinging @aciidb0mb3r and @neonichu as the SwiftPMs from the toolchain won't even run in the situation that's preferable to us which is that there's no libBlocksRuntime.so* in /usr/lib.

[aciidgh]

It sounds like we just need to configure the rpaths?

[weissi]

@aciidb0mb3r yes, configuring the rpaths and making sure the right soname is picked sounds like it will work.

[aciidgh]

Ok, let me build on linux and check what's going on.

[weissi]

awesome, thanks @aciidb0mb3r

[aciidgh]

SwiftPM's rpaths are correct and the stage1 SwiftPM has the correct linkage where as stage2 SwiftPM is linking both for some reason.

[kevints]

@aciidb0mb3r for the ones needing libBlocksRuntime.so.0 (this is the SONAME of the library provided by the libblocksruntime0 Ubuntu package) the issue is actually linker search path. -L${DISPATCH_BUILD_DIR}/lib needs to be in front of -L/usr/lib/x86_64-linux-gnu. With CMake we should probably just export the BlocksRuntime target from the dispatch build tree and import it into dependee projects. For other build systems passing the full path to libBlocksRuntime.so rather than -lBlocksRuntime should work.

[aciidgh]

I found the SwiftPM issue. The problem is that SwiftPM passes linker paths using -Xlinker prefix in stage2. This somehow decreases the search path priority and the library from system is picked instead. I need to think about how to fix this. Maybe we should be passing linker flags as-is.

[aciidgh]

Talked to @belkadan. He says -Xlinker args are always passed at the end so we end up finding the library in the system instead of at our custom location. We need to drop the -Xlinker prefix for -L flags.

[spevans]

If chrpath is installed on the build hosts it can be used to alter the RUNPATH of the binaries which could be used after installing the files, but before running the swift-integration-tests. I think those tests are passing in some cases because of the absolute paths to the build directories so these could be removed and then the tests fixed up - we shouldn't be shipping binaries or libraries with those paths in them anyway.

[aciidgh]

Fixed SwiftPM here: swiftlang/swift-package-manager#1895