build: perform parallel builds of BlocksRuntime

[compnerd]

Build both the static and shared variants of the BlocksRuntime. This is needed
for proper shared linkage to the BlocksRuntime (e.g. when building SourceKit).
Build both variants and install to the appropriate location.

[compnerd]

CC: @gottesmm @kevints

[kevints]

What does this do on platforms that don't use import libraries?

[compnerd]

It does nothing meaningful on those platforms. The install rules are what matter, which need to be guarded.

[kevints]

My experience using ExternalProject hasn't been great compared to creating library targets that are directly addressable in the build system and I strongly suggest we don't use it in dispatch.

1. We need to propagate all possible interesting CMake variables from the outer build system to the inner ExternalProject build system via CMAKE_ARGS. For example here you don't propagate CMAKE_C_FLAGS, CMAKE_AR or CMAKE_LIBRARY_PATH which could cause the downstream libraries to be built differently. There are too many influential variables to enumerate here.
2. ExternalProjects don't get good dependency tracking compared to targets that cmake understands natively. You have to declare BUILD_BYPRODUCTS manually and it's easy to under or over-depend. ExternalProjects also don't have great IDE integration as they appear opaque.
3. Setting up a library target as IMPORTED prevents dependency tracking and prevents it being exported from the build system (via install(EXPORT) for example).
4. Ninja is not designed to be invoked recursively - you can get into a situation where the downstream ninja is performing tasks equal to the number of CPUs, so you can have (N^2-1) tasks running in parallel (where N is the number of cores your build machine has). Console output is also broken.

Having said all that, have you considered making BlocksRuntime and BlocksRuntime_static targets using add_library instead?

[compnerd]

Well, we already are using ExternalProject for libdispatch in swift, and it has been working quite well :-). That said, there is interest in moving to the LLVM wrapper. Doing so will automatically propagate those other variables implicitly.

The dependency tracking issue that you portray is not entirely correct. There is tracking of the dependencies in the sense that the external project's byproducts are linked into the imported library target. So if the sub-library changes, CMake/ninja realize that and rebuild the dependent targets.

Yes, I did consider that. I suppose that if we want to just duplicate the rules, thats certainly an option. I really am strongly opposed to creating the custom wrappers for both variants. I suppose that we could always extract the BlocksRuntime rules into a separate CMakeList and include it in this one to make this easier to work with.

[kevints]

By placing this inside an ExternalProject there's no way to trigger this option anymore.

[compnerd]

Oh, but there is! There is the still the option in libdispatch, which just passes it through :-) However, note that the installation for the swift side is done in the libdispatch side, so that we can install both variants into the install'ed image for libdispatch.

[compnerd]

Discussed with @kevints offline. We have a better approach that we are going to be taking and this change will be revised with a new version.