[Modules] Disappointing O(n^2) scaling of compile times with modules and optimizations

[davidstone]

Given the following set of translation units

export module a;

#define REPEAT8(x) \
	(x), \
	(x), \
	(x), \
	(x), \
	(x), \
	(x), \
	(x), \
	(x)


export unsigned a() {
	unsigned x = 0;
	REPEAT8(REPEAT8(REPEAT8(REPEAT8(REPEAT8(REPEAT8(++x))))));
	return x;
}

export module b;

import a;

export unsigned b() {
	return a();
}

export module c;

import b;

unsigned c() {
	return b();
}

Compiling with optimizations turned on (say, -O3) takes a few seconds for a, which is reasonable (a is designed to be expensive to compile and optimize in this example). But then it takes a few seconds for b, and a few seconds for c. My expectation was that the compiler would compile and optimize the function a once, when compiling module a, and then compiling the importers of a would be fast. The net effect effect of this problem is that after converting a program to use modules (and not using any module partitions), the last few modules are each taking about 30 minutes to compile, as each high-level module is optimizing almost my entire program serially.

This is effectively quadratic scaling of compile times -- a does all the work of a, b does all the work of a + b, c does all the work of a + b + c, etc.

I understand how we got to this point -- we want the generation of the .pcm file to happen as quickly as possible, because then any downstream consumer that doesn't care about optimizations can get access to that data quickly (just precompiling b took about 300 milliseconds on my machine, and precompiling c took about 15 milliseconds). However, this seems to sell out the common case of compiling your program.

[llvmbot]

@llvm/issue-subscribers-clang-modules

[ChuanqiXu9]

I tell the reason in #61015. And you can look at https://clang.llvm.org/docs/StandardCPlusPlusModules.html#how-modules-speed-up-compilation if you are interested.

[ChuanqiXu9]

Sent https://reviews.llvm.org/D144844

[davidstone]

I don't think this is the solution I'm looking for. I expect that turning off all inlining and IPO of functions from other modules would reduce my performance too much. I just want clang to be smarter about how it does the work.

The solution I'm imagining is that when optimizations are turned on, we would optimize each module and emit an optimized pcm. Then consumers of those modules wouldn't need to run all of the optimization steps on those functions again. In my example code, a is optimized down to the equivalent of return 262144;. If module b could start with that information, then it would compile and optimize quickly, and we would do the optimization work once.

I would expect a little bit of "additional" work because the importing modules would still want to decide whether it was profitable to inline functions from other modules, and if so, run further optimization passes on the inlined function, but I would expect this to reduce my total time still.

[ChuanqiXu9]

I got your point. It would be best to have an optimized pcm. But given the current status of modules, we may wait much longer for that. So D144844 is not the solution and I won't close this one if that is accepted. This should be the long-existing issue.

[davidstone]

As a workaround, is there a way to say "Just emit a function call and don't even try looking inside"? I've tried [[gnu::noinline]] and that didn't seem to do anything. I have several modules that are just composing the rest of the my program, and these modules take several minutes each even though they are ~40 lines of code and aren't instantiating any templates. It would be nice if I could annotate the functions they call in the interim.