[mlir][gpu] Add the OffloadEmbeddingAttr offloading translation attr

[fabianmcg]

This patch adds the offloading translation attribute. This attribute uses LLVM
offloading infrastructure to embed GPU binaries in the IR. At the program start,
the LLVM offloading mechanism registers kernels and variables with the runtime
library: CUDA RT, HIP RT, or LibOMPTarget.

The offloading mechanism relies on the runtime library to dispatch the correct
kernel based on the registered symbols.

This patch is 3/4 on introducing the OffloadEmbeddingAttr GPU translation
attribute.

Note: Ignore the base commits; those are being reviewed in PRs #78057, #78098,
and #78073.

[jsjodin]

It seems like this is doing the same kind of work that the OffloadInfoManager is doing. Would it be possible to refactor that to not have to add this class?

[fabianmcg]

Currently, the OffloadInfoManager creates the entries and adds them to the omp_offloading_entries section. However, the OffloadInfoManager performs no explicit construction of the entry array needed by the binary descriptor. It's the linker's job to implicitly create the array using all the entries in the section.

The problem with this approach is that LLJIT doesn't handle the implicit creation of the array very well. To overcome this limitation of LLJIT, the attribute constructs the entry array explicitly.

In summary, this class can be removed up to an extent, but then JIT compilation is impossible, and a real linker is needed to obtain the final executable.

[jsjodin]

It sounds like Clang isn't able to be used with the LLJIT in that case, or if it does, then there is already a solution in Clang. I think making this work both for Clang and MLIR would be useful. If there is already a solution in Clang then it should be migrated to the OpenMPIRBuilder.

[fabianmcg]

The real problem is the lack of comprehensive support of linker sections in LLJIT, so I wouldn't say clang, or the clang-linker-wrapper are at fault. The easiest solution that I found was complying with LLJIT.
I think @jhuber6 was looking into changing the registration mechanism of LibOMPTarget binaries, so maybe we can found a solution that works for all.

[jhuber6]

I don't have the full view of what LLJIT does here, but the use-case in clang is that we need each TU to be able to emit values that need to be registered by the runtime. There are a few alternate solutions to this, but having the linker handle it is the best overall. The rework I was talking about was to simply change the offloading entry struct so it's more generic.

How does LLJIT work exactly? If you put globals into a section they will generally appear in order, so if you had a pointer to the first and last globals in that section you could just traverse it once it's gone through the backend. This is somewhat similar to the COFF linker handling which just gives an object at the beginning and end of the others in that section.

[fabianmcg]

There are a few alternate solutions to this, but having the linker handle it is the best overall.

I agree, I think the best solution would be to make LLJIT work.

The rework I was talking about was to simply change the offloading entry struct so it's more generic.

I see.

How does LLJIT work exactly?

Honestly, I'm not 100% sure, I only know that the same IR would work if linked with a regular linker and fail with LLJIT.
I asked around on LLJIT discord a couple months ago why it was not picking out the symbols and they didn't give an answer.

I'll inquire further with them and comeback with a more definitive answer.