[mlir][linalg] `Linalg::PackOp` `Linalg::UnPackOp` to support memref input

[ita9naiwa]

Currently, Linalg::PackOp and Linalg::UnPackOp are restricted to tensor inputs.
But most general linalg ops support memref inputs.

Aim is to extend Pack/Unpack ops to accept memrefs.

PR(s):

• [mlir][linalg] Add Linalg::generateScalarImplementation #128816
• [MLIR][Linalg] pack, unpack to take memref inputs #129036

[llvmbot]

@llvm/issue-subscribers-mlir

Author: Hyunsung Lee (ita9naiwa)

Currently, Linalg::PackOp and Linalg::UnPackOp are restricted to tensor inputs. But most general linalg ops support memref inputs.

Aim is to extend Pack/Unpack ops to accept memrefs.

Related issue: #128816

[llvmbot]

@llvm/issue-subscribers-mlir-linalg

Author: Hyunsung Lee (ita9naiwa)

Currently, Linalg::PackOp and Linalg::UnPackOp are restricted to tensor inputs. But most general linalg ops support memref inputs.

Aim is to extend Pack/Unpack ops to accept memrefs.

Related issue: #128816

[MaheshRavishankar]

I think it is worth extending it to support memref operands.

[rengolin]

A key point to move these ops to linalg was to be able to support memrefs, so I strongly support this issue.

I had a brief look at it and the biggest problem I see is the presumed value semantics that came from tensor. The original code didn't have that issue and I believe IREE's extension doesn't either, so I know it's doable, but there has been a lot of changes to the tensor side that will need to be propagated to memrefs.

[banach-space]

@ita9naiwa , could you keep an up-to-date list of PRs for this in the summary? I'm under the impression that this might take multiple PRs?

@MaheshRavishankar and @rengolin , if you don't mind, could we use this "issue" for the higher-level/design-level parts of the discussion (if any)? And for co-ordination - @rengolin mentioned that he's also looking at packing/unpacking MemRef.

Thank you 🙏🏻

[banach-space]

Mindful of this comment:

Because we developed it in IREE, and the author help upstream the code.

it looks like this is an upstreaming effort? That's totally fine, but should we make sure that the original authors are called out in the summary via GitHub's co-author magic?

I couldn't find any official LLVM policy for this, so I started this thread:

• https://discourse.llvm.org/t/code-review-process-when-upstreaming-patches

Thanks!

[hanhanW]

IMO, we can use co-author mechanism for this case. IIRC, they were mainly developed by @chelini and me.

The code was developed a few years ago, and we can revisit some implementation during the upstream. I'll also help the review, and let's make sure that we get an approval other than me.

For this work, we mainly need to support memrefs and upstream the generateScalarImplementation. I'm not sure if it works directly or not, but let's at least add a bufferization test in a PR: https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/bufferize.mlir

[banach-space]

Mostly as an FYI, I've just landed this:

• [llvm][docs] Add notes on upstreaming code from downstream projects #129743