[mlir][vector] Constrain Emulated Types in Narrow Type Emulation

[banach-space]

Hi folks,

I wanted to check whether we could constrain the emulated types in the narrow type emulation. I propose that:

• The emulated type should always be a sub-byte type.

Current Situation

Right now, we broadly have two categories of patterns:

1. Patterns that assume the emulated type is a sub-byte type, e.g., i2 is emulated using i8.
2. Patterns with weaker assumptions about the emulated type, allowing, for example, i8 to be emulated using i32.

The first category includes patterns like:

llvm-project/mlir/lib/Dialect/Vector/Transforms/VectorEmulateNarrowType.cpp

Lines 2127 to 2138 in d928a67

	// Patterns for aligned cases. We set higher priority as they are expected to
	// generate better performance for aligned cases.
	// The container type is always i8.
	patterns.add<RewriteAlignedSubByteIntExt<arith::ExtSIOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntExt<arith::SIToFPOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntTrunc>(patterns.getContext(),
	benefit.getBenefit() + 1);
	// The container type is always i8.
	patterns
	.add<RewriteAlignedSubByteIntExt<arith::ExtUIOp, /*isSigned=*/false>,
	RewriteAlignedSubByteIntExt<arith::UIToFPOp, /*isSigned=*/false>>(
	patterns.getContext(), benefit.getBenefit() + 1);

The second category includes patterns such as ConvertVectorLoad

Why This Is a Problem

This mixed level of support creates several challenges:

• Lack of consistency – This isn't well-documented, and the intended design direction is unclear.
• Harder to maintain and extend – Without consistency, creating common code paths is much trickier.
• Unclear guidelines for new patterns – When adding new transformations, how do we decide what level of support to provide and test?

Discussion: Should We Constrain to Sub-Byte Types?

I don’t want to artificially limit narrow-type emulation, so if anyone requires emulating i8 using i32, please let me know.

For the targets I care about, i8 is natively supported, so this wouldn't be an issue for me. However, we need to ensure this approach works for all relevant targets.

That said, please consider the maintenance cost—especially for testing. Supporting all possible scenarios would require covering:

• i2 → i8 vs. i2 → i16
• i8 → i16 vs. i8 → i32
• …and many more combinations.

Even if restricting to sub-byte types is too limiting, we should still aim to define a clear and focused subset of cases to support.

Thanks!

[llvmbot]

@llvm/issue-subscribers-mlir

Author: Andrzej Warzyński (banach-space)

Hi folks,

I wanted to check whether we could constrain the emulated types in the narrow type emulation. I would like to propose that:

• The emulated type is always required to be a sub-byte type.

Right now, broadly speaking, we have two types of patterns:

1. Patterns that assume that the emulated type is indeed a sub-byte type, e.g. i2 is emulated using i8.
2. Patterns that make weaker assumptions about the emulated type (so that e.g. i8 can be emulated using i32 ).

The patterns that fall into the first category are here:

llvm-project/mlir/lib/Dialect/Vector/Transforms/VectorEmulateNarrowType.cpp

Lines 2127 to 2138 in d928a67

	// Patterns for aligned cases. We set higher priority as they are expected to
	// generate better performance for aligned cases.
	// The container type is always i8.
	patterns.add<RewriteAlignedSubByteIntExt<arith::ExtSIOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntExt<arith::SIToFPOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntTrunc>(patterns.getContext(),
	benefit.getBenefit() + 1);
	// The container type is always i8.
	patterns
	.add<RewriteAlignedSubByteIntExt<arith::ExtUIOp, /*isSigned=*/false>,
	RewriteAlignedSubByteIntExt<arith::UIToFPOp, /*isSigned=*/false>>(
	patterns.getContext(), benefit.getBenefit() + 1);

ConvertVectorLoad is one pattern that falls into the second category.

This "dual" state with mixed level of support between patterns is problematic:

• This lack of inconsistency is not really documented and the desired end-state is not clear.
• Without consistency, it's much trickier to create common code paths.
• When new patterns are added, how do we decide what level of support to add and test?

Now, I don't want to artificially constrain "narrow-type-emulation", so please let me know if you do require emulating i8 using e.g. i32. The targets that I care about all support i8 natively, so this wouldn't be a problem for me. However, we need to make sure that this works for everyone. I would ask though that folks take into account the maintenance cost of this.

[llvmbot]

@llvm/issue-subscribers-mlir-vectorops

Author: Andrzej Warzyński (banach-space)

Hi folks,

I wanted to check whether we could constrain the emulated types in the narrow type emulation. I would like to propose that:

• The emulated type is always required to be a sub-byte type.

Right now, broadly speaking, we have two types of patterns:

1. Patterns that assume that the emulated type is indeed a sub-byte type, e.g. i2 is emulated using i8.
2. Patterns that make weaker assumptions about the emulated type (so that e.g. i8 can be emulated using i32 ).

The patterns that fall into the first category are here:

llvm-project/mlir/lib/Dialect/Vector/Transforms/VectorEmulateNarrowType.cpp

Lines 2127 to 2138 in d928a67

	// Patterns for aligned cases. We set higher priority as they are expected to
	// generate better performance for aligned cases.
	// The container type is always i8.
	patterns.add<RewriteAlignedSubByteIntExt<arith::ExtSIOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntExt<arith::SIToFPOp, /*isSigned=*/true>,
	RewriteAlignedSubByteIntTrunc>(patterns.getContext(),
	benefit.getBenefit() + 1);
	// The container type is always i8.
	patterns
	.add<RewriteAlignedSubByteIntExt<arith::ExtUIOp, /*isSigned=*/false>,
	RewriteAlignedSubByteIntExt<arith::UIToFPOp, /*isSigned=*/false>>(
	patterns.getContext(), benefit.getBenefit() + 1);

ConvertVectorLoad is one pattern that falls into the second category.

This "dual" state with mixed level of support between patterns is problematic:

• This lack of inconsistency is not really documented and the desired end-state is not clear.
• Without consistency, it's much trickier to create common code paths.
• When new patterns are added, how do we decide what level of support to add and test?

Now, I don't want to artificially constrain "narrow-type-emulation", so please let me know if you do require emulating i8 using e.g. i32. The targets that I care about all support i8 natively, so this wouldn't be a problem for me. However, we need to make sure that this works for everyone. I would ask though that folks take into account the maintenance cost of this.