[mlir][linalg] Simplify vectorization test output using `-canonicalize -cse`

[banach-space]

The Linalg vectorization tests are currently quite complex and hard to navigate (see full list with links below). One area I’d like to improve is simplifying the expected test output by updating the mlir-opt invocation to include:

• -canonicalize -cse.

Why add -cse?

CSE alone is a huge win. It eliminates redundant constants like:

%c0 = arith.constant  0 : index
%c0_1 = arith.constant  0 : index
%c0_2 = arith.constant  0 : index

Without CSE, test updates often involve unnecessarily matching different SSA values representing the same constant, which adds noise and overhead.

Why add -canonicalize?

Adding -canonicalize helps simplify tensor.dim, affine.apply, and other commonly duplicated constructs.

Current output from the vectorizer

  func.func @test_masked_vectorize_dynamic_pad(%arg0: tensor<?x?xf32>, %arg1: index, %arg2: index) -> tensor<?x?xf32> {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %c0_0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0_0 : tensor<?x?xf32>
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_1 = tensor.dim %arg0, %c1 : tensor<?x?xf32>
    %1 = affine.apply #map()[%arg2, %dim_1]
    %c0_2 = arith.constant 0 : index
    %c0_3 = arith.constant 0 : index
    %dim_4 = tensor.dim %arg0, %c0_3 : tensor<?x?xf32>
    %c1_5 = arith.constant 1 : index
    %dim_6 = tensor.dim %arg0, %c1_5 : tensor<?x?xf32>
    %2 = vector.create_mask %dim_4, %dim_6 : vector<2x4xi1>
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0_2, %c0_2], %cst {in_bounds = [true, true]} : tensor<?x?xf32>, vector<2x4xf32> } : vector<2x4xi1> -> vector<2x4xf32>
    %4 = tensor.empty(%0, %1) : tensor<?x?xf32>
    %c0_7 = arith.constant 0 : index
    %c0_8 = arith.constant 0 : index
    %dim_9 = tensor.dim %4, %c0_8 : tensor<?x?xf32>
    %c1_10 = arith.constant 1 : index
    %dim_11 = tensor.dim %4, %c1_10 : tensor<?x?xf32>
    %5 = vector.create_mask %dim_9, %dim_11 : vector<2x4xi1>
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0_7, %c0_7] {in_bounds = [true, true]} : vector<2x4xf32>, tensor<?x?xf32> } : vector<2x4xi1> -> tensor<?x?xf32>
    return %6 : tensor<?x?xf32>
  }

There is a lot of duplication of arith.constant and tensor.dim.

Output from the vectorizer after adding -cse:

  func.func @test_masked_vectorize_dynamic_pad(%arg0: tensor<?x?xf32>, %arg1: index, %arg2: index) -> tensor<?x?xf32> {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor<?x?xf32>
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_0 = tensor.dim %arg0, %c1 : tensor<?x?xf32>
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector<2x4xi1>
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor<?x?xf32>, vector<2x4xf32> } : vector<2x4xi1> -> vector<2x4xf32>
    %4 = tensor.empty(%0, %1) : tensor<?x?xf32>
    %dim_1 = tensor.dim %4, %c0 : tensor<?x?xf32>
    %dim_2 = tensor.dim %4, %c1 : tensor<?x?xf32>
    %5 = vector.create_mask %dim_1, %dim_2 : vector<2x4xi1>
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector<2x4xf32>, tensor<?x?xf32> } : vector<2x4xi1> -> tensor<?x?xf32>
    return %6 : tensor<?x?xf32>
  }

No duplication of arith.constant, but tensor.dim is still unnecessarily duplicated.

Output from the vectorizer after adding -canonicalize -cse:

  func.func @test_masked_vectorize_dynamic_pad(%arg0: tensor<?x?xf32>, %arg1: index, %arg2: index) -> tensor<?x?xf32> {
    %c1 = arith.constant 1 : index
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor<?x?xf32>
    %0 = affine.apply #map()[%arg1, %dim]
    %dim_0 = tensor.dim %arg0, %c1 : tensor<?x?xf32>
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector<2x4xi1>
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor<?x?xf32>, vector<2x4xf32> } : vector<2x4xi1> -> vector<2x4xf32>
    %4 = tensor.empty(%0, %1) : tensor<?x?xf32>
    %5 = vector.create_mask %0, %1 : vector<2x4xi1>
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector<2x4xf32>, tensor<?x?xf32> } : vector<2x4xi1> -> tensor<?x?xf32>
    return %6 : tensor<?x?xf32>
  }

No duplication :)

Pros vs Cons

Pros:

• Easier to focus on the semantic intent of vectorization output.
• Reduces test maintenance (less duplication, fewer fragile SSA names).
• Aligns with FileCheck best practices: test the minimal necessary.

Cons:

• Tests will now depend on CSE and canonicalization, making them indirectly sensitive to unrelated changes.
• Tests will no longer isolate vectorization alone - they will validate a pipeline of transformations.

Next steps

While there are trade-offs, I believe this change will be beneficial overall.

My first patch is here:

• [mlir][linalg] Update vectorization tests for tensor.pad and tensor.insert_slice #138267

Assuming there are no strong objections, I’d like to use this issue for discussion and long-term context.
CC @dcaballe @hanhanW - you've reviewed most of my patches in this area. Anyone else I should include?

Thanks!

List of test files

• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-pad-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-unsupported.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-with-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-conv-masked-and-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution-flatten.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract-masked.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract.mlir

[llvmbot]

@llvm/issue-subscribers-mlir

Author: Andrzej Warzyński (banach-space)

The Linalg vectorization tests are currently quite complex and hard to navigate (see full list with links below). One area I’d like to improve is simplifying the expected test output by updating the `mlir-opt` invocation to include: * `-canonicalize -cse`.

Why add -cse?

CSE alone is a huge win. It eliminates redundant constants like:

%c0 = arith.constant  0 : index
%c0_1 = arith.constant  0 : index
%c0_2 = arith.constant  0 : index

Without CSE, test updates often involve unnecessarily matching different SSA values representing the same constant, which adds noise and overhead.

Why add -canonicalize?

Adding -canonicalize helps simplify tensor.dim, affine.apply, and other commonly duplicated constructs.

Current output from the vectorizer

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %c0_0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0_0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_1 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_1]
    %c0_2 = arith.constant 0 : index
    %c0_3 = arith.constant 0 : index
    %dim_4 = tensor.dim %arg0, %c0_3 : tensor&lt;?x?xf32&gt;
    %c1_5 = arith.constant 1 : index
    %dim_6 = tensor.dim %arg0, %c1_5 : tensor&lt;?x?xf32&gt;
    %2 = vector.create_mask %dim_4, %dim_6 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0_2, %c0_2], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %c0_7 = arith.constant 0 : index
    %c0_8 = arith.constant 0 : index
    %dim_9 = tensor.dim %4, %c0_8 : tensor&lt;?x?xf32&gt;
    %c1_10 = arith.constant 1 : index
    %dim_11 = tensor.dim %4, %c1_10 : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %dim_9, %dim_11 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0_7, %c0_7] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

There is a lot of duplication of arith.constant and tensor.dim.

Output from the vectorizer after adding -cse:

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_0 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %dim_1 = tensor.dim %4, %c0 : tensor&lt;?x?xf32&gt;
    %dim_2 = tensor.dim %4, %c1 : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %dim_1, %dim_2 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

No duplication of arith.constant, but tensor.dim is still unnecessarily duplicated.

Output from the vectorizer after adding -canonicalize -cse:

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %c1 = arith.constant 1 : index
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %dim_0 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %0, %1 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

No duplication :)

Pros vs Cons

Pros:

• Easier to focus on the semantic intent of vectorization output.
• Reduces test maintenance (less duplication, fewer fragile SSA names).
• Aligns with FileCheck best practices: test the minimal necessary.

Cons:

• Tests will now depend on CSE and canonicalization, making them indirectly sensitive to unrelated changes.
• Tests will no longer isolate vectorization alone - they will validate a pipeline of transformations.

Next steps

While there are trade-offs, I believe this change will be beneficial overall.

My first patch is here:

• TODO

Assuming there are no strong objections, I’d like to use this issue for discussion and long-term context.
CC @dcaballe @hanhanW - you've reviewed most of my patches in this area. Anyone else I should include?

Thanks!

List of test files

• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-pad-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-unsupported.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-with-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-conv-masked-and-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution-flatten.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract-masked.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract.mlir

[llvmbot]

@llvm/issue-subscribers-mlir-linalg

Author: Andrzej Warzyński (banach-space)

The Linalg vectorization tests are currently quite complex and hard to navigate (see full list with links below). One area I’d like to improve is simplifying the expected test output by updating the `mlir-opt` invocation to include: * `-canonicalize -cse`.

Why add -cse?

CSE alone is a huge win. It eliminates redundant constants like:

%c0 = arith.constant  0 : index
%c0_1 = arith.constant  0 : index
%c0_2 = arith.constant  0 : index

Without CSE, test updates often involve unnecessarily matching different SSA values representing the same constant, which adds noise and overhead.

Why add -canonicalize?

Adding -canonicalize helps simplify tensor.dim, affine.apply, and other commonly duplicated constructs.

Current output from the vectorizer

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %c0_0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0_0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_1 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_1]
    %c0_2 = arith.constant 0 : index
    %c0_3 = arith.constant 0 : index
    %dim_4 = tensor.dim %arg0, %c0_3 : tensor&lt;?x?xf32&gt;
    %c1_5 = arith.constant 1 : index
    %dim_6 = tensor.dim %arg0, %c1_5 : tensor&lt;?x?xf32&gt;
    %2 = vector.create_mask %dim_4, %dim_6 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0_2, %c0_2], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %c0_7 = arith.constant 0 : index
    %c0_8 = arith.constant 0 : index
    %dim_9 = tensor.dim %4, %c0_8 : tensor&lt;?x?xf32&gt;
    %c1_10 = arith.constant 1 : index
    %dim_11 = tensor.dim %4, %c1_10 : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %dim_9, %dim_11 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0_7, %c0_7] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

There is a lot of duplication of arith.constant and tensor.dim.

Output from the vectorizer after adding -cse:

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %c1 = arith.constant 1 : index
    %dim_0 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %dim_1 = tensor.dim %4, %c0 : tensor&lt;?x?xf32&gt;
    %dim_2 = tensor.dim %4, %c1 : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %dim_1, %dim_2 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

No duplication of arith.constant, but tensor.dim is still unnecessarily duplicated.

Output from the vectorizer after adding -canonicalize -cse:

  func.func @<!-- -->test_masked_vectorize_dynamic_pad(%arg0: tensor&lt;?x?xf32&gt;, %arg1: index, %arg2: index) -&gt; tensor&lt;?x?xf32&gt; {
    %c1 = arith.constant 1 : index
    %cst = arith.constant 4.243000e+01 : f32
    %c0 = arith.constant 0 : index
    %dim = tensor.dim %arg0, %c0 : tensor&lt;?x?xf32&gt;
    %0 = affine.apply #map()[%arg1, %dim]
    %dim_0 = tensor.dim %arg0, %c1 : tensor&lt;?x?xf32&gt;
    %1 = affine.apply #map()[%arg2, %dim_0]
    %2 = vector.create_mask %dim, %dim_0 : vector&lt;2x4xi1&gt;
    %3 = vector.mask %2 { vector.transfer_read %arg0[%c0, %c0], %cst {in_bounds = [true, true]} : tensor&lt;?x?xf32&gt;, vector&lt;2x4xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; vector&lt;2x4xf32&gt;
    %4 = tensor.empty(%0, %1) : tensor&lt;?x?xf32&gt;
    %5 = vector.create_mask %0, %1 : vector&lt;2x4xi1&gt;
    %6 = vector.mask %5 { vector.transfer_write %3, %4[%c0, %c0] {in_bounds = [true, true]} : vector&lt;2x4xf32&gt;, tensor&lt;?x?xf32&gt; } : vector&lt;2x4xi1&gt; -&gt; tensor&lt;?x?xf32&gt;
    return %6 : tensor&lt;?x?xf32&gt;
  }

No duplication :)

Pros vs Cons

Pros:

• Easier to focus on the semantic intent of vectorization output.
• Reduces test maintenance (less duplication, fewer fragile SSA names).
• Aligns with FileCheck best practices: test the minimal necessary.

Cons:

• Tests will now depend on CSE and canonicalization, making them indirectly sensitive to unrelated changes.
• Tests will no longer isolate vectorization alone - they will validate a pipeline of transformations.

Next steps

While there are trade-offs, I believe this change will be beneficial overall.

My first patch is here:

• TODO

Assuming there are no strong objections, I’d like to use this issue for discussion and long-term context.
CC @dcaballe @hanhanW - you've reviewed most of my patches in this area. Anyone else I should include?

Thanks!

List of test files

• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-pad-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-unsupported.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization-with-patterns.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorization.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-conv-masked-and-scalable.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution-flatten.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-convolution.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract-masked.mlir
• https://github.com/llvm/llvm-project/blob/main/mlir/test/Dialect/Linalg/vectorize-tensor-extract.mlir

[joker-eph]

There is a lot of duplication of arith.constant

Could the vectorizer implement its own constant cache? Avoiding the duplication early would improve compile time.

In general I'm a bit nervous about this kind of things: this is breaking the "unit-test" logic and we actually no longer match the output of the vectorizer. Canonicalize is a pretty involved transformation:

Easier to focus on the semantic intent of vectorization output.

It's not clear to me: on the contrary you end up with a canonicalized IR which doesn't look like the output of the vectorizer, so now you have some sort of proxy to the vectorizer output, and are one step further: shouldn't make it "Easier to focus on the semantics intent".

Reduces test maintenance (less duplication, fewer fragile SSA names).

That's a FileCheck problem to me if you depend on this.

Aligns with FileCheck best practices: test the minimal necessary.

Not clear to me what you mean here? The FileCheck guidelines says you check the minimal amount of properties in the current output.

[banach-space]

Could the vectorizer implement its own constant cache? Avoiding the duplication early would improve compile time.

The duplication comes from numerous helper hooks external to the vectorizer logic, e.g. getMixedSizes that creates tensor.dim Ops:

llvm-project/mlir/lib/Dialect/Tensor/IR/TensorOps.cpp

Lines 60 to 68 in 0009a17

	OpFoldResult tensor::getMixedSize(OpBuilder &builder, Location loc, Value value,
	int64_t dim) {
	auto tensorType = llvm::cast<RankedTensorType>(value.getType());
	SmallVector<OpFoldResult> result;
	if (tensorType.isDynamicDim(dim))
	return builder.createOrFold<tensor::DimOp>(loc, value, dim);
	return builder.getIndexAttr(tensorType.getDimSize(dim));
	}

We'd need some more global cache, perhaps attached to e.g. Op Builder. Is there anything like this that I could use as reference?

[jpienaar]

I have one of these (I need to see where I put it), basically yes it was just builder but specialized for constants (as I was generating a lot of them).

This is interesting, I see both sides here. The extra constants make checking actually difficulty as you have to match on one of n duplicate constants with same value. But also it could be hiding some really terrible behavior that is cleaned up and nobody notices. We don't want to reimplement all the optimizations everywhere though. Is it fair to say that if we had an OpBuilder with constant cache (per scope) and maybe used createOrFold for all of those, it would address the majority of the issue? (At the cost of folks needing to follow example vs general passes post which could potentially not show actual effect of pass but is uniform)

[banach-space]

Thanks for the feedback so far!

I now see why it may not be appropriate to use -cse -canonicalize for all vectorization tests. That said, it might still make sense to apply them to more complex vectorization patterns - particularly those involving:

• tensor.pad + tensor.insert_slice + linalg.pack + linalg.unpack.

These ops have dedicated vectorization hooks, and the resulting IR tends to be quite noisy.

Btw, I will be away for a week. I will return to this once I am back.

[dcaballe]

I agree with Mehdi and Jacques. It's very easy to hide underlying issues behind cse and canonicalization and create a strong dependency with those passes at different levels. Canonicalization is big hammer.

Having a generic cache for constants would be fantastic! I’d be happy to help with this if needed. I implemented one a couple of weeks ago which hoists constants to the closest region isolated from above. My current implementation has some limitations and it’s not well integrated with the existing builders so I’d love to compare notes with Jacques😊.

I think that the constant cache + using createOrFold + revisit tests and CHECK rules to be minimal would be a huge improvement, IMHO.

[banach-space]

Hey, thanks again for the feedback. I went ahead and prototyped a constant cache that's integrated with createOrFold. It's merely a PoC (still need to polish it), but the result is promising:

Output from the vectorizer without caching:

  func.func private @insert_slice_static_sizes(%arg0: tensor<?x3x?x1xi32>) -> tensor<5x3xi32> {
    %c2 = arith.constant 2 : index
    %0 = tensor.empty() : tensor<5x3xi32>
    %extracted_slice = tensor.extract_slice %arg0[0, %c2, 0, 0] [1, 1, 5, 1] [1, 1, 1, 1] : tensor<?x3x?x1xi32> to tensor<5x1xi32>
    %c0_i32 = arith.constant 0 : i32
    %c0 = arith.constant 0 : index
    %c0_0 = arith.constant 0 : index
    %c5 = arith.constant 5 : index
    %c1 = arith.constant 1 : index
    %1 = vector.create_mask %c5, %c1 : vector<8x1xi1>
    %2 = vector.mask %1 { vector.transfer_read %extracted_slice[%c0_0, %c0_0], %c0_i32 {in_bounds = [true, true]} : tensor<5x1xi32>, vector<8x1xi32> } : vector<8x1xi1> -> vector<8x1xi32>
    %c0_1 = arith.constant 0 : index
    %c5_2 = arith.constant 5 : index
    %c3 = arith.constant 3 : index
    %3 = vector.create_mask %c5_2, %c3 : vector<8x1xi1>
    %4 = vector.mask %3 { vector.transfer_write %2, %0[%c0_1, %c2] {in_bounds = [true, true]} : vector<8x1xi32>, tensor<5x3xi32> } : vector<8x1xi1> -> tensor<5x3xi32>
    return %4 : tensor<5x3

Output from the vectorizer with caching:

  func.func private @insert_slice_static_sizes(%arg0: tensor<?x3x?x1xi32>) -> tensor<5x3xi32> {
    %c2 = arith.constant 2 : index
    %0 = tensor.empty() : tensor<5x3xi32>
    %extracted_slice = tensor.extract_slice %arg0[0, %c2, 0, 0] [1, 1, 5, 1] [1, 1, 1, 1] : tensor<?x3x?x1xi32> to tensor<5x1xi32>
    %c0_i32 = arith.constant 0 : i32
    %c0 = arith.constant 0 : index
    %c5 = arith.constant 5 : index
    %c1 = arith.constant 1 : index
    %1 = vector.create_mask %c5, %c1 : vector<8x1xi1>
    %2 = vector.mask %1 { vector.transfer_read %extracted_slice[%c0, %c0], %c0_i32 {in_bounds = [true, true]} : tensor<5x1xi32>, vector<8x1xi32> } : vector<8x1xi1> -> vector<8x1xi32>
    %c3 = arith.constant 3 : index
    %3 = vector.create_mask %c5, %c3 : vector<8x1xi1>
    %4 = vector.mask %3 { vector.transfer_write %2, %0[%c0, %c2] {in_bounds = [true, true]} : vector<8x1xi32>, tensor<5x3xi32> } : vector<8x1xi1> -> tensor<5x3xi32>
    return %4 : tensor<5x3xi32>
  }

For this to work, I had to update various "global" utils used by the vectoriser (create --> createOrFold). These tools are from:

• mlir/lib/Dialect/Arith/Utils/Utils.cpp
• mlir/lib/Dialect/Vector/Utils/VectorUtils.cpp

So, what I have right now will have impact on things beyond the Vectorizer. Once I am happy with my draft, I will share it on GitHub.

Thanks!

EDIT

I just chatted with @dcaballe on Discord - he also has a working prototype, likely more advanced than mine (he had a head start 😅). I’ll wait for Diego to share his implementation first.