`extension` method with same name as underlying method is not called from inside extension

[lolgab]

Compiler version

Scala 3.3.1-RC1-bin-20230327-a569057-NIGHTLY

Minimized code

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def length: Int = 10
    def printLength(): Unit = {
      println(s"length = $length")
    }

@main
def main =
  val headers = foo.create(Array("foo", "bar"))
  println(s"length = ${headers.length}")
  headers.printLength()

Output

length = 10
length = 2

Expectation

The extension method length should be called both from outside of the extension and inside.
If it is not possible to call it from inside the extension itself, then it should fail to compile instead
of silently having the underlying type behavior.

[som-snytt]

That's a good one. The doc is clear about "implicitly assuming the common extended value ss as receiver." In this scope, if I write headers.length, it means the underlying type. But if I write length, I clearly intended length(headers).

So that language would have to change if this is expected to print different values:

    def printLength(x: Foo): Unit =
      println(s"length = $length")
      println(s"length = ${x.length}")

The language would be "implicitly assume the first argument list for dispatch to other collective extension methods."

Also this is the answer to the puzzler question, Is collective extension semantically the same as separate extensions?

  extension (headers: Foo)
    def length: Int = 10
  extension (headers: Foo)
    def printLength(x: Foo): Unit =
      println(s"length = ${length(headers)}")
      println(s"length = ${x.length}")

Edit: I clarified my comment to say, "That can't be correct behavior, but I can see why it makes weird sense."

In the first snippet, if we are selecting from Foo, then both lines should behave the same. So maybe the behavior should be just that it will call the method in scope "keeping the first arg", which is the intuition.

In the second snippet, as a workaround of sorts, I can put the method in a separate extension element and it is clear how I must invoke it (as an ordinary method). Moreover, I cannot invoke it as an ordinary method from the same collective extension.

[lolgab]

I'm sorry @som-snytt, but I couldn't understand your answer.
Do you mean that the current one is expected behavior?

[som-snytt]

I edited my comment. I don't know what the right answer is, obviously.

[odersky]

 println(s"length = $length")

gets expanded to

println(s"length = ${headers.length}")

But inside object Foo we know that headers is an Array[String]. So its length method takes priority over the extension method .length. That's why you get 2.

So it's a combination of

• expansion of extension methods (which works as expected)
• the fact that members take priority over extension methods
• the fact that opaque types reveal members in some scope and hide them in another

It makes for a nice puzzler but is definitively correct.

[som-snytt]

The dotty equivalent of "feature or bug" is "I can see why it makes weird sense."

I just reopened a Scala 2 ticket from two years ago today, which I'd closed as, "Working as unexpected." I had commented, "A lint would be nice," which applies here. Even if I understand the mechanism, since it invites error, it deserves a lint from the compiler, which is an implicit admission that it is setting us up to fail.

This must be a theme on some season of Westworld, or the "Kobayashi Maru" from Star Trek. It's understood that Kirk beat that test using Scala 2 macros.

Someone was recently using x.asInstanceOf without an arg, which is a Scala 2 lint, so that I was reminded of my hope that linting lands during a spree.

[odersky]

One could maybe lint when defining extension methods that have the same name and arguments as member methods. But the example here shows why this would be tricky. For instance, the following program looks very much like the one that is given, but is actually quite reasonable:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def length: Int = headers.length
    ...

We have an opaque type and publish one of its members as an extension method. The only difference of this program and the bad one is that now length's RHS is correct. But how would a linter detect this without proving general program equivalence?

[som-snytt]

The tricky usage is only the "implicit receiver", length to mean length(headers) but the explicit call is not allowed (for a reason I don't know yet; usually, an extension is "just an ordinary method you can call in the ordinary way"). That is, lint the usage not the definition. That lint would say, "Did you mean to put length in a separate extension and use length(headers)?"

[lolgab]

If it's not possible to make the calls to length inside the extension to call the locally defined length method instead of the the one in the Array type, then I think that calling extension methods that have the same name and parameters as member methods should be forbidden to be called from inside the extension itself.

Given this code:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def length: Int = 10
    def printLength(): Unit = {
      println(s"length = $length")
    }

I would be happy to see a compilation failure like:

[error] extension method length shadows the method length in Array[String]
[error] its usage is forbidden inside its defining extension

Knowing that, I would edit my code to be something like this:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    private def _length: Int = 10
    def length: Int = _length
    def printLength(): Unit = {
      println(s"length = $_length")
    }

@main
def main =
  val headers = foo.create(Array("foo", "bar"))
  println(s"length = ${headers.length}")
  headers.printLength()

Which prints:

length = 10
length = 10

[odersky]

I would be happy to see a compilation failure like:

[error] extension method length shadows the method length in Array[String]
[error] its usage is forbidden inside its defining extension

That would also eliminate the good case I showed. The problem with lint warnings is they look always reasonable and helpful until you discover another related case where they are clearly out of place. And one false negative weighs heavier than 10 correct warnings. So one has to be super careful, and in my experience generally when we write linters we are not careful enough.

[lolgab]

I'm not saying to forbid the ability to define extension methods with the same name as the underlying class methods, that would make it impossible to create an opaque type based on an Array that exposes a method like def length: Int.
What I'm proposing is to forbid calling that method form inside the extension itself, since, even if is "correct" by some definition, is a dangerous foot gun (I created a bug thanks to this, and it took me a while to figure out where the bug came from).

I think it's the not end of the world writing this:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    private def _length: Int = headers.length
    def length: Int = _length
    def otherMethod = _length + 1

instead of this:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def length: Int = headers.length
    def otherMethod = length + 1

Moreover,

For instance, the following program looks very much like the one that is given, but is actually quite reasonable:

object foo:
 opaque type Foo = Array[String]
 def create(array: Array[String]): Foo = array
 extension (headers: Foo)
   def length: Int = headers.length
   ...

I think that this code is reasonable because the definition of length does what is written on the right side of the equal.
But my programming experience tells me that if I change the code after the equal then the result should change.

Imagine this code:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def doubleLength: Int = length * 2

I try to compile it and I get:

[error] ./a.scala:5:29
[error] Not found: length
[error]     def doubleLength: Int = length * 2
[error]                             ^^^^^^

Then I say, "of course, I didn't implement length!" and I do:

object foo:
  opaque type Foo = Array[String]
  def create(array: Array[String]): Foo = array
  extension (headers: Foo)
    def doubleLength: Int = length * 2
    def length: Int = // my custom length algorithm

my code now compiles but my method length is not called.

I have another, even more dangerous example.
Imagine I want to wrap a class from an external library using opaque types.
I do:

object foo:
  opaque type Foo = ExternalLibClass
  def create(underlying: ExternalLibClass): Foo = underlying
  extension (foo: Foo)
    def bar: Int = // my custom code
    def otherMethod: Unit = // does stuff with `bar`

and it compiles and works fine for years.
Then a new minor version of the library gets published which, among other changes, adds a new method to ExternalLibClass. The method is def bar: Int.
By updating a minor version in an external library (which adds a method I wasn't calling at all) I created a bug in my program.