Allow to implement abstract inline methods for type classes in quotes

[goshacodes]

Hi! I want to implement abstract inline method in macro and it fails. The only way I can derive this is via quotation (there are some compile time configuration, which properly can be used only in macro), so it would be really great to allow this.

Compiler version

3.3.5 and 3.6.3

Minimized code

trait TokenWriter

trait JsonWriter[A]:
 inline def write(value: A, tokenWriter: TokenWriter): Unit

object JsonWriter:
 inline def derived[A]: JsonWriter[A] = ${ derivedMacro[A] }

def derivedMacro[A](using quotes: scala.quoted.Quotes): scala.quoted.Expr[JsonWriter[A]] =
  import quotes.reflect.*
 '{
     new JsonWriter[A]:
       inline def write(value: A, tokenWriter: TokenWriter): Unit = ()
  }

Output

[error] 308 |          inline def write(value: T, tokenWriter: TokenWriter): Unit =
[error]     |                     ^
[error]     |                     inline def cannot be within quotes

Expectation

Compiles

[prolativ]

Macros have some known limitations originating from their design. Making this work should be treated as a new feature in the language rather than a bug fix, even though I'm not sure if this would be doable at all (or with a justifiable cost).
However, I believe there should be some workaround for your use case, but you would have to describe it with more details.
If all you need is to pass some compile time values to the implementation of a type class instance, make write a non-inline method and inject the config to the implementation like this

import scala.quoted.*

trait TokenWriter

trait JsonWriter[A]:
  def write(value: A, tokenWriter: TokenWriter): Unit

object JsonWriter:
  inline def derived[A]: JsonWriter[A] = ${ derivedMacro[A] }

def derivedMacro[A : Type](using quotes: Quotes): Expr[JsonWriter[A]] =
  val compileTimeConfig: String = "Value from compile time" // fetch compile-time configuration instead
  val configExpr = Expr(compileTimeConfig)
  '{
    new JsonWriter[A]:
      def write(value: A, tokenWriter: TokenWriter): Unit =
        val config: String = ${configExpr}
        println(s"Using config: $config")
        // Serialize A here using config
  }

[goshacodes]

Thank you. My goal is to add an inline to that method since it will really improve runtime performance. So I definitely not want to remove an inline. If my case were simple enough, I would do it without quotes, but I can't.

This is the actual code:
https://github.com/tethys-json/tethys/blob/master/modules/core/src/main/scala-3/tethys/derivation/Derivation.scala#L113-L216

[prolativ]

Some high-level description of what you need to optimize would still be useful, so that I could advise something.
But maybe you don't really need type classes actually at all? Like instead of

summon[JsonWriter[A]].write(...)

you should do something like

writeJson[A](...)

with writeJson being itself an inline method whose implementation recursively computes Exprs corresponding to about-to-be-inlined implementations of JsonWriter[B]#write (given that B is one of the types referenced by A for which the writer's logic has to be derived) and just then at the very end splice your combined implementation inside writeJson macro

[goshacodes]

We thought about solution without type classes, but we want it to be configurable using type classes as it is now

Consider following models:

case class Foo(x: Int, bar: String) derives JsonWriter

case class Baz(y: Boolean, foo: Foo) derives JsonWriter

Now for Baz it generates

new JsonObjectWriter[Baz]:
  def write(baz: Baz, out: TokenWriter): Unit =
    out.writeObjectStart()
    out.writeFieldName("y")
    summon[JsonWriter[Boolean].write(baz.y, out) // which calls out.writeBoolean(baz.y)
    out.writeFieldName("foo")
    summon[JsonWriter[Foo]].write(baz.foo, out) // which calls some methods on out
    out.writeObjectEnd()

I want it to generate:

new JsonObjectWriter[Baz]:
  def write(baz: Baz, out: TokenWriter): Unit =
     out.writeObjectStart()
     out.writeFieldName("y")
     out.writeBoolean(baz.y)
     out.writeFieldName("foo")
     out.writeObjectStart()
     out.writeFieldName("x")
     out.writeNumber(baz.foo.x)
     out.writeFieldName("bar")
     out.writeString(baz.foo.bar)
     out.writeObjectEnd()
     out.writeObjectEnd()

Or even just:

  out.writeObjectStart()
  out.writeFieldName("y")
  out.writeBoolean(baz.y)
  out.writeFieldName("foo")
  out.writeObjectStart()
  out.writeFieldName("x")
  out.writeNumber(baz.foo.x)
  out.writeFieldName("bar")
  out.writeString(baz.foo.bar)
  out.writeObjectEnd()
  out.writeObjectEnd()

Where final inlined method is this one:

extension[A] (a: A)
  def writeJson(out: TokenWriter)(using jsonWriter: JsonWriter[A]): Unit =
    jsonWriter.write(a, tokenWriter)

So expansion should depend on type class instance method implementation

[prolativ]

@goshacodes I played with the problem for a while and here's what I managed to get.
Below there's a quite naïve, but compiling (and runnable) implementation of a JSON writer which inlines all the writeX commands without using any other intermediate methods or objects at runtime, at the same time allowing users to customize the behaviour by defining given instances of a type class.
Here are some things that should be noticed:

• If the default implementation for a case class is desired, one should not try to derive JsonWriter for it - the compiler derives the implementation on flight without an instance of the type class. If you'd rather have JsonWriter as an opt-in feature, you could add require some explicit marker on a class (e.g. an annotation) and then check for its presence in the macro
• writeProxy inline method seems to be needed to by-pass the current limitations forbidding invoking write method from JsonWriter trait directly in the body of a makro
• The given instance of JsonWriter[Qux[A]] should be defined as inline - otherwise a dangling reference to the instance will remain in the bytecode even though the invocation of .write on it will be inlined as expected. For given JsonWriter[Foo] inlined is not required as it takes no parameters so it will be desugared to an object

TokenWriter.scala:

trait TokenWriter:
  def writeFieldName(name: String): Unit
  def writeNumber(i: Int): Unit
  def writeString(s: String): Unit
  def writeBoolean(b: Boolean): Unit
  def writeObjectStart(): Unit
  def writeObjectEnd(): Unit
  def writeSeparator(): Unit

class StdoutTokenWriter extends TokenWriter:
  def writeFieldName(name: String): Unit = print(s"\"$name\":")
  def writeNumber(i: Int): Unit = print(i)
  def writeString(s: String): Unit = print(s"\"$s\"")
  def writeBoolean(b: Boolean): Unit = print(b)
  def writeObjectStart(): Unit = print("{")
  def writeObjectEnd(): Unit = print("}")
  def writeSeparator(): Unit = print(",")

JsonWriter.scala:

import scala.quoted.*
import scala.deriving.*

trait JsonWriter[A]:
  inline def write(value: A, writer: TokenWriter): Unit

object JsonWriter:
  inline def write[A](value: A, tokenWriter: TokenWriter) = ${ writeImpl[A]('value, 'tokenWriter) }

  inline def writeProxy[A](jsonWriter: JsonWriter[A], value: A, tokenWriter: TokenWriter): Unit = jsonWriter.write(value, tokenWriter)

  def writeImpl[A : Type](value: Expr[A], tokenWriter: Expr[TokenWriter])(using q: Quotes): Expr[Unit] =
    Expr.summon[JsonWriter[A]] match
      case Some(jsonWriter) =>
        '{ JsonWriter.writeProxy(${jsonWriter}, ${value}, ${tokenWriter}) }
      case None =>
        defaultWriteImpl(value, tokenWriter)

  def defaultWriteImpl[A : Type](value: Expr[A], tokenWriter: Expr[TokenWriter])(using q: Quotes): Expr[Unit] =
    import quotes.reflect.*

    Expr.summon[Mirror.Of[A]] match
      case Some('{ $m: Mirror.ProductOf[A] { type MirroredElemLabels = labels; type MirroredElemTypes = types } }) =>
        JsonWriterHelpers.writeJsonObject(value, tokenWriter, Type.of[labels], Type.of[types])

      case _ =>
        report.errorAndAbort(s"JsonWriter.write not supported for type ${Type.show[A]}")

  given JsonWriter[Int] with
    inline def write(value: Int, tokenWriter: TokenWriter): Unit =
      tokenWriter.writeNumber(value)

  given JsonWriter[String] with
    inline def write(value: String, tokenWriter: TokenWriter): Unit =
      tokenWriter.writeString(value)

  given JsonWriter[Boolean] with
    inline def write(value: Boolean, tokenWriter: TokenWriter): Unit =
      tokenWriter.writeBoolean(value)

JsonWriterHelpers.scala:

import scala.quoted.*
import scala.collection.mutable.ArrayBuffer

object JsonWriterHelpers:
  def writeJsonObject[A : Type](value: Expr[A], tokenWriter: Expr[TokenWriter], labels: Type[?], types: Type[?])(using Quotes) =
    val statements = ArrayBuffer.empty[Expr[Unit]]

    statements.append('{ ${tokenWriter}.writeObjectStart() })
    writeObjectFields(value, tokenWriter, labels, types, acc = statements, needsLeadingSeparator = false)
    statements.append('{ ${tokenWriter}.writeObjectEnd() })

    Expr.block(statements.toList, '{})


  private def writeObjectFields[A : Type](value: Expr[A], tokenWriter: Expr[TokenWriter], labels: Type[?], types: Type[?], acc: ArrayBuffer[Expr[Unit]], needsLeadingSeparator: Boolean)(using q: Quotes): Unit =
    import quotes.reflect.*

    (labels, types) match
      case ('[EmptyTuple], '[EmptyTuple]) =>
        {}
      case ('[label *: labelsTail], '[tpe *: typesTail]) =>
        val fieldName = TypeRepr.of[label] match
          case ConstantType(StringConstant(name)) => name
        val memberSymbol = TypeRepr.of[A].classSymbol.get.fieldMember(fieldName)
        val fieldValue = Select(value.asTerm, memberSymbol).asExprOf[tpe]

        if needsLeadingSeparator then
          acc.append('{ ${tokenWriter}.writeSeparator() })

        acc.append('{ ${tokenWriter}.writeFieldName(${Expr(fieldName)}) })
        acc.append('{ JsonWriter.write[tpe](${fieldValue}, ${tokenWriter}) })

        writeObjectFields(value, tokenWriter, Type.of[labelsTail], Type.of[typesTail], acc, needsLeadingSeparator = true)

Test.scala:

case class Foo(x: Int, bar: String)

object Foo:
  given JsonWriter[Foo] with
    inline def write(value: Foo, tokenWriter: TokenWriter): Unit =
      tokenWriter.writeObjectStart()
      tokenWriter.writeFieldName("x")
      tokenWriter.writeNumber(value.x)
      tokenWriter.writeSeparator()
      tokenWriter.writeFieldName("bar")
      tokenWriter.writeString(value.bar)
      tokenWriter.writeSeparator()
      tokenWriter.writeFieldName("$extraField")
      tokenWriter.writeNumber(123)
      tokenWriter.writeObjectEnd()

case class Qux[A](nested: A)

object Qux:
  inline given [A]: JsonWriter[Qux[A]] with
    inline def write(value: Qux[A], tokenWriter: TokenWriter): Unit =
      tokenWriter.writeObjectStart()
      tokenWriter.writeFieldName("value")
      JsonWriter.write(value.nested, tokenWriter)
      tokenWriter.writeObjectEnd()

case class Baz(y: Boolean, foo: Foo)

@main def test() =
  val foo = Foo(42, "hello")
  val baz = Baz(true, foo)
  val qux = Qux(baz)
  val writer = new StdoutTokenWriter

  JsonWriter.write(qux, writer)

[goshacodes]

Thank you for diving so deep into it! I'll try to use it this week

[goshacodes]

That workaround is making things too hard in my opinion. I have one more idea how to make things work better without inlines. Thank you anyway. I'm not closing this, but you are totally free to do so

[goshacodes]

Not found a way which can potentially satisfy users.
I changed some code, so that i don't need an inline def in macro. But I still need to create a derived instance somehow in derived method.

inline def derived[A](using
      mirror: Mirror.ProductOf[A]
  ): JsonObjectWriter[A] = new JsonObjectWriter[A]:
    override inline def writeValues(value: A, tokenWriter: TokenWriter): Unit =
      Derivation.writeProductTokens[A](value, tokenWriter)

And the code which tries to do that fails with

Implementation restriction: nested inline methods are not supported

Is it somehow possible to support nested inline methods in context of derived?

[prolativ]

I don't think handling things specially for derived would be any easier than removing the restriction in general. Maybe @jchyb could shed some more light on how feasible that would be, but anyway, if this was actually doable, I wouldn't expect this to work any time soon