cyclic reference crash with `-from-tasty` with parents/params loop

[bishabosha]

Compiler Version

3.0.2-RC1

Minimized code

I can now make a cyclic reference crash when compiling -from-tasty with this augmented example to tests/pos/i12834.scala

class C(val d: D, val ref: Option[d.Type])
class D extends C(null, None) {
  type Type
}

Exception in thread "main" java.lang.AssertionError: assertion failed: Cyclic reference while unpickling definition at address 47 in unit tests/pos/i12834.scala
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:770)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:122)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:167)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:610)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:602)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:644)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:642)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1223)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1297)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1297)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.$anonfun$4(TreeUnpickler.scala:932)
	at dotty.tools.tasty.TastyReader.collectWhile(TastyReader.scala:137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:935)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:857)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:122)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:167)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.computeMembersNamed(SymDenotations.scala:2030)
	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.membersNamed(SymDenotations.scala:2000)
	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.findMember(SymDenotations.scala:2051)
	at dotty.tools.dotc.core.Types$Type.go$1(Types.scala:683)
	at dotty.tools.dotc.core.Types$Type.findMember(Types.scala:870)
	at dotty.tools.dotc.core.tasty.TreeUnpickler.dotty$tools$dotc$core$tasty$TreeUnpickler$TreeReader$$_$accessibleDenot$1(TreeUnpickler.scala:1091)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.completeSelect$1(TreeUnpickler.scala:1082)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readSimpleTerm$1(TreeUnpickler.scala:1110)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1297)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTpt(TreeUnpickler.scala:1327)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.$anonfun$27(TreeUnpickler.scala:1263)
	at dotty.tools.tasty.TastyReader.until(TastyReader.scala:125)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1263)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1297)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTpt(TreeUnpickler.scala:1327)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:875)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedParams$$anonfun$2(TreeUnpickler.scala:1042)
	at dotty.tools.tasty.TastyReader.collectWhile(TastyReader.scala:137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedParams(TreeUnpickler.scala:1042)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readParams(TreeUnpickler.scala:1047)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readParamss$1(TreeUnpickler.scala:794)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:830)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:948)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:857)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:776)

before #12844 this code would create the same error as #12834

Originally posted by @bishabosha in #12844 (comment)

[odersky]

This looks pretty hard to fix. What happens is:

• we try to read fully class C
• this means we need to read & typecheck its parameter definitions
• this means we need to know what D is since we have a d.type
• this means we need to complete D
• completion of a class means completion of its parents
• so we need to complete C again which gives a CyclicReference

One way to fix this would be to carefully analyze what Namer does in this case (it's not pretty!) and duplicate the same behavior in TreeUnpickler. But I don't have the bandwidth to do this right now, so it would be good is someone else could take a look. @bishabosha can you try, or find somebody else to take this on?

[nicolasstucki]

Can be reproduced with scalac -Ytest-pickler

[KacperFKorban]

I found a very similar issue recently when trying to migrate scala.rx to Scala 3.

Reproduction:

// _1.scala
class Ctx(rx0: Ctx.Owner)

object Ctx {
  class Owner() extends Ctx(???)
}

// _2.scala
def test = {
  Ctx.Owner
}

Crash msg:

java.lang.AssertionError: assertion failed: Cyclic reference while unpickling definition at address 35 in unit main.scala while compiling Ctx.tasty
Exception in thread "main" java.lang.AssertionError: assertion failed: Cyclic reference while unpickling definition at address 35 in unit main.scala
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:796)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.liftedTree1$1(TreeUnpickler.scala:133)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:137)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:171)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:187)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:189)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:613)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.signature(Denotations.scala:605)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:647)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:645)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1299)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1373)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readLengthTerm$1(TreeUnpickler.scala:1210)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1373)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.$anonfun$19(TreeUnpickler.scala:962)
	at dotty.tools.tasty.TastyReader.collectWhile(TastyReader.scala:137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:965)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:883)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:802)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.liftedTree1$1(TreeUnpickler.scala:133)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:137)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:171)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:187)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:189)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.ensureCompleted(SymDenotations.scala:373)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.annotations(SymDenotations.scala:228)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.getAnnotation(SymDenotations.scala:249)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.targetName(SymDenotations.scala:530)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.hasTargetName(SymDenotations.scala:520)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:657)
	at dotty.tools.dotc.core.Denotations$SingleDenotation.atSignature(Denotations.scala:645)
	at dotty.tools.dotc.core.tasty.TreeUnpickler.dotty$tools$dotc$core$tasty$TreeUnpickler$TreeReader$$_$accessibleDenot$1(TreeUnpickler.scala:1130)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.completeSelect$1(TreeUnpickler.scala:1121)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readSimpleTerm$1(TreeUnpickler.scala:1149)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTerm(TreeUnpickler.scala:1373)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTpt(TreeUnpickler.scala:1401)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:911)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:802)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedParams$$anonfun$2(TreeUnpickler.scala:1083)
	at dotty.tools.tasty.TastyReader.collectWhile(TastyReader.scala:137)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedParams(TreeUnpickler.scala:1083)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readParams(TreeUnpickler.scala:1088)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readParamss$1(TreeUnpickler.scala:820)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:856)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:802)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readTemplate(TreeUnpickler.scala:978)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readNewDef(TreeUnpickler.scala:883)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$TreeReader.readIndexedDef(TreeUnpickler.scala:802)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.liftedTree1$1(TreeUnpickler.scala:133)
	at dotty.tools.dotc.core.tasty.TreeUnpickler$Completer.complete(TreeUnpickler.scala:137)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:171)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:187)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:189)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.ensureCompleted(SymDenotations.scala:373)
	at dotty.tools.dotc.core.Symbols$ClassSymbol.rootTreeContaining(Symbols.scala:396)
	at dotty.tools.dotc.core.Symbols$ClassSymbol.rootTree(Symbols.scala:387)
	at dotty.tools.dotc.fromtasty.ReadTasty.compilationUnit$1(ReadTasty.scala:40)
	at dotty.tools.dotc.fromtasty.ReadTasty.readTASTY(ReadTasty.scala:70)
	at dotty.tools.dotc.fromtasty.ReadTasty.runOn$$anonfun$1(ReadTasty.scala:25)
	at scala.collection.immutable.List.flatMap(List.scala:293)
	at dotty.tools.dotc.fromtasty.ReadTasty.runOn(ReadTasty.scala:25)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:234)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1328)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:245)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:253)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:262)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:262)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:192)
	at dotty.tools.dotc.fromtasty.TASTYRun.compile(TASTYRun.scala:14)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
	at dotty.tools.dotc.Driver.process(Driver.scala:195)
	at dotty.tools.dotc.Driver.process(Driver.scala:163)
	at dotty.tools.dotc.Driver.process(Driver.scala:175)
	at dotty.tools.dotc.Driver.main(Driver.scala:205)
	at dotty.tools.MainGenericCompiler$.run$1(MainGenericCompiler.scala:162)
	at dotty.tools.MainGenericCompiler$.main(MainGenericCompiler.scala:186)
	at dotty.tools.MainGenericCompiler.main(MainGenericCompiler.scala)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at coursier.bootstrap.launcher.a.a(Unknown Source)
	at coursier.bootstrap.launcher.Launcher.main(Unknown Source)

The difference here is that it crashed the compiler ever since 3.0.0 (and probably earlier).

The weird thing is also that the following code compiles correctly: (Owner and Ctx are put in different .tasty files)

// _1.scala
class Ctx(rx0: Ctx.Owner)

class Owner() extends Ctx(???)

Details

The situation here seems to be similar to what @odersky described.

1. Try to unpickle object Ctx
2. Try to unpickle class Owner (requires its parents -> class Ctx)
3. Try to unpickle class Ctx (requires params of it's constructor -> class Owner)
4. Try to unpickle class Owner -> Cyclic reference error.