REPL: add syntax highlighting, like the Scala 3 REPL (and Ammonite)

[SethTisue]

as @allanrenucci commented at scala/scala#7645 (comment) ,

Here is the implementation in Dotty for reference. I believe it would be straigforward to port our syntax highlighter

someone want to have a look and see it how even works? does it involve external dependencies?

JLine info page on this is https://github.com/jline/jline3/wiki/Highlighting-and-parsing

[soronpo]

Maybe I'll try to tackle this one.

[SethTisue]

@soronpo are you still interested in working on this?

[soronpo]

Yes, but currently swamped with work to finish my Phd.
If anyone wants to tackle this, be my guest.

[SethTisue]

@retronym passed along a link to this which might be useful: scala/scala@60ee992

oh, but Paul ripped some of that out again in scala/scala@317a105

[retronym]

scala> val unit = newCompilationUnit("class      C { def foo = for (x <- (1 to 10)) yield null }"); val scanner = new syntaxAnalyzer.UnitScanner(unit); scanner.init(); while (scanner.token != ast.parser.Tokens.EOF) { val x = scanner.toString; val o = scanner.offset; scanner.
nextToken(); println((x, o, scanner.lastOffset))}
('class',0,5)
(id(C),11,12)
('{',13,14)
('def',15,18)
(id(foo),19,22)
('=',23,24)
('for',25,28)
('(',29,30)
(id(x),30,31)
('<-',32,34)
('(',35,36)
(int(1),36,37)
(id(to),38,40)
(int(10),41,43)
(')',43,44)
(')',44,45)
('yield',46,51)
('null',52,56)
('}',57,58)
val unit: $r.intp.global.CompilationUnit = <console>
val scanner: $r.intp.global.syntaxAnalyzer.UnitScanner = eof

[retronym]

If you'd like to detect // comments, a small change to the the parser API would be needed.

Here's what can be done already:

val unit = newCompilationUnit("class      C { def foo = for (x <- (1 to 10)) yield null /* C1 */ // C2}"); val scanner = new global.syntaxAnalyzer.UnitScanner(unit, Nil) { override def processCommentChar(): Unit = { println((s"COMMENT($ch)", charOffset)) } }; scanner.
init(); while (scanner.token != ast.parser.Tokens.EOF) { val x = scanner.toString; val o = scanner.offset; scanner.nextToken(); println((x, o, scanner.lastOffset))}
('class',0,5)
(id(C),11,12)
('{',13,14)
('def',15,18)
(id(foo),19,22)
('=',23,24)
('for',25,28)
('(',29,30)
(id(x),30,31)
('<-',32,34)
('(',35,36)
(int(1),36,37)
(id(to),38,40)
(int(10),41,43)
(')',43,44)
(')',44,45)
('yield',46,51)
(COMMENT( ),60)
(COMMENT(C),61)
(COMMENT(1),62)
(COMMENT( ),63)
(COMMENT(*),64)
(COMMENT(/),65)
('null',52,56)
val unit: $r.intp.global.CompilationUnit = <console>
val scanner: $r.global.syntaxAnalyzer.UnitScanner = eof