Optional braces, part 2

[eed3si9n]

Ref #43

Problem

Currently Scala 3 optional braces syntax does not work.
Specifically, in this PR I want to address decls/defns not properly grouped under the right owner in part 1 (#61).

Solution

1. This brings in indent/outdent impl by @keynmol
2. Introduce _indentable_expression.
   This allows us to limit where indent/outdent shows up without
   confusing tree-sitter too much.
   Internally I've made a change to track last_indentation_size in the payload to indicate last
   outdent (default: -1). If it's not -1 that means there was an outdent.
   This then checks if the last_indentation_size qualified for another outdent.
3. Add support for control strctures:
   • if expression
   • try-catch expression
   • match expression
   • while expression
   • for expression

The test cases for these are:

=======================================
Top-level Definitions (Scala 3 syntax)
=======================================

class A:
  def a() =
    ()

def a() = 1

---

(compilation_unit
  (class_definition
    (identifier)
    (template_body
      (function_definition
        (identifier)
        (parameters)
        (indented_block
          (unit)))))
  (function_definition
    (identifier)
    (parameters)
    (integer_literal)))

  =======================================
  Value declarations (Scala 3 syntax)
  =======================================
  
  class A:
-   val b: String
+   val b, c : Int
+   val d : String

Demo

Here's a demonstration of before and after using Sonokai. The theme is a bit too color happy, but it's good for checking what changed:

Before #61

Note: def run is not recognized as a method.

After this PR

def run is highlighted as a method. The body of def dealiasBaseDirectory improved as well, since val is now acceptable without { ... }.

[ckipp01]

I don't have much to add apart from what @keynmol already mentioned. I think we're all on the same page that to fully handle the new syntax would need some more C work, but this is a significant improvement over what we had before.

Thanks @eed3si9n

[eed3si9n]

For later record keeping, I described some of the changes in this PR in https://eed3si9n.com/fast-scala3-parsing-with-tree-sitter/ post as well.

I think the next challenge would be handling fewer braces syntax, since likely newlines starts to add semantics, and thus can't be ignored globally.

[keynmol]

I think before that (=fewer braces) we'll need to run this on a larger codebases - I.e. I've been using your fork and it still has problems in quite a few places, so I would suggest ignoring fewerBraces completely and instead getting the number of correctly parsed files up on something like Dotty codebase.

There are several of my PRs for Scala 3 I will now revive and ensure that they work with your changes as well, which should improve things.

Additionally, would be good to keep track of Scala 2 as well, for which we can also use the code of Scala 2 compiler as a bench.

To get that locally, you can use this command (thanks to @ckipp01 for showing me the light):

> tree-sitter parse ~/projects/dotty/compiler/**/*.scala --quiet --stat | tail -n2
Total parses: 745; successful parses: 282; failed parses: 463; success percentage: 37.85%

[ckipp01]

I think before that (=fewer braces) we'll need to run this on a larger codebases - I.e. I've been using your fork and it still has problems in quite a few places, so I would suggest ignoring fewerBraces completely and instead getting the number of correctly parsed files up on something like Dotty codebase.

Yea I'm in agreement here. Even without fewer braces I still hit on a lot of issues. I think if we get your stuff revived and in @keynmol I can start paying way more attention to errors in the tree and reporting them now that I feel more confident that we can get things merged in.

[eed3si9n]

Yea. Generally I agree that we should try to go after lower hanging Scala 3 constructs first, but I was replying to the "need some more C work" part.