Line numbers from TASTy

[nicolasstucki]

When we have a position on a tree the was loaded from TASTy we have no way to compute line numbers and columns as we only have the offset but no source file.

We have identified the following scenarios where we require the line numbers from TASTy files:

• Error reporting on code inlined from a library for which we do not have the sources (see Error reporting for inlined code now depends on source code of library #6538 and Fix #6371: Make error message rendering resilient to missing sources #6412)
• Recompiling from TASTy without sources available. The line numbers are required for the bytecode stack trace information.

So far the most robust solutions that we have are:

• Keep the line sizes in the TASTy file. Has a small memory footprint.
• Keep the actual source in the TASTy file. This would have a large footprint but would also give the possibility of printing better errors for inlined positions.

Which solution should be implemented?

[smarter]

Keep the actual source in the TASTy file. This would have a large footprint but would also give the possibility of printing better errors for inlined positions.

I vote for this, but I think we should make the source an optional section of the tasty such that we can still produce some output without crashing if it's stripped for some reason (e.g., proprietary code).

[liufengyun]

As discussed in the meeting, we take the first approach:

Keep the line sizes in the TASTy file. Has a small memory footprint.

[liufengyun]

Storing ranges or even source files would require big refactoring of the TASTY infrastructure, as now TASTY chunks are associated with top-level classes. There is no infrastructure to uniquely identify source files and share data (content/line ranges).

The simplest thing to do seems to convert the representation of Span to (line, column) mode during pickling. The conversion can be lossless, i.e. we keep three pairs of (line, column) for start, end and point respectively. The approach keeps the size & information of TASTY files unchanged, and there is no need to change the infrastructure of TASTY. It only requires change Span to be in two different modes: offset mode and line/column mode. However, during a brief discussion, @odersky mentioned this is a wrong approach, for reasons I don't fully understand.

Any ideas @nicolasstucki @smarter ?

[nicolasstucki]

I had a branch where I added an extra section with the sizes of the lines of the source file. This was not too difficult.

[liufengyun]

I had a branch where I added an extra section with the sizes of the lines of the source file. This was not too difficult.

In your branch, do you share the line ranges for the same source file across library boundaries?

[nicolasstucki]

The lines of the same source. We need to avoid duplication.