Closed
Description
Right now this is done while lexing:
rust/src/libsyntax/parse/lexer/mod.rs
Lines 423 to 435 in f861b6e
While that may seem like it avoids going twice over the file contents, lexing is slow, while newline and non-width-1 (which is also improperly computed by means of multi-byte characters, instead of the unicode-width
library) character finding can be optimized one way or another.
Cleaning this up would get rid of hacks such as the save_new_lines_and_multibyte
flag in the code linked above and having to reimplement this in anything else using CodeMap
outside of libsyntax.