Syntax highlighting is broken for list comprehensions

[bitemyapp]

Result of running chris-done style from hindent on some Yesod code.

[geraldus]

I also use similar style, but I believe it does not matter and this is related to recent quasi-quoters updates.

Thank for report.

[gracjan]

Does this code parse in GHC when TemplateHaskell is enabled?

[gracjan]

No, it does not:

GHCi, version 7.10.2: http://www.haskell.org/ghc/  :? for help
Prelude> :set -XQuasiQuotes 
Prelude> let x = [ v | v <- [1,2,3]]
Prelude> let x = [v| v <- [1,2,3]]
<interactive>:4:12: unterminated quasiquotation at end of input
Prelude> 

[geraldus]

hvr:
well... there is an ambiguity
…
well, it's been a long known issue

I tested this. Adding spaces around variable name in Chris' example fixes fontifycation. So, should we force users to always put surrounding spaces even if Template Haskell is not enabled?

Here is some complicated example:

hvr:
[here| here <- [1,2,3]] -- |]
is this a list-comprehension or a quasiquotation?
where 'here' is from http://hackage.haskell.org/package/heredoc-0.2.0.0/docs/Text-Heredoc.html

• to support both options we have to check cabal file for default extension enabled for current file and a list of pragmas defined in file itself
• in this case we can adjust lexer how to parse list comprehension
• I'm not sure how much complexity will this approach bring

[gracjan]

First QuasiQuote syntax required a $ like this:

    let x = [$sql| SELECT * FROM table |]

Then somebody decided to make the lexing ambiguous.

Overall I find this frustrating. There are plenty of $ in Haskell in useless places just before do:

    runTransaction $ do
        foo bar

How about removing requirement for useless dollars and using them where they are actually useful?

[bgamari]

@gracjan I agree that the ambiguity in quoting syntax is a bit unfortunate. I've opened a GHC ticket to track the ambiguity here. In short, Simon removed the requirement for a dollar sign in 2010, with the intent of removing the old syntax entirely. Given how long ago this was, I suspect it is too late to revert this change.

[gracjan]

Thanks @bgamari. So what do we do with the pink flood in @bitemyapp screenshot?

[geraldus]

I'm afraid all we can is to offer surrounding spaces.

[bgamari]

It's a tricky question; here are various thoughts in no particular order,

• As I said earlier, I think dropping the requirement for a dollar sign was a mistake, but any attempt at reverting this change would take several GHC releases (e.g. years) since it would be a breaking change.
• it's a shame you can't rely on the presence of a LANGUAGE QuasiQuotes pragma to modify the parser behavior since the extension may be specified in a Cabal file or on the command line
• I can think of various hacks introducing lookahead into the parser, looking for matching close brackets but this would be terrible for both performance and highlighting robustness
• I find the style of omitting spaces around the pipe in a list comprehension to be rather unreadable; of course, this doesn't excuse our inability to syntax highlight it sensibly.
• We could consider mandating whitespace around this pipe, although I'm not terribly excited about this idea; while I don't like the style, I like even less the idea of making Haskell even more whitespace sensitive than it already is
• I really can't think of any other good solution here

So, short of simply recommending that people include whitespace around the list comprehension pipe, I'm not sure we have any good course of action here. This seems like a pretty terrible situation on the whole; we could consider beginning the process of reintroducing the dollar sign in quasi-quotes.

[gracjan]

@bitemyapp, seems like we cannot do much here. I've reported this issue to hindent https://github.com/chrisdone/hindent/issues/180 so that it does not generate invalid code when QuasiQuotes are enabled.

Syntax ambiguity is very unfortunate.

[purcell]

One possibility would be to add a local variable like haskell-font-lock-enable-quasi-quotes, defaulting to nil. Only when that var is set would we font-lock the quasi-quoting syntax. If we can trivially detect TH or QuasiQuotes for the current file, then we'd enable that var automatically, and TH-heavy projects could enable it in their .dir-locals.el.

I'm not a big fan of adding lots of special-case config vars, though.

There are presumably many other cases of syntax ambiguity depending on which exact language extensions are enabled in the current file, e.g. LambdaCase.

So a variation on this approach might be to provide a general way of detecting (or user-specifying) the language extensions active for a project and/or listed in each file, and then using that information to inform the font-lock choices on a file-by-file basis.

[bgamari]

@purcell's suggestion may be the best option we realistically have. Moreover, with cooperation from ghc-mod you could potentially query GHC to determine whether QuasiQuotes is enabled, which would be reliable at least in the case where the user is editing a Cabal package. This could be made even more reliable by also parsing LANGUAGE pragmas in the current file.