Distributable binaries and install-less VS Code extension

[lukel97]

It's becoming increasingly obvious that one of the major turnoffs for newcomers is the manual installation. Since we're pulling in so many dependencies, and for a complete install of all the different versions this can take multiple hours. It's a problem that we've had in haskell-ide-engine, and has carried over to haskell-language-server.

One of the most promising solutions was the idea of static binaries. Much work has already been carried out to make haskell-ide-engine more portable, by removing data files in hlint and libexec runtime executables in cabal-helper. And now with haskell-language-server and ghcide, I am not aware of any major blockers.

I've started working on a GitHub Actions workflow to generate binaries of haskell-language-server, building across Linux, macOS and Windows for GHC 8.6 through to 8.10. It gets triggered whenever a new release is created, then after building the binaries uploads them to the release as assets.

My aim is to have the vscode-hie-server VS Code extension automatically download these binaries for the appropriate platform and GHC version on an ad-hoc basis. This way newcomers can simply just install the extension from the VS Code marketplace, create a .hs file and start coding – without the need to build ghcide/haskell-language-server from source, or even having to install Stack or Cabal.

The only prerequisite is that GHC still needs to be installed, for the GHC libdir that the GHC API uses. And more importantly, this libdir (usually /usr/local/lib/ghc-8.10.1/ or something like that) needs to be fetched on the fly for static binaries – currently (in the non-nix scenario) it is obtained via ghc-paths, which bakes the path the libdir at compile time. Unfortunately this path will be different on the GitHub Actions runners than it will be on the users machine, so we will have to go back to how haskell-ide-engine called ghc --print-libdir to get this on the fly:

https://github.com/haskell/haskell-ide-engine/blob/aca3241eb0ea00de171f791d48a4861820327132/hie-plugin-api/Haskell/Ide/Engine/Cradle.hs#L99-L131

ghc is cradle specific however – on Stack, it will be the ghc that it downloaded, whilst for Cabal it may depend on the --with-compiler flag. Therefore I thought it made sense to delegate this work to hie-bios. I have a patch for hie-bios which returns the libdir alongside the GHC flags, calling stack exec ghc and cabal v2-exec ghc as needed. Or just ghc --print-libdir if a direct cradle is being used

One last crazy idea might be to actually start shipping and downloading the GHC libraries, so that there are zero dependencies entirely. How feasible this actually is, I'm not sure.

[lukel97]

I'm testing out the releases here, on the branch https://github.com/bubba/haskell-language-server/tree/github-action-builds. A typical GitHub Actions run is here.

I haven't pushed any code to the vscode extension yet, but it's currently just querying the GitHub REST API to get the list of current releases. One minor note is that since it's making unauthenticated requests, it's rate limited to 60 requests per hour per IP address.

I've also added a new --project-ghc-version argument to haskell-language-server-wrapper, which consults the cradle and prints the project's GHC version, and then exits. The aim is that the vscode extension will download the wrapper first, call it with this argument to first get the project version, and then download the corresponding GHC version of haskell-language-server, if it isn't already downloaded

[pepeiborra]

I have wanted this since day one, it's one of the (many) things that @chrisdone got completely right with intero. Reading this plan felt like every question that came to mind had already been answered in the paragraph below. Moving the libdir query to hie-bios feels right to me, and the --project-ghc-version wrapper is reasonable too.

Looking forward to this!!

[puffnfresh]

Running binaries on NixOS can be a big pain. I would love to help make this work.

[fendor]

I am not aware of any major blockers.

Last "major" blocker is brittany. It uses ghc-paths as well to find the libdir for certain code-paths.
To fix this, the internal API can be used: https://hackage.haskell.org/package/brittany-0.12.1.1/docs/Language-Haskell-Brittany-Internal.html#v:pPrintModule
But we are missing some important bits, such as Anns from ghc-exactprint which are a bit hard to obtain.

That was the last blocker of static binaries in HIE. @jneira also had a full static binary pipeline ready with Azure Pipelines for the case we would solve it.

[fendor]

@pepeiborra I would be happy for your input on haskell/hie-bios#151.
I have discussed it with @mpickering before, but he wanted to wait a bit to see what is really required by different cradle types, such rules_haskell and obelisk, iirc.

[ndmitchell]

Could we just exclude brittany? It's an AGPL odd-ball which is far from the most important part of HLS.

[fendor]

Yes we could, I would still like to have it fixed eventually.

[lukel97]

The binaries on GitHub actions were built without the -fagpl flag passed to Cabal, so weren’t included in the first place anyway as far as I can tell — a quick strings search for the lib dir revealed nothing

[enricopolanski]

As someone who just tried to setup hls on vscode yesterday, I'm a haskell noob, I can just say it is a painful experience.

Not only downloading and installing everything took a very huge amount of time (more than an hour), but looking at the logs of the installation I was seeing that:

-plenty of stuff was being downloaded up to 3 times (it's like the process of cloning and then executing the install script went on 3 times).

• I was even downloading jQuery at some point, like...why?

https://i.imgur.com/R4O3jQf.png

• why was everything being compiled from sources? I kept getting warnings in console about this or that library not respecting some import rule.

All in all it took between 1 and 2 hours on my 7200u powered machine to have what I consider very basic ide features.

I'm not here to speak with entitlement that I expect everythng to work flawlessly (even tho, that's my experience for much of the languages I tried in my life, old or new ones), just to give feedback, because every time I bring up haskell tooling (especially related to developer experience), there is a very strong survivorship bias from the people who went through it already or don't care.

[jneira]

@bubba thanks to take care of that, needed for so long.
github actions looks nice, no need for azure if they work.

[lukel97]

hie-bios PR for GHC libdir is in haskell/hie-bios#207

[ndmitchell]

FWIW, the JQuery dependency is mine, and it supports our fairly advanced profiling UI for the IDE. It's also cheap to compile. But the point still stands - the experience is not great.