2.10.0.0: Bindist not working when built with GHC versions ≥ 9.10.1

[benz0li]

Your environment

Which OS do you use? Alpine Linux 3.21.3

Which version of GHC do you use and how did you install it? 9.10.1
👉 Docker image quay.io/benz0li/ghc-musl:9.10.1 (https://github.com/benz0li/ghc-musl)

Which version of HLS do you use and how did you install it? See below.

Steps to reproduce

On a host with docker installed

docker run --rm -ti quay.io/benz0li/ghc-musl:9.10.1 bash

Inside the container

Build HLS bindist:

export HLS_VERSION=2.10.0.0 \
  && apk add --no-cache patchelf findutils \
  && cd /tmp \
  && curl -sSL "https://github.com/haskell/haskell-language-server/archive/refs/tags/$HLS_VERSION.tar.gz" \
    -o "haskell-language-server-$HLS_VERSION.tar.gz" \
  && tar -xzf "haskell-language-server-$HLS_VERSION.tar.gz" \
  && cd "haskell-language-server-$HLS_VERSION" \
  && . .github/scripts/env.sh \
  && . .github/scripts/common.sh \
  && sed -i.bak -e '/DELETE MARKER FOR CI/,/END DELETE/d' cabal.project \
  && GHCS="$GHC_VERSION" \
  && export GHCS \
  && ARTIFACT="$(uname -m)-linux-alpine" \
  && export ARTIFACT \
  && cabal update \
  && emake hls-ghc \
  && CABAL_VERSION_NUMERIC="$(cabal --numeric-version)" \
  && if dpkg --compare-versions "${CABAL_VERSION_NUMERIC%.*.*}" ge "3.11"; then \
    sed -i 's|"$(STORE_DIR)/ghc-$(GHC_VERSION)"|"$(STORE_DIR)/ghc-$(GHC_VERSION)-inplace"|g' \
      GNUmakefile; \
  fi \
  && emake bindist \
  && strip "out/bindist/$ARTIFACT/haskell-language-server-$HLS_VERSION/lib/$GHC_VERSION"/*.so \
  && emake bindist-tar

Install from bindist:

cd /var/tmp/
cp /tmp/haskell-language-server-"$HLS_VERSION"/out/haskell-language-server-"$HLS_VERSION"-aarch64-linux-alpine.tar.xz .
mv /tmp/haskell-language-server-"$HLS_VERSION" /tmp/haskell-language-server-"$HLS_VERSION".bak
tar -xJf haskell-language-server-"$HLS_VERSION"-*-linux-alpine.tar.xz
cd haskell-language-server-"$HLS_VERSION"
make install

Size info:

ls -lh /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

-rwxr-xr-x    1 root     root      203.5K Apr  1 07:49 /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper

File info:

file /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

/usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, BuildID[sha1]=29fc4b7f5ba577757e8c7fc6a40f9339482e014b, stripped

Expected behaviour

shared libraries e.g. for HLS 2.9.0.1:

ldd /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

	/lib/ld-musl-aarch64.so.1 (0xffff885ac000)
	libz.so.1 => /usr/lib/libz.so.1 (0xffff8857b000)
	libgmp.so.10 => /usr/lib/libgmp.so.10 (0xffff884fa000)
	libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff885ac000)

Actual behaviour

shared libraries:

ldd /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

	/lib/ld-musl-aarch64.so.1 (0xffff9cf0d000)
Error loading shared library libHShaskell-language-server-2.10.0.0-c822100215721b3053b21638b3472d2369d7871ad9357fc342d9a5545d638251-ghc9.10.1.so: No such file or directory (needed by /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper)
...

ldd_out.txt

Debug information

-

[benz0li]

When moving the build dir back in place, it shows that the shared libraries are linked to the build dir:

mv /tmp/haskell-language-server-"$HLS_VERSION".bak /tmp/haskell-language-server-"$HLS_VERSION"
ldd /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

	/lib/ld-musl-aarch64.so.1 (0xffff91c18000)
	libHShaskell-language-server-2.10.0.0-c822100215721b3053b21638b3472d2369d7871ad9357fc342d9a5545d638251-ghc9.10.1.so => /tmp/haskell-language-server-2.10.0.0/store/aarch64-linux-alpine/ghc-9.10.1-inplace/haskell-language-server-2.10.0.0-c822100215721b3053b21638b3472d2369d7871ad9357fc342d9a5545d638251/lib/libHShaskell-language-server-2.10.0.0-c822100215721b3053b21638b3472d2369d7871ad9357fc342d9a5545d638251-ghc9.10.1.so (0xffff91ba1000)
...

ldd_out.txt

[benz0li]

@fendor Same result for GHC version 9.12.2. Working HLS 2.10.0.0 bindists for GHC versions 9.6.7 and 9.8.4.

ℹ I noticed, because of the docker image size: https://quay.io/repository/benz0li/hlssi?tab=tags

[fendor]

Did these bindists work for 2.9.0.{0, 1} and GHC 9.10.1?
I don't think these scripts were changed a lot, so I am bit surprised if this is a new regression...

[benz0li]

Did these bindists work for 2.9.0.{0, 1} and GHC 9.10.1?

Yes. Built using

• https://github.com/benz0li/ghc-musl/blob/7621695049ae68c6d1271723d404ac82fc43c380/packages/hls/2.9.0.0.Dockerfile
• https://github.com/benz0li/ghc-musl/blob/7621695049ae68c6d1271723d404ac82fc43c380/packages/hls/2.9.0.1.Dockerfile

and GHC 9.10.1.

[benz0li]

You may test using the GHC musl Dev Container GHC 9.10 and changing HLS_VERSION to either 2.9.0.0 or 2.9.0.1.

[fendor]

I followed your instructions, but the result seems to be working for me:

$ ./haskell-language-server-9.10.1.in 
2025-04-02T08:55:22.509247Z | Info | haskell-language-server version: 2.10.0.0 (GHC: 9.10.1) (PATH: /var/tmp/haskell-language-server-2.10.0.0/bin/haskell-language-server-9.10.1)
2025-04-02T08:55:22.510355Z | Info | Directory: /var/tmp/haskell-language-server-2.10.0.0
2025-04-02T08:55:22.510776Z | Info | Logging heap statistics every 60.00s
 ghcide setup tester in /var/tmp/haskell-language-server-2.10.0.0.
Report bugs at https://github.com/haskell/haskell-language-server/issues

after explicitly replacing @@EXE_NAME@@ with the bin/ path.

But I had the same issue with GHC 9.8.4, like I get a very similar ldd output for GHC 9.8.4 when running ldd bin/haskell-language-server-9.8.4. Once I specify the @@EXE_NAME@@, I can run ./haskell-language-server-9.8.4.in.

[benz0li]

But I had the same issue with GHC 9.8.4, like I get a very similar ldd output for GHC 9.8.4 when running ldd bin/haskell-language-server-9.8.4

Using the GHC musl Dev Container GHC 9.8 (with HLS 2.10.0.0):

Version info:

/usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper --version

haskell-language-server version: 2.10.0.0 (GHC: 9.8.4) (PATH: /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper)

Size info:

ls -lh /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

-rwxr-xr-x 1 root root 177M Mar 31 15:33 /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper

File info:

file /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

/usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-aarch64.so.1, BuildID[sha1]=222385fb0cce874d7b66ab6ca61f9977bdd6558a, stripped

Shared libraries:

ldd /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper

        /lib/ld-musl-aarch64.so.1 (0xffff972bb000)
        libz.so.1 => /usr/lib/libz.so.1 (0xffff9728a000)
        libgmp.so.10 => /usr/lib/libgmp.so.10 (0xffff97209000)
        libc.musl-aarch64.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff972bb000)

[fendor]

Yeah, for GHC 9.10.1, it looks almost identical, except that the hls-wrapper has dynamically linked haskell dependencies as well.

The reduced size looks like a bonus right now, not like an issue.

[benz0li]

after explicitly replacing @@EXE_NAME@@ with the bin/ path.

Where do I need to make adjustments in order to get a working HLS installation [at /usr/local] after a make install?

[fendor]

Your instructions work out-of-the-box for me for GHC 9.10.1 🤷

cd42666586dc:/var/tmp/haskell-language-server-2.10.0.0# ls -lh /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper
-rwxr-xr-x    1 root     root      179.6K Apr  2 09:42 /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper
cd42666586dc:/var/tmp/haskell-language-server-2.10.0.0# /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper --version
haskell-language-server version: 2.10.0.0 (GHC: 9.10.1) (PATH: /usr/local/lib/haskell-language-server-2.10.0.0/bin/haskell-language-server-wrapper)
cd42666586dc:/var/tmp/haskell-language-server-2.10.0.0# ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.10.1
cd42666586dc:/var/tmp/haskell-language-server-2.10.0.0# file /usr/local/lib/haskell-language-server-"$HLS_VERSION"/bin/haskell-language-server-wrapper --version
file-5.46
magic file from /usr/share/misc/magic

That looks correct to me.

[fendor]

@wz1000 pointed out, you might be using cabal 3.14, is that possible?

[benz0li]

@wz1000 pointed out, you might be using cabal 3.14, is that possible?

GHC 9.10.1: cabal 3.12.1.0
GHC 9.12.2: cabal 3.14.1.1

👉 See the Version Matrix for detailed information.

[benz0li]

On both linux/arm64 and linux/amd64 I run into the following using the GHC musl Dev Container GHC 9.10 (with HLS 2.10.0.0):

Clone the Pandoc repository:

mkdir ~/projects/jgm
cd ~/projects/jgm
git clone https://github.com/jgm/pandoc.git

Open /home/vscode/projects/jgm/ via File > Open Folder...

Open File src/Text/Pandoc.hs

Result: The Haskell (pandoc) server crashed 5 times in the last 3 minutes. The server will not be restarted.

Output: Haskell (pandoc):

2025-04-02 09:57:41.9200000 [client] INFO Finding haskell-language-server
2025-04-02 09:57:41.9210000 [client] INFO Searching for server executables haskell-language-server-wrapper,haskell-language-server in $PATH
2025-04-02 09:57:41.9210000 [client] INFO $PATH environment variable: /vscode/vscode-server/bin/linux-alpine/ddc367ed5c8936efe395cffeec279b04ffd7db78/bin/remote-cli:/home/vscode/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-04-02 09:57:41.9250000 [client] INFO Found server executable in $PATH: haskell-language-server-wrapper
2025-04-02 09:57:41.9260000 [client] INFO Activating the language server in working dir: /home/vscode/projects/jgm/pandoc (the workspace folder)
2025-04-02 09:57:41.9260000 [client] INFO run command: haskell-language-server-wrapper --lsp
2025-04-02 09:57:41.9260000 [client] INFO debug command: haskell-language-server-wrapper --lsp
2025-04-02 09:57:41.9260000 [client] INFO server cwd: /home/vscode/projects/jgm/pandoc
2025-04-02 09:57:41.9260000 [client] INFO server environment variables:
2025-04-02 09:57:41.9360000 [client] INFO Starting language server
Error loading shared library libHShaskell-language-server-2.10.0.0-24a4e75616f653cdb4e670fe870ae501bd4c6da788cf9f11e9321f7318fa2764-ghc9.10.1.so: No such file or directory (needed by /usr/local/bin/haskell-language-server-wrapper)
...