Description
Ghcide has an excellent benchmark suite that tracks time and space usage across multiple revisions, experiments and examples. But by running it only on exe:ghcide, we are missing crucial data on the impact of HLS plugins on performance.
What are the steps required to run the ghcide benchmark suite on HLS plugins? And on HLS?
There' more than one option:
✔️ 1. Port the benchmark suite to HLS
We just move the Cabal stanzas (ghcide-bench and bench-hist) and the code across, add new experiments to exercise the relevant plugins, and run it on the whole of HLS + all the plugins.
2. Extract the benchmark suite to a package lsp-bench
As things stand right now, its' not even possible to package integration tests for plugins. All the integration tests live in HLS (func-test). We probably want to avoid ending up in the same situation with benchmarks.
A standalone package would be a way to allow each plugin to define its own benchmark suite.