Stable-only benchmarks don't run locally

[Nadrieril]

Some benchmarks like encoding are category: stable instead of category: primary/secondary. When I run collector bench_local --include encoding,..., encoding is not run. Is there a way to make it run, short of locally changing its category?

[Kobzol]

It's not currently possible because of this. We can probably just remove that line though.

Out of curiousity, what is your use-case for running a stable benchmark?

[Nadrieril]

Removing this line would fix my problem :D Maybe --include primary,secondary could filter just these for anyone who needs that? I only ever run a selected set of benchmarks myself since I only touch match checking.

My use-case is testing whether we can remove an old hack that specifically regressed encoding (here).

[Kobzol]

@nnethercote Did you have any specific motivation for disabling stable benchmarks in bench_local?

[nnethercote]

Because running the stable benchmarks locally is almost never what you want to do. More detail here.