[llvm-cov] Add new 'llvm-cov' argument '--binaries <binary1> <binary2> ...' to simplify passing in the list of binaries

[bartlettroscoe]

Description

The current llvm-cov tool takes in this of binaries in a very non-conventional and inconvenient way that make it difficult use with simple scripts. As shown in:

llvm-project/llvm/utils/prepare-code-coverage-artifact.py

Lines 53 to 58 in 65b85bf

	objects = []
	for i, binary in enumerate(binaries):
	if i == 0:
	objects.append(binary)
	else:
	objects.extend(("-object", binary))

you have to pass the first binary as an unadorned positional argument and next set of binaries prefixed with -object as:

<binary1> -object <binary2> -object <binary3> ...

That is very non-intuitive and inconvenient and makes it very hard to write simple shell scripts to pass a list of binaries to the llvm-cov command.

For example, I have a simple shell script find-all-llvm-execs.sh:

#!/bin/bash
#
# Find all of the executables built with llvm coverage
#
#   find-all-llvm-execs.sh <base-dir>
#
baseDir=$1
for exe in $(find ${baseDir} -type f -executable); do
  if llvm-readelf --sections "$exe" 2>/dev/null | grep -q '__llvm_covmap'; then
    echo "$exe"
  fi
done

that gathers up a list of binaries that have LLVM coverage support for any arbitrary directory. (It is fast and seems to be pretty robust.)

But I can't just use this to call an llvm-cov command as:

$ llvm-cov show $(./find-all-llvm-execs.sh <some-subdir>) [other args]

because it only uses the first binary and ignores the rest of them. (It took me a while to figure that out.)

But, ironically, the tool prepare-code-coverage-artifact.py takes in a flat list of binaries as shown at:

llvm-project/llvm/utils/prepare-code-coverage-artifact.py

Lines 133 to 139 in 65b85bf

	parser.add_argument(
	"binaries",
	metavar="B",
	type=str,
	nargs="*",
	help="Path to an instrumented binary",
	)

so you can just call:

$ time ./prepare-code-coverage-artifact.py --use-existing-profdata coverage.profdata \
  --unified-report -C ${PWD} $(which llvm-profdata) $(which llvm-cov) ${PWD} \
   cov_report \
   $(./find-all-llvm-execs.sh .)

Whoever wrote prepare-code-coverage-artifact.py knows it is much easier to pass in a flat list of binaries. (They could have duplicated the approach used by llvm-cov, but they did not.)

Suggested solution

A simple fix for this is to add a new llvm-cov argument:

--binaries <binary1> <binary2> <binary3> ...

that operates like the existing --sources <src1> <src2> ... argument. And you could also support the short option name -b (which is not being used yet by llvm-cov as of LLVM 20.1) as:

-b <binary1> <binary2> <binary3> ...

So you could just call:

$ llvm-cov show -b $(./find-all-llvm-execs.sh <subdir>) [other args]

Then I would not even use prepare-code-coverage-artifact.py because it would be so easy to call the individual llvm-cov commands. (Like taking candy from a baby.)

Now, you could keep supporting the existing inconvenient and non-intuitive <binary1> -object <binary2> -object <binary3> ... arguments. The new --binaries <binary1> <binary2> <binary3> ... argument keeps backward compatibility (for people that like that way of passing in binaries), so they can co-exist. (However, the llvm-cov command should likely error out if both modes were used to pass in the list of binaries.)

[bartlettroscoe]

FYI, a workaround is to use a simple command tool to fix the arguments. I wrote the Python tool llvm-cov-bin-args.py:

#!/usr/bin/env python3
#
# Convert '<bin1> <bin2> ...' args from STDIN or cmnd-line to '<bin1> -object
# <bin2> ...' to pass into llv-cov commands.
#
import sys

# Get the list of binaries
if len(sys.argv) > 1:
  binariesList = sys.argv[1:]
else:
  binariesList = []
  for line in sys.stdin:
    binariesList.append(line.strip())

# Print the arguments
print(binariesList[0])
if len(binariesList) > 1:
  for binary in binariesList[1:]:
    print("-object")
    print(binary)

That, together with a simple script like find-all-llvm-execs.sh:

#!/bin/bash
#
# Find all of the executables built with llvm coverage
#
#   find-all-llvm-execs.sh <base-dir>
#
baseDir=$1
for exe in $(find ${baseDir} -type f -executable); do
  if llvm-readelf --sections "$exe" 2>/dev/null | grep -q '__llvm_covmap'; then
    echo "$exe"
  fi
done

and you can call llvm-cov <command> commands like:

llvm-cov show $(find-all-llvm-execs.sh <subdir> | llvm-cov-bin-args.py) [other args]

It is not super pretty but it is a reasonable workaround until --binaries <bin1> <bin2> ... can be added to llvm-cov.