Skip to content

Commit fad5388

Browse files
committed
Simplify coverage tests
This change reduces the risk of impacting coverage tests on unrelated changes (such as MIR and Span changes), and reduces the burden when blessing coverage changes in case it is necessary. * Remove all spanview tests. The spanview tests were useful during development, but they can be generated as needed, via compiler command line flags. They aren't critical to confirming coverage results. (The coverage report tests are sufficient.) When spanview regeneration was necessary, the diffs were way too hard to read to be useful anyway. So I'm removing them to reduce friction from a feature that is no longer useful. * Remove the requirement for `llvm-cov show --debug` when blessing tests. The `--debug` flag is, unfortunately, only available if LLVM is built with `optimize = false` (in Rust's config.toml). This adds significant time and resource burdens to the contributor's build. As it turns out, for other reasons in the past, I wasn't actually using the debug output (counter info) to validate coverage anymore either, so it was required for no reason, I now realize.
1 parent 4fa76a4 commit fad5388

File tree

131 files changed

+0
-26664
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

131 files changed

+0
-26664
lines changed

src/test/run-make-fulldeps/coverage-reports/Makefile

-80
Original file line numberDiff line numberDiff line change
@@ -61,13 +61,6 @@ endif
6161
LLVM_COV_IGNORE_FILES=\
6262
--ignore-filename-regex='(uses_crate.rs|uses_inline_crate.rs)'
6363

64-
# When generating `expected_*` results (using `x.py test --bless`), the `--debug` flag is forced.
65-
# If assertions are disabled, the command will fail with an error, rather than attempt to generate
66-
# only partial results.
67-
ifdef RUSTC_BLESS_TEST
68-
DEBUG_FLAG=--debug
69-
endif
70-
7164
all: $(patsubst $(SOURCEDIR)/lib/%.rs,%,$(wildcard $(SOURCEDIR)/lib/*.rs)) $(patsubst $(SOURCEDIR)/%.rs,%,$(wildcard $(SOURCEDIR)/*.rs))
7265

7366
# Ensure there are no `expected` results for tests that may have been removed or renamed
@@ -177,76 +170,3 @@ else
177170
false \
178171
)
179172
endif
180-
181-
####################################################################################################
182-
183-
# The following Makefile content was used to copy the generated `counters` files
184-
# to `expected_` files (when `--bless`ed) and to compare them via `diff`; but for
185-
# multiple reasons, these files cannot easily be used for test validation:
186-
#
187-
# * Output lines can be produced in non-deterministic order (depending on the
188-
# target platform, and sometimes on unrelated codegen changes).
189-
# * Some lines include demangled function names, making them more challenging
190-
# to interpret and compare.
191-
#
192-
# The files are still generated (in `$(TMPDIR)`) to support developers wanting
193-
# to inspect the counters, for debugging purposes.
194-
#
195-
# ifdef RUSTC_BLESS_TEST
196-
# cp "$(TMPDIR)"/[email protected] \
197-
198-
# else
199-
#
200-
# ifdef DEBUG_FLAG
201-
# $(DIFF) [email protected] "$(TMPDIR)"/[email protected] || \
202-
# ( grep -q '^\/\/ ignore-llvm-cov-show-diffs' $(SOURCEDIR)/[email protected] && \
203-
# >&2 echo 'diff failed, but suppressed with `// ignore-llvm-cov-show-diffs` in $(SOURCEDIR)/[email protected]' \
204-
# ) || \
205-
# ( >&2 echo 'diff failed, and not suppressed without `// ignore-llvm-cov-show-diffs` in $(SOURCEDIR)/[email protected]'; \
206-
# >&2 echo '(Ignore anyway until mangled function names in "counters" files are demangled.)' \
207-
# )
208-
# endif
209-
#
210-
# endif
211-
212-
####################################################################################################
213-
214-
# The following Makefile content, and short JSON script, were used to generate
215-
# coverage reports in JSON when the `llvm-cov show` reports were less reliable for
216-
# testing. At the present time, however, the `llvm-cov show` results, and methods
217-
# for comparing them, are working for all tests, making the JSON reports redundant.
218-
#
219-
# If this changes in the future, the scripts are left here, commented out, but can
220-
# be resurrected if desired. This could be used to compare *only* the JSON files;
221-
# and in that case, the `llvm-cov show` reports can be ignored by inserting
222-
# `// ignore-llvm-cov-show-diffs` at the top of the source file.
223-
#
224-
# # Generate a coverage report in JSON, using `llvm-cov export`, and fail if
225-
# # there are differences from the expected output.
226-
# "$(LLVM_BIN_DIR)"/llvm-cov export \
227-
# $(LLVM_COV_IGNORE_FILES) \
228-
# --summary-only \
229-
# --instr-profile="$(TMPDIR)"/[email protected] \
230-
# $(call BIN,"$(TMPDIR)"/$@) \
231-
# | "$(PYTHON)" $(BASEDIR)/prettify_json.py \
232-
# > "$(TMPDIR)"/[email protected]
233-
#
234-
# ifdef RUSTC_BLESS_TEST
235-
236-
# else
237-
# # Check that exported JSON coverage data matches what we expect (`--bless` refreshes `expected`)
238-
239-
# endif
240-
#
241-
# # # If generating coverage reports in JSON, this Makefile is accompanied by
242-
# # # a Python script, `prettify_json.py`, which is defined:
243-
# #
244-
# # #!/usr/bin/env python
245-
# #
246-
# # import sys
247-
# # import json
248-
# #
249-
# # # Try to decode line in order to ensure it is a valid JSON document
250-
# # for line in sys.stdin:
251-
# # parsed = json.loads(line)
252-
# # print (json.dumps(parsed, indent=2, separators=(',', ': '), sort_keys=True))

src/test/run-make-fulldeps/coverage-spanview/Makefile

-92
This file was deleted.

src/test/run-make-fulldeps/coverage-spanview/escape_url.py

-12
This file was deleted.

src/test/run-make-fulldeps/coverage-spanview/expected_mir_dump.abort/abort.main.-------.InstrumentCoverage.0.html

-106
This file was deleted.

0 commit comments

Comments
 (0)