Skip to content

Commit db85592

Browse files
xguptaShivam Gupta
authored and
Shivam Gupta
committed
[Code Coverage] Add a tool to check test coverage of a patch
This script create a patch from the HEAD commit, extract modified or added source files, test case files and source code lines, add coverage instrumentation for the affected source files, runs Lit tests, and records which test cases cause each counter to be executed. Then report the number of test cases executing the counter and the number of test cases executing the counter that are also changed in some way by the patch. Thus providing developer the information of inadequately tested source lines.
1 parent 4a9c71b commit db85592

File tree

3 files changed

+844
-0
lines changed

3 files changed

+844
-0
lines changed

llvm/utils/code-coverage/README.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Patch based Code Coverage git integration
2+
3+
git-check-coverage is a Python script designed to facilitate code coverage integration with Git. This tool allows developers to track code coverage for their commits, providing insights into which parts of the codebase are tested and which are not. One of the Future plan is to also integrate into github ci and provide code coverage data for each pull request to help reviewer in reviewing the changes.
4+
5+
### Requirements
6+
7+
Python 3.x
8+
Git
9+
LLVM build tools (CMake, Ninja)
10+
unidiff library (install via pip)
11+
A recent version of clang/clang++ to build intrumented build directory
12+
LLVM tools (llvm-cov, llvm-profdata, llvm-lit)
13+
14+
### Setup & Usage
15+
16+
```shell
17+
$ cd llvm-project
18+
$ export PATH=/home/user/local-llvm/bin:$PATH
19+
$ mkdir build
20+
$ cmake -G Ninja -Bbuild -Sllvm -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_LINKER=lld -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_ENABLE_ASSERTIONS=ON
21+
$ ninja -C build
22+
$ cp llvm/utils/git-check-coverage .
23+
$ chmod +x git-check-coverage
24+
$ git check-coverage -b build bin/opt llvm/test
25+
```
26+
27+
Parameters
28+
29+
-b or --build-dir: Specify the build directory (default is build).
30+
<binary_name>: Provide the name of the binary to analyze.
31+
<test_suite_path>: Path to the test suite that will be executed.
32+
33+
Optional Arguments
34+
35+
-n or --num-commits: Specify the number of recent commits to analyze.
36+
-i or --instrumented-build-dir: Provide a separate directory for instrumented builds.
37+
38+
> Note : clang and clang++ we are using is of recent version and install in path /home/user/local-llvm/bin.
39+
40+
### Functions Overview
41+
42+
The script contains several key functions:
43+
44+
```
45+
extract_source_files_from_patch(patch_path): Extracts modified and test files from a patch.
46+
write_source_file_allowlist(source_files, allowlist_path): Writes an allowlist for source files.
47+
build_llvm(build_dir): Configures and builds LLVM in the specified build directory.
48+
run_modified_lit_tests(llvm_lit_path, patch_path, tests): Runs modified lit tests identified from the patch file.
49+
process_coverage_data(cpp_files, build_dir, binary): Converts profraw coverage data to profdata format and generates human-readable reports.
50+
read_coverage_file(coverage_file): Reads a coverage file to return covered and uncovered line numbers.
51+
print_common_uncovered_lines(cpp_file, uncovered_lines, modified_lines): Displays common uncovered lines for each modified source file.
52+
find_lit_tests(lit_path, test_paths): Identifies lit tests based on the provided paths.
53+
```
54+
55+
### Known Issues
56+
57+
1. Unit test
58+
2. LLDB tests(.py) coverage
59+

0 commit comments

Comments
 (0)