Description
Is your feature request related to a problem? Please describe.
Currently, when using code coverage with complex boolean expressions in conditional statements (if
, elif
), the reporting lacks sufficient granularity.
For example, consider a condition like:
if a or b or c or d:
# code block A
elif e or f:
# code block B
else:
# code block C
Standard branch coverage often marks the entire line if a or b or c or d:
as "covered" if the condition evaluates to True
, regardless of which specific sub-expression (a
, b
, c
, or d
) caused it to be true due to short-circuiting. This means we might execute code block A
because b
was True
, but never actually test the cases where a
, c
, or d
would have individually caused the condition to be True
.
This makes it difficult to ensure that all logical paths within a complex boolean expression have been exercised by the test suite, particularly in large or legacy codebases.
Describe the solution you'd like
I would like the coverage tool to provide more detailed insight into the evaluation of complex boolean expressions within conditionals. Specifically, it would be beneficial to know which individual sub-expressions within a compound condition were evaluated and which ones were decisive in determining the outcome (True or False) of the condition.
Ideally, the coverage report could indicate if a boolean expression was only "partially" covered, meaning not all potential evaluation paths or short-circuiting scenarios within that specific condition were tested.
Possible ways to report this could include:
-
Annotated Source View: Highlighting or annotating parts of the boolean expression that were evaluated or determined the outcome during a test run.
# Example annotation showing 'b' was the decisive factor: if a # F, evaluated # or b # T, decisive # or c # Not evaluated # or d # Not evaluated #: ...
Or perhaps marking the line as "partial" if not all sub-expressions were evaluated across all test runs hitting that line.
-
Detailed Report/Table: A separate report or section detailing complex conditions and which specific components within them were triggered by which test cases.
Let's consider the expression if a or (b and (c or d)):
.
Test Case | Section 1 (a ) Evaluation |
Section 2 (c or d ) Evaluation (if reached) |
Section 3 (b and (c or d) ) Evaluation |
Overall Condition Outcome |
---|---|---|---|---|
test_a_is_true |
True | Short-Circuited | Short-Circuited | True |
test_a_false_b_true_c_true |
False | True (because c is True) |
True (because b & Sec 2 are True) |
True |
test_a_false_b_true_d_true |
False | True (because d is True) |
True (because b & Sec 2 are True) |
True |
test_a_false_b_true_c_d_false |
False | False | False (because Sec 2 is False) | False |
test_a_false_b_false |
False | Short-Circuited | False (because b is False) |
False |
Please note that I'm only advocating for this to be shown on or
since the current solution works amazing for and
.
Describe alternatives you've considered
The primary alternative is to refactor the complex boolean expressions into simpler conditions, potentially breaking them down into multiple lines, intermediate variables, or helper functions.
However, this presents a catch-22 situation: refactoring risky legacy code requires high confidence in the existing test suite to ensure no behavior is changed unintentionally. Without detailed condition coverage information, it is difficult to know if the current tests are sufficient to cover all the nuances of the existing complex logic before attempting to refactor it. This feature would provide the necessary insight to confidently identify testing gaps and improve coverage prior to or alongside refactoring efforts.