Mark partially evaluated if conditions are not covered

[nedbat]

Originally reported by Marc Schlaich (Bitbucket: schlamar, GitHub: schlamar)

---

This is a follow up from my comment in #198.

Consider the case:

#!python

def test(a, b):
    if a or b:
        print 1

If you run this it counts as covered if either a or b is True. But I think for a fully covered test you should consider both cases (True, False) and (False, True) before it is actually correctly covered. b could be a complicated statement which is never checked in unit tests even if this line has code coverage. Even a bug would get 100% coverage:

#!python

def test(a, b):
    if a or failing_statement:
        print 1

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/292

[nedbat]

Original comment by Marc Schlaich (Bitbucket: schlamar, GitHub: schlamar)

---

Just to be clear, this should be something between normal and branch coverage. In the example testing against (True, False) and (False, True) should be sufficient.

[nedbat]

Thanks, this feature has been requested a few times, but will be very difficult to implement, if possible at all. Here's a blog post from almost six years ago explaining a possible technique: http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html

[nedbat]

Issue #557 was marked as a duplicate of this issue.

[omry]

Another possible approach is code rewrite.

if a or b:
  ...

Can be rewritten as:

def body():
  ...

if a and b:
  body() # body covered via both a and b
elif a and not b:
  body() # body covered via a
elif not a and b:
  body() # body covered via b
else
  ...    # body not covered at all.

[L3viathan]

Since @nedbat's blogpost two things have happened (or will happen) in CPython:

• opcode tracing became a supported feature (by setting frame.f_trace_opcodes to True)
• code objects contain line and character range information per bytecode instruction via the new co_positions() method (this is not yet in a stable release, but is in the latest beta for 3.11)

I've written a small proof-of-concept that shows that this could, in theory, now be done: https://gist.github.com/L3viathan/9be572bbf6659da5b9ae8af023ab3713

However, there's many obstacles in the way, and doing this would probably be quite brittle and require adjustments any time bytecode instructions or even just the way code is compiled is changed.

[mthuurne]

Maybe it's easier to do #509 first, as it would establish some infrastructure for opcode analysis and reporting coverage within a single line of code, which this ticket could then build upon.