Skip to content

Split tests into smaller sub-tests #316

Open
@asmeurer

Description

@asmeurer

Right now there is one test per function. This has worked so far, but it has limitations. The main problem is that if a single thing fails in a test, you can't really get at any of the assertions that come after it. And even if hypothesis could generate separate examples for separate assertions, if you are maintaining a library and you want to xfail a given assertion, you have to xfail the whole test.

This is going to be important for reporting, because it will make it easier to report on exactly which things a given function isn't compliant on. However, this is also a pretty big refactor. It also could be challenging in some cases if some assertion actually depends on a previous assertion working to make sense (e.g., maybe some assertion will implicitly assume a given shape because it was already asserted). For this reason, I would consider this to be lower priority for now, and not worry about implementing it until it becomes needed for reporting (although it certainly doesn't hurt to start thinking about how it might work).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions