Property-Based Testing with Hypothesis

Discover edge cases automatically by testing properties instead of examples.

Overview

Testing functions with many possible inputs
Validating invariants that must hold for all inputs
Finding boundary conditions and edge cases
Testing serialization/deserialization roundtrips
Stateful testing of APIs and state machines

Quick Reference

Example-Based vs Property-Based

# Example-based: Test specific inputs
def test_sort_examples():
    assert sort([3, 1, 2]) == [1, 2, 3]
    # But what about [-1], [1.5, 2.5], ...?

# Property-based: Test properties for ALL inputs
from hypothesis import given
from hypothesis import strategies as st

@given(st.lists(st.integers()))
def test_sort_properties(lst):
    result = sort(lst)
    assert len(result) == len(lst)  # Same length
    assert all(result[i] <= result[i+1] for i in range(len(result)-1))  # Ordered

See strategies-guide.md for complete strategy reference.

Common Strategies

from hypothesis import strategies as st

st.integers(min_value=0, max_value=100)  # Bounded integers
st.text(min_size=1, max_size=50)         # Bounded text
st.lists(st.integers(), max_size=10)     # Bounded lists
st.from_regex(r"[a-z]+@[a-z]+\.[a-z]+")  # Pattern-based

# Composite for domain objects
@st.composite
def user_strategy(draw):
    return User(
        name=draw(st.text(min_size=1, max_size=50)),
        age=draw(st.integers(min_value=0, max_value=150)),
    )

Common Properties

# Roundtrip (encode/decode)
@given(st.dictionaries(st.text(), st.integers()))
def test_json_roundtrip(data):
    assert json.loads(json.dumps(data)) == data

# Idempotence
@given(st.text())
def test_normalize_idempotent(text):
    assert normalize(normalize(text)) == normalize(text)

# Oracle (compare to known implementation)
@given(st.lists(st.integers()))
def test_sort_matches_builtin(lst):
    assert our_sort(lst) == sorted(lst)

See stateful-testing.md for state machine testing.

Key Decisions

Decision	Recommendation
Strategy design	Composite strategies for domain objects
Example count	100 for CI, 10 for dev, 1000 for release
Database tests	Use explicit mode, limit examples
Deadline	Disable for slow tests, 200ms default
Stateful tests	RuleBasedStateMachine for state machines

Anti-Patterns (FORBIDDEN)

# NEVER ignore failing examples
@given(st.integers())
def test_bad(x):
    if x == 42:
        return  # WRONG - hiding failure!

# NEVER use filter with low hit rate
st.integers().filter(lambda x: x % 1000 == 0)  # WRONG - very slow

# NEVER test with unbounded inputs
@given(st.text())  # WRONG - includes 10MB strings
def test_username(name):
    User(name=name)

# NEVER mutate strategy results
@given(st.lists(st.integers()))
def test_mutating(lst):
    lst.append(42)  # WRONG - mutates generated data

Related Skills

pytest-advanced - Custom markers and parallel execution
unit-testing - Basic testing patterns
contract-testing - API contract testing with Pact

References

Strategies Guide - Complete strategy reference
Stateful Testing - State machine patterns
Hypothesis Conftest - Production setup

Capability Details

strategies

Keywords: strategy, hypothesis, generator, from_type, composite Solves: Generate test data, create strategies for custom types

properties

Keywords: property, invariant, roundtrip, idempotent, oracle Solves: What properties to test, roundtrips, invariants

stateful

Keywords: stateful, state machine, RuleBasedStateMachine, rule Solves: Test stateful systems, model state transitions

schemathesis

Keywords: schemathesis, openapi, api testing, fuzzing Solves: Fuzz test API endpoints, generate from OpenAPI spec

hypothesis-settings

Keywords: max_examples, deadline, profile, suppress_health_check Solves: Configure for CI vs dev, speed up slow tests

property-based-testing