Python Testing

Overview

Test observable behavior and contracts, not internal implementation. Keep unit tests fast, deterministic, and patched at module boundaries.

These are preferred defaults for common cases. When a default conflicts with project constraints, suggest a better-fit alternative, call out tradeoffs, and note compensating controls.

Invocation Notice

• Inform the user when this skill is being invoked by name: python-testing.

When to Use

• Writing or reviewing unit, integration, or reliability-sensitive tests.
• Tests are flaky, slow, or coupled to implementation details.
• Adding regression tests after a bugfix.
• Testing async lifecycles, cancellation, or cleanup paths.
• Unsure what test coverage a change needs.
• Testing across multiple Python versions (nox, CI matrix).
• Validating thread safety for free-threaded Python (GIL-disabled builds).

When NOT to use:

• Pure data-shape or schema validation (see python-types-contracts).
• Production observability or monitoring concerns (see python-runtime-operations).
• Concurrency design decisions outside of test harnesses (see python-concurrency-performance).

Quick Reference

• Test observable behavior, not internals.
• Keep unit tests fast and deterministic.
• Patch at module boundaries and import locations used by the unit under test.
• Add regression tests for bugfixes.
• Include timeout/retry/cancellation/cleanup coverage where relevant.
• For multi-Python: use nox with uv backend; parametrize for dependency matrices.
• For free-threaded Python: use pytest-run-parallel, set PYTHON_GIL=0, always set CI timeouts.

Change-Specific Diagnostics

• Dependency updates: run uv run pytest scripts/test_pypi_security_audit.py -v
• Async-heavy lifecycle changes: run pyleak diagnostics.
• Multi-Python support changes: run full matrix via nox.
• Free-threaded compatibility: run PYTHON_GIL=0 uv run --python 3.Xt pytest --parallel-threads=auto --timeout=300 on a free-threaded build (3.13t+).

Common Mistakes

• Single write-site coverage — testing the canonical code path but not alternative paths (dedup shortcut, cache fast-path, retry branch) that write the same contract-asserted value. Each write-site needs its own test. See references/testing-strategy.md § Multi-Path and Derived-Field Patterns.
• Missing post-composition invariant — when field_a == f(field_b) is a required relationship and each field is written by a different producer (resolver, fetcher, merge step), stage-local tests don't prove the invariant holds after composition. Write a test that asserts the relationship on the assembled output. See references/testing-strategy.md § Multi-Path and Derived-Field Patterns.
• Mocking too deep — patching internals instead of module-boundary seams makes tests brittle and coupled to implementation.
• Testing the mock — verifying mock call counts without asserting on observable output proves nothing about behavior.
• Missing regression test — fixing a bug without a test that reproduces it first; the bug will recur.
• Non-deterministic time/order — relying on wall-clock time or dict/set ordering instead of injecting clocks and sorting explicitly.
• Skipping cleanup assertions — verifying the happy path but never asserting that resources are released on failure or cancellation.
• No free-threaded CI entry — shipping multi-threaded code without a free-threaded (t-suffixed) matrix entry; the GIL hides race conditions that will surface when free-threaded Python becomes the default.
• Ignoring GIL re-enablement — importing a C extension without Py_mod_gil silently re-enables the GIL; check sys._is_gil_enabled() after imports.
• YAML version float — writing 3.10 unquoted in CI matrix YAML; it parses as 3.1 and installs the wrong Python.

References

• references/testing-strategy.md
• references/pytest-practices.md
• references/async-and-concurrency-testing.md
• references/reliability-lifecycle-testing.md
• references/multi-python-testing.md
• references/free-threaded-testing.md