test-challenger

Installation
SKILL.md

Test Challenger

Core principle: A test is only as valuable as its independence from the code it tests. When the test's expected behavior was derived by observing the code rather than understanding the specification, the test cannot catch bugs — it can only catch regressions. This skill systematically identifies tests that are likely validating what the code does rather than what the code should do.


The Oracle Problem

Every meaningful test has two components:

  1. Stimulus — calling the code under test with specific inputs
  2. Oracle — the expected result, derived from an independent source of truth

When a human writes a test, the oracle typically comes from a specification, domain knowledge, or a requirements document. When an AI generates a test by observing code, the oracle comes from the code itself. This creates a circular dependency: the test validates the code, but the code defined the test. The test becomes a tautology — it can never fail for the right reasons.

This skill applies a structured set of challenge patterns to each test to estimate the probability that its oracle was derived from the code rather than from genuine domain understanding.


How to Execute This Skill

Related skills

More from andurilcode/craftwork

Installs
2
GitHub Stars
6
First Seen
Apr 2, 2026