mandela

Audit a validation for leakage: does outside ground-truth actually enter, or is everyone confirming a result no one independently produced?

Goal

The name is the Mandela Effect — a whole population confidently remembers something that never independently happened; a leaky validation is the same shape. Walk the 8 patterns below. mandela checks one thing: whether a validation is independent, or whether the designer, model, and scorer are only confirming each other.

Workflow

1. Identify the validation (eval / metric / experiment / holdout / "how we'll know"). Name its components — what plays model, scorer, designer, dataset.
2. Ask the core question: does external ground-truth enter independently?
3. Test the validation against all 8 patterns below (some apply only to certain components — a human subject, a scorer); report only the ones that fire, each by name.
4. Give the independent-ground-truth fix for each hit.

The 8 leakage patterns