wikieval

Installation
SKILL.md

Skill: wikieval

Turn wiki goldens into a CI gate: cheap deterministic asserts decide most goldens, a committed baseline catches regressions, and the one expensive thing (an LLM judge) is quarantined behind a single adapter file.

WHAT

Purpose và context

  • Purpose: make wiki golden pages a regression suite checked in pure code (no model), with a committed baseline and a CI exit code that fires only when a metric drops below it.
  • Trigger (when to use):
    • You want wiki pages to act as a regression suite an agent (or PR) cannot silently break.
    • You have golden input→expected pairs and want them checked in pure code, no model needed.
    • You need a CI step that exits non-zero only when a metric drops below a committed baseline.
    • User says "wikieval", "eval suite for the wiki", "regression gate for goldens", "assertion cascade", "block CI on eval drop", or /wikieval (model-invocation disabled — user invokes).
  • Non-goals: never calls an LLM (tier 3 is reported needs-judge, not judged); does not generate candidate outputs (they come from --outputs); does not author goldens' content for you.

Mental model

golden (input · expected · rubric? · asserts?) + candidate output → cascade: tier 1 asserts → [tier 2 similarity] → [tier 3 needs-judge] → per-golden pass+score → baseline (write) | regression diff (check). Cheap tiers short-circuit; the expensive one sits behind one adapter file.

Installs
198
Repository
rheinmir/setup
GitHub Stars
7
First Seen
Jun 28, 2026
wikieval — rheinmir/setup