langchain-eval-engineering

Installation
SKILL.md

Eval Engineering

Flow

  • Inspect all inputs first: the repository, Harness, optional traces, existing Tasks and runs, existing World knowledge, and the human goal. Identify the source files and skill references that apply before proposing work.
  • Create or update the small project World Knowledge Skill from reusable facts in those inputs. Use it to propose one grounded Task.
  • Draft the Task Spec and World Skill together. Show both to the user, keep exact Task truth only in Task.md, and refine both until the user approves them.
  • Implement the approved Task, validate its Environment and Verifier, run the real Harness, and inspect the full evidence. Fix the Task or Verifier when the eval does not measure the intended behavior, and update the agent when the eval exposes a real product failure.
  • Reconcile the World Skill with what the run proved, then repeat this flow for the next Task.

Terms

Installs
298
GitHub Stars
1.4K
First Seen
7 days ago