Agent Evals

Create repeatable checks so agent behavior improves safely over time.

Evaluation Layers

• Unit evals: prompt-level correctness
• Tool evals: API/tool call decision quality
• End-to-end evals: realistic multi-step tasks
• Safety evals: prompt injection and data leak resistance

CI/CD Integration

# Example eval pipeline steps
make evals-smoke
make evals-regression
make evals-safety

Best Practices

• Version datasets with expected outputs.
• Track pass rates and score drift over time.
• Block deploys on critical safety regressions.

Related Skills

• github-actions - Eval automation in CI
• ai-agent-security - Security-focused eval cases