The Agent Skills Directory

Dynamic Execution via Graders: The skill is designed to execute code defined in configuration files to perform evaluations.
Evidence: The references/EVAL-SPEC.md file describes graders that use type: code for Python assertions and type: script for external Python scripts (e.g., script: graders/my.py).
Context: This allows the skill to run arbitrary logic to validate behavior. While this is standard for a test runner, it means that the eval.yaml and associated script files have the capability to execute commands on the host system. Users should ensure the evaluation suites they run are from trusted sources.
Indirect Prompt Injection Surface: The skill uses LLM-based graders to evaluate the output of other AI agents, which creates a surface for indirect instruction injection.
Ingestion Points: The skill ingests data from eval.yaml, task definitions in tasks/*.yaml, and the runtime output generated by the skill being evaluated.
Boundary Markers: The provided specification does not detail specific delimiters or sanitization logic used to separate the rubric instructions from the untrusted skill output being graded.
Capability Inventory: The skill can execute Python scripts, perform LLM calls (via LLM graders), and output report files.
Sanitization: There is no evidence of sanitization or escaping of the skill's output before it is processed by the llm or llm_comparison graders. If a skill under test produces malicious content designed to influence an LLM, it could potentially manipulate the evaluation results or the grader's behavior.

waza-runner