02-ref-hallucination-arena
Pass
Audited by Gen Agent Trust Hub on Oct 25, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- [EXTERNAL_DOWNLOADS]: The skill requires the installation of the
py-openjudgeandmatplotlibpackages via pip. These are standard libraries used for LLM evaluation and data visualization respectively. - [COMMAND_EXECUTION]: The skill provides CLI instructions to execute the evaluation pipeline using
python -m cookbooks.ref_hallucination_arena. This command runs the benchmarking logic contained within the installed or local package. - [INDIRECT_PROMPT_INJECTION]: The skill processes external data that could contain adversarial instructions.
- Ingestion points: Data enters the agent context through the
dataset.path(JSON/JSONL files) and raw responses from target LLM endpoints. - Boundary markers: The instructions do not define specific delimiters or "ignore previous instructions" guards for the interpolated data.
- Capability inventory: The skill performs network operations (API calls to Crossref, PubMed, arXiv, DBLP, and model providers) and file system operations (writing results and reports to the output directory).
- Sanitization: There is no explicit mention of sanitization or validation of the content within the queries or model responses before they are processed or written to reports.
- [SAFE]: The skill implements secure credential handling by instructing the user to provide API keys (OpenAI, Dashscope, etc.) via environment variables rather than hardcoding them. It also interacts exclusively with well-known, legitimate academic and technology services.
Audit Metadata