google-agents-cli-eval

Pass

Audited by Gen Agent Trust Hub on Nov 17, 2026

Risk Level: SAFEDYNAMIC_EXECUTIONINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
  • Dynamic Execution via Custom Metrics: The skill documents the ability to define custom evaluation metrics using Python code via the custom_function or custom_function_file properties. These evaluators run locally by default (execution: "local"), inheriting the permissions of the agents-cli tool. This is a core extensibility feature of the evaluation framework, and the documentation includes clear guidance for users to ensure the code they execute is trusted.
  • Indirect Prompt Injection Surface: The evaluation workflow involves processing external data from EvaluationDataset JSON files, which includes user prompts and agent conversation traces. This data is interpolated into metric templates (e.g., {prompt}, {response}) or passed to custom Python evaluators. This creates a surface where embedded instructions in the dataset could potentially influence the judge's assessment. The skill manages this by using structured schemas and providing specific metric definitions to handle the data.
  • Command Execution: The instructions provide numerous examples of executing the agents-cli tool for various tasks like running evaluations (eval run), generating traces (eval generate), and scoring results (eval grade). These operations involve spawning local processes, including the execution of the agent code itself during trace generation.
  • Trusted External Resource: The skill guides users to install the google-agents-cli package using the well-known uv tool. These resources originate from a trusted organization and follow standard development practices for Python-based agent projects, ensuring a secure installation path.
Audit Metadata
Risk Level
SAFE
Analyzed
Nov 17, 2026, 08:49 PM