langsmith-dataset

Fail

Audited by Gen Agent Trust Hub on Feb 16, 2026

Risk Level: HIGHPROMPT_INJECTIONCOMMAND_EXECUTION
Full Analysis
  • [Indirect Prompt Injection] (HIGH): The script ingests untrusted data from the LangSmith API and local JSON/CSV files which is then processed by the agent. \n
  • Ingestion points: client.list_examples in show and export commands; open(path) in view_file and structure commands in scripts/query_datasets.py. \n
  • Boundary markers: Absent. The content is passed directly to the console/agent context without delimiters or instructions to ignore embedded commands. \n
  • Capability inventory: The skill has file-write capabilities via the export command. \n
  • Sanitization: Absent. Content is printed as raw or formatted JSON/text. \n- [Arbitrary File Write] (HIGH): The export command in scripts/query_datasets.py (line 166) accepts an output_file argument without path validation. \n
  • Evidence: with open(Path(output_file), \"w\") as f: json.dump(examples, f...) (line 167). \n
  • Risk: A malicious actor could influence the agent to overwrite sensitive files such as ~/.bashrc or ~/.ssh/authorized_keys by providing a malicious path to the export command. \n- [Data Exposure] (LOW): The script accesses the LANGSMITH_API_KEY from environment variables. While standard for this API, it represents a sensitive credential that must be managed carefully.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Feb 16, 2026, 09:13 AM