langsmith-dataset
Fail
Audited by Gen Agent Trust Hub on Feb 16, 2026
Risk Level: HIGHPROMPT_INJECTIONCOMMAND_EXECUTION
Full Analysis
- [Indirect Prompt Injection] (HIGH): The script ingests untrusted data from the LangSmith API and local JSON/CSV files which is then processed by the agent. \n
- Ingestion points:
client.list_examplesinshowandexportcommands;open(path)inview_fileandstructurecommands inscripts/query_datasets.py. \n - Boundary markers: Absent. The content is passed directly to the console/agent context without delimiters or instructions to ignore embedded commands. \n
- Capability inventory: The skill has file-write capabilities via the
exportcommand. \n - Sanitization: Absent. Content is printed as raw or formatted JSON/text. \n- [Arbitrary File Write] (HIGH): The
exportcommand inscripts/query_datasets.py(line 166) accepts anoutput_fileargument without path validation. \n - Evidence:
with open(Path(output_file), \"w\") as f: json.dump(examples, f...)(line 167). \n - Risk: A malicious actor could influence the agent to overwrite sensitive files such as
~/.bashrcor~/.ssh/authorized_keysby providing a malicious path to the export command. \n- [Data Exposure] (LOW): The script accesses theLANGSMITH_API_KEYfrom environment variables. While standard for this API, it represents a sensitive credential that must be managed carefully.
Recommendations
- AI detected serious security threats
Audit Metadata