langsmith-code-eval
Pass
Audited by Gen Agent Trust Hub on Sep 14, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONDYNAMIC_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
- Indirect Prompt Injection: The skill ingests data from external sources, including local agent code, LangSmith traces, and dataset records. This presents a surface for potential instructions embedded in that data to influence agent behavior.
- Ingestion points: The skill reads local agent source code (SKILL.md Step 3) and processes LangSmith trace and dataset structures (via scripts/inspect_trace.py and scripts/inspect_dataset.py).
- Capability inventory: The skill allows for file reading/writing and shell command execution to run its bundled inspection scripts.
- Boundary markers: No explicit delimiters or boundary markers are instructed for the data being analyzed to prevent the agent from following instructions contained within those files.
- Sanitization: There are no explicit instructions for sanitizing or escaping the content retrieved from the LangSmith API before it is processed by the agent.
- Command Execution: The skill instructs the agent to execute bundled Python scripts (
inspect_trace.pyandinspect_dataset.py) to analyze LangSmith infrastructure. These operations are performed using the user's preferred Python environment. - Dynamic Execution: A primary function of the skill is to have the agent generate and write new Python scripts for evaluator logic and experiment runners based on the analysis of the project's structure.
- External Dependencies: The skill requires the
langsmithPython package. This is a well-known library from the skill's author (langchain-ai) used for LLM application observability.
Audit Metadata