langsmith-code-eval

Pass

Audited by Gen Agent Trust Hub on Sep 14, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONDYNAMIC_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
  • Indirect Prompt Injection: The skill ingests data from external sources, including local agent code, LangSmith traces, and dataset records. This presents a surface for potential instructions embedded in that data to influence agent behavior.
  • Ingestion points: The skill reads local agent source code (SKILL.md Step 3) and processes LangSmith trace and dataset structures (via scripts/inspect_trace.py and scripts/inspect_dataset.py).
  • Capability inventory: The skill allows for file reading/writing and shell command execution to run its bundled inspection scripts.
  • Boundary markers: No explicit delimiters or boundary markers are instructed for the data being analyzed to prevent the agent from following instructions contained within those files.
  • Sanitization: There are no explicit instructions for sanitizing or escaping the content retrieved from the LangSmith API before it is processed by the agent.
  • Command Execution: The skill instructs the agent to execute bundled Python scripts (inspect_trace.py and inspect_dataset.py) to analyze LangSmith infrastructure. These operations are performed using the user's preferred Python environment.
  • Dynamic Execution: A primary function of the skill is to have the agent generate and write new Python scripts for evaluator logic and experiment runners based on the analysis of the project's structure.
  • External Dependencies: The skill requires the langsmith Python package. This is a well-known library from the skill's author (langchain-ai) used for LLM application observability.
Audit Metadata
Risk Level
SAFE
Analyzed
Sep 14, 2026, 08:16 PM
Security Audit — agent-trust-hub — langsmith-code-eval