langsmith-trace-analyzer

Fail

Audited by Gen Agent Trust Hub on Feb 14, 2026

Risk Level: CRITICALREMOTE_CODE_EXECUTIONEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONCREDENTIALS_UNSAFE
Full Analysis
  • [REMOTE_CODE_EXECUTION] (HIGH): An automated scan identified a function ensure_langsmith_fetch() in the script logic that downloads an external utility and executes it via subprocess. This download-then-execute pattern is a high-risk vector for RCE if the download source or transmission is compromised.
  • [COMMAND_EXECUTION] (HIGH): The skill utilizes multiple CLI-based workflows (uv run, ts-node, langsmith-fetch). If inputs such as project names, filter strings, or metadata from the downloaded traces are not strictly sanitized before being passed to these commands, it creates a significant command injection vulnerability.
  • [CREDENTIALS_UNSAFE] (MEDIUM): The skill requires the use of LANGSMITH_API_KEY. While the documentation recommends environment variables, the skill's primary purpose is downloading traces which are explicitly acknowledged in the SKILL.md to contain 'user prompts, outputs, metadata, and other sensitive runtime data,' creating a high-risk data exposure surface.
  • [EXTERNAL_DOWNLOADS] (MEDIUM): The skill depends on langsmith-fetch, a package that is not part of the standard 'trusted' repository list. The installation via uv pip install happens at runtime/setup without integrity verification.
  • [INDIRECT_PROMPT_INJECTION] (HIGH):
  • Ingestion points: LangSmith API traces (external content).
  • Boundary markers: None specified in the documentation or provided logic.
  • Capability inventory: Scripts execute subprocesses and generate analysis reports that influence agent debugging and evaluation decisions.
  • Sanitization: No evidence of sanitizing trace content (prompts/outputs) before analysis.
  • Risk: Malicious instructions embedded in a logged trace (e.g., inside a user message or tool output) could manipulate the analyze_traces.py report to mislead the agent or trigger unintended downstream actions.
Recommendations
  • CRITICAL: Downloads and executes remote code from untrusted source(s): unknown (check file) - DO NOT USE
  • AI detected serious security threats
Audit Metadata
Risk Level
CRITICAL
Analyzed
Feb 14, 2026, 04:00 PM