langsmith-trace-analyzer
Fail
Audited by Gen Agent Trust Hub on Feb 14, 2026
Risk Level: CRITICALREMOTE_CODE_EXECUTIONEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONCREDENTIALS_UNSAFE
Full Analysis
- [REMOTE_CODE_EXECUTION] (HIGH): An automated scan identified a function
ensure_langsmith_fetch()in the script logic that downloads an external utility and executes it via subprocess. This download-then-execute pattern is a high-risk vector for RCE if the download source or transmission is compromised. - [COMMAND_EXECUTION] (HIGH): The skill utilizes multiple CLI-based workflows (
uv run,ts-node,langsmith-fetch). If inputs such as project names, filter strings, or metadata from the downloaded traces are not strictly sanitized before being passed to these commands, it creates a significant command injection vulnerability. - [CREDENTIALS_UNSAFE] (MEDIUM): The skill requires the use of
LANGSMITH_API_KEY. While the documentation recommends environment variables, the skill's primary purpose is downloading traces which are explicitly acknowledged in theSKILL.mdto contain 'user prompts, outputs, metadata, and other sensitive runtime data,' creating a high-risk data exposure surface. - [EXTERNAL_DOWNLOADS] (MEDIUM): The skill depends on
langsmith-fetch, a package that is not part of the standard 'trusted' repository list. The installation viauv pip installhappens at runtime/setup without integrity verification. - [INDIRECT_PROMPT_INJECTION] (HIGH):
- Ingestion points: LangSmith API traces (external content).
- Boundary markers: None specified in the documentation or provided logic.
- Capability inventory: Scripts execute subprocesses and generate analysis reports that influence agent debugging and evaluation decisions.
- Sanitization: No evidence of sanitizing trace content (prompts/outputs) before analysis.
- Risk: Malicious instructions embedded in a logged trace (e.g., inside a user message or tool output) could manipulate the
analyze_traces.pyreport to mislead the agent or trigger unintended downstream actions.
Recommendations
- CRITICAL: Downloads and executes remote code from untrusted source(s): unknown (check file) - DO NOT USE
- AI detected serious security threats
Audit Metadata