langsmith

Warn

Audited by Gen Agent Trust Hub on Mar 13, 2026

Risk Level: MEDIUMEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTIONCOMMAND_EXECUTIONPROMPT_INJECTION
Full Analysis
  • [REMOTE_CODE_EXECUTION]: The skill documentation and setup scripts instruct the user to download and execute a shell script from a remote URL.\n
  • Evidence: curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh found in references/cli.md and scripts/setup.sh.\n
  • Note: The script is sourced from the official repository of LangChain, which is a trusted organization.\n- [COMMAND_EXECUTION]: The scripts/setup.sh script executes system commands to manage the environment and install dependencies, including the modification of shell profiles for persistence.\n
  • Evidence: Uses pip install, npm install, and yarn add to install langsmith, openai, and openevals packages.\n
  • Evidence: Appends LANGSMITH_API_KEY and LANGSMITH_TRACING to ~/.bashrc or ~/.zshrc (lines 39-44 in scripts/setup.sh) to persist configuration across sessions.\n- [EXTERNAL_DOWNLOADS]: The skill installs several external software packages during the setup process.\n
  • Evidence: Installs langsmith, openai, and openevals via standard package managers.\n
  • Note: These are official packages from well-known registries and trusted vendors.\n- [PROMPT_INJECTION]: The skill provides functionality to pull prompts from the LangSmith Prompt Hub, which introduces a surface for indirect prompt injection if the pulled prompts are untrusted.\n
  • Ingestion points: The client.pull_prompt() method is used to fetch prompt templates from the remote hub (found in SKILL.md and references/python-sdk.md).\n
  • Boundary markers: No delimiters or warnings to ignore embedded instructions were identified in the implementation snippets.\n
  • Capability inventory: The skill has broad capabilities including file system access (Read, Write, Edit), shell command execution (Bash), and network requests (WebFetch), as specified in SKILL.md and demonstrated in scripts/setup.sh.\n
  • Sanitization: There is no evidence of sanitization, escaping, or validation of the content pulled from the Prompt Hub before it is used.
Recommendations
  • INFO: Downloads code from trusted source(s): https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh
Audit Metadata
Risk Level
MEDIUM
Analyzed
Mar 13, 2026, 06:56 AM