langsmith-dataset
Pass
Audited by Gen Agent Trust Hub on Sep 15, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
- Remote Script Execution: The skill installs the LangSmith CLI by downloading and executing a shell script directly from the vendor's official GitHub repository (
github.com/langchain-ai). While this is a standard installation method for the tool, it involves executing remote code in the local environment. - Command Execution: The skill makes extensive use of the
langsmithCLI to perform operations such as listing, creating, and deleting datasets. These commands involve network interactions with the LangSmith API and local system process execution. - Indirect Prompt Injection Surface: The skill processes trace data from external sources (JSONL files) to create datasets. This introduces a potential surface where untrusted data could influence agent behavior.
- Ingestion points: Trace data is ingested from the
./tracesdirectory using file system operations in both Python and TypeScript scripts. - Boundary markers: No explicit boundary markers or instructions to ignore embedded content are used during the parsing of these files.
- Capability inventory: The skill possesses the capability to perform network requests via the CLI and write processed data to the local file system (e.g.,
/tmp/dataset.json). - Sanitization: The skill relies on standard JSON parsing without additional validation or filtering of the content within the trace inputs or outputs.
- Credential Handling: The skill provides instructions for setting the
LANGSMITH_API_KEY. It correctly uses placeholders for sensitive values and recommends standard environment variable configuration for secret management.
Audit Metadata