langsmith-evaluator
Fail
Audited by Runlayer on Mar 5, 2026
Risk Level: HIGH
Scan Summary
Flagged Files (3)
langsmith-evaluator/scripts/upload_evaluators.pyHIGH
92.3%
Malicious tool definition detected
langsmith-evaluator/scripts/upload_evaluators.tsMEDIUM
87.9%
Malicious tool definition detected
Tool: langsmith-evaluator/scripts/upload_evaluators.ts [2/2] Description: // CRITICAL: Block global evaluators at upload time if (!payload.target_dataset_ids && !payload.target_project_ids) { throw new Error( "Global evaluators are not supported.
langsmith-evaluator/SKILL.mdLOW
72.5%
Malicious tool definition detected
For dataset (offline) evaluators, prefer running locally with `evaluate(evaluators=[...])` first. This gives you full package access.
Audit Metadata