Promptfoo Patterns

Quick Guide: Use promptfoo for systematic LLM evaluation. Define prompts, providers, and test cases in promptfooconfig.yaml. Use assertion types (contains, is-json, llm-rubric, similar, cost, latency) to validate outputs. Use promptfoo eval to run (exits with code 100 on test failures), promptfoo view for results UI. Use model-graded assertions (llm-rubric, factuality) for subjective quality. Use promptfoo redteam run for security scanning. Use --share flag or promptfoo share to share results. All provider API keys come from environment variables -- never hardcode them.

<critical_requirements>

CRITICAL: Before Using This Skill

(You MUST define test cases with explicit assert arrays -- tests without assertions only capture output without validating it)

(You MUST use llm-rubric for subjective quality evaluation -- do NOT rely solely on deterministic assertions for natural language output)

(You MUST set threshold on similarity and model-graded assertions -- omitting thresholds uses defaults that may not match your quality bar)

(You MUST use environment variables for all API keys -- never hardcode keys in promptfooconfig.yaml or provider configs)

(You MUST verify promptfoo eval exit code in CI pipelines -- it returns exit code 100 on test failures, exit code 1 on other errors)