A/B Test Setup Skill

Overview

Production-ready A/B testing toolkit for calculating sample sizes, designing rigorous test plans, and analyzing results with statistical significance testing. Designed for growth teams, product managers, and marketers who need to make data-driven decisions from controlled experiments.

Clarify First

Before designing the test, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

• Hypothesis + primary metric: what change you expect and the single metric that judges it (drives test plan + analysis).
• Baseline conversion rate: the current rate the metric sits at today (drives sample size calculation).
• Minimum detectable effect (MDE): smallest lift worth detecting (drives required samples + duration).
• Daily traffic available: eligible visitors per day per variant (determines how long the test must run).

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.

Quick Start