Domain Context

This skill implements a proven product management framework. The approach combines best practices from industry leaders and is designed for practical application in day-to-day PM work.

Input Requirements

Context about your product, feature, or problem
Relevant data, research, or constraints (recommended but optional)
Clear articulation of what you're trying to achieve

A/B Test Designer

When to Use

Testing a new feature or design variation
Validating a hypothesis before full rollout
Optimizing conversion rates or key metrics
Choosing between multiple design approaches
Need to make a data-driven decision on a change

What This Skill Does

Helps you design rigorous A/B tests with clear hypotheses, success metrics, sample size calculations, and analysis plans.

Instructions

Help me design an A/B test for [feature/change]. Include:

Hypothesis
- Current situation and metrics
- Proposed change
- Expected impact and why
Test Design
- Primary success metric
- Secondary metrics
- Sample size needed
- Test duration
- User segments to include/exclude
Variants
- Control (A): current experience
- Variant (B): new experience
- Any additional variants (C, D, etc.)
Risks and Controls
- Potential negative impacts
- Guardrail metrics
- When to stop the test early
Analysis Plan
- Statistical significance threshold
- How to handle edge cases
- Decision criteria

Feature context: [Add context about the change you want to test]

Best Practices

Start with a clear, falsifiable hypothesis
Choose one primary metric to avoid multiple comparison issues
Calculate sample size upfront based on expected effect size
Run tests for full weekly cycles to account for day-of-week effects
Set a minimum test duration (usually 1-2 weeks)
Define success criteria before running the test
Monitor guardrail metrics (revenue, errors, performance)

Example

Input: Testing new onboarding flow vs current 3-step process Output: Hypothesis (new 1-step flow will increase co...

ab-test-designer