Experiment Design Expert

Overview

Design fast, low-cost experiments to validate product hypotheses before committing to full development. This skill applies Alberto Savoia's pretotyping philosophy ("Make sure you are building The Right It before you build It right") alongside lean experimentation methods for both new and existing products.

Core Capabilities

• XYZ hypotheses — frame every test as "At least X% of Y will do Z" with a pre-set pass/fail threshold.
• SITG + YODA discipline — prefer skin-in-the-game signals (money, time, reputation) and Your Own Data over surveys and benchmarks.
• Method selection — landing page, explainer video, pre-order, concierge MVP (new products); fake door, feature stub, A/B test, Wizard of Oz, in-app survey (existing).
• 5-step process — hypothesis, method, metric/threshold, timeboxed run, evaluate (pass/fail/inconclusive).
• Automated design — experiment_designer.py suggests 2-3 experiments per hypothesis with metric, threshold, effort, and duration.

When to Use

• You have a product idea or feature hypothesis and need to validate it cheaply.
• You want to test willingness to pay or genuine user interest, not just stated preference.
• You need to choose the right experiment method for your context (new vs. existing product).