experimental-design
Experimental Design Best Practice
- ALWAYS include meaningful baselines (not just random):
- At least one classical method baseline
- At least one recent SOTA method baseline
- A simple-but-strong baseline (e.g., linear probe, k-NN)
- Use MULTIPLE random seeds (minimum 3, ideally 5)
- Report mean +/- std across seeds
- Design ablations that isolate EACH key component:
- Remove one component at a time
- Each ablation must be meaningfully different from baseline
- Control variables: change only ONE thing per comparison
- Use standard splits (train/val/test) — never test on training data
- Report wall-clock time and memory usage alongside accuracy
More from aiming-lab/autoresearchclaw
scientific-writing
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines. Use when drafting or revising research papers.
10hypothesis-formulation
Structured scientific hypothesis generation from observations. Use when formulating testable hypotheses, competing explanations, or experimental predictions.
9scientific-visualization
Publication-ready scientific figure design with matplotlib and seaborn. Use when creating journal submission figures with proper formatting, accessibility, and statistical annotations.
9literature-search
Systematic literature review methodology including search strategy, screening, and synthesis. Use when conducting literature reviews or writing background sections.
9statistical-reporting
Statistical test selection, assumption checking, and APA-formatted reporting. Use when analyzing experimental results or writing results sections.
9a-evolve
>
8