distributed-training
Distributed Training Best Practice
- Use DistributedDataParallel (DDP) over DataParallel for multi-GPU
- Initialize process group: dist.init_process_group(backend='nccl')
- Use DistributedSampler for data sharding
- Synchronize batch norm: nn.SyncBatchNorm.convert_sync_batchnorm()
- Only save checkpoint on rank 0
- Scale learning rate linearly with world size
- Use gradient accumulation for effectively larger batch sizes
More from aiming-lab/autoresearchclaw
scientific-writing
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines. Use when drafting or revising research papers.
10hypothesis-formulation
Structured scientific hypothesis generation from observations. Use when formulating testable hypotheses, competing explanations, or experimental predictions.
9scientific-visualization
Publication-ready scientific figure design with matplotlib and seaborn. Use when creating journal submission figures with proper formatting, accessibility, and statistical annotations.
9literature-search
Systematic literature review methodology including search strategy, screening, and synthesis. Use when conducting literature reviews or writing background sections.
9statistical-reporting
Statistical test selection, assumption checking, and APA-formatted reporting. Use when analyzing experimental results or writing results sections.
9a-evolve
>
8