Training Optimization

Use this contract for the concrete task. Apply a short relevant check for a small change; expand investigation when the failure, risk or requested artifact warrants it. Resolve facts from the repository and runtime before imposing a process. Delegation is optional and should answer a bounded unresolved question.

Training contract

Inspect the data/split, task metric, objective, loss/gradient traces, update order and environment before changing hyperparameters. Use evidence available in the code/run; ask only for missing facts that affect diagnosis.

• Check that labels, sampling, objective and evaluation metric represent the intended task. Leakage or a broken update is not repaired by a different optimizer.
• Distinguish flat/unstable training, poor generalization and throughput limits. Reproduce on a small batch/task and verify gradient flow before broad sweeps.
• Preserve accumulation normalization, clipping/scaling order, scheduler/optimizer cadence and resume state. Effective batch changes may alter optimization statistics; accumulation is not always equivalent with batch-dependent operations.
• Choose precision and memory/sharding strategy from hardware support, stability and measured memory/throughput. Keep strategy separate from version-specific framework APIs.
• Compare optimizer/schedule/regularization candidates under common splits, budgets and metrics. Familiar defaults are baselines, not universally correct choices.
• Apply augmentation according to task invariances and the evaluation protocol. Evaluation-time transformations are valid when explicitly part of the protocol; never leak fitted augmentation choices through test outcomes.
• Allocate search budget and define selection/early-stopping rules. Log configurations, data/code identity, randomization and failures so the winner can be re-evaluated.

Evidence and output

Produce a diagnosis and minimal intervention or a budgeted experiment plan. Include baseline, hypothesis, changed factors, acceptance metric, resource budget, run identity and observed result/limitations. Report a proposed optimizer change as a hypothesis until compared.