Fine-Tune Models on Your Data

Guide the user through deciding whether to fine-tune, preparing data, running fine-tuning with DSPy, distilling to cheaper models, and deploying. Fine-tuning is powerful but expensive — always confirm prerequisites first.

Should you fine-tune?

Before writing any code, walk through these questions with the user:

1. Have you optimized prompts first? If not, use /ai-improving-accuracy — prompt optimization is 10x cheaper and often sufficient.
2. Do you have 500+ labeled examples? Fine-tuning with less data usually overfits. Collect more data first.
3. Is your baseline accuracy above 50%? If your prompt-optimized program is below 50%, your task definition or data has problems. Fix those first.
4. What's the goal — quality or cost?
   • Quality: You've maxed out prompt optimization and need more accuracy
   • Cost: You want a small cheap model to match an expensive one

When to fine-tune

• You've already optimized prompts with MIPROv2 and hit a ceiling
• You have 500+ labeled examples (1000+ is better)