BetterTogether: Joint Prompt + Weight Optimization

Guide the user through using dspy.BetterTogether to get the best possible quality by combining prompt optimization and model fine-tuning in alternating rounds. Each round builds on the improvements from the previous one, creating compounding gains that beat either approach alone.

What it is

BetterTogether is a DSPy optimizer that alternates between prompt optimization (instructions, few-shot examples) and weight optimization (fine-tuning). Instead of running these independently, it chains them so each phase builds on the previous one's improvements:

1. Prompt optimization discovers effective task decompositions and reasoning strategies
2. Weight optimization specializes the model to execute those discovered patterns efficiently
3. Repeated rounds compound the gains -- each phase benefits from the prior improvements

Research shows this consistently outperforms either approach alone, with 5-78% gains over individual techniques (arXiv 2407.10930v2). A Databricks case study on IE Bench showed GEPA alone +2.1 points, fine-tuning alone +1.9 points, but combined they achieved +4.8 points over baseline.

When to use

• You have 500+ labeled examples and a reliable metric
• You've already tried prompt optimization (MIPROv2) and fine-tuning (BootstrapFinetune) separately and want more
• You want the absolute best quality and have the compute budget for multiple optimization rounds