Why is objective design the real bottleneck in AI discovery?

The usual framing of AI-for-science agents locates the hard part in the search: the hypothesis space is vast and experiments are expensive, so we build agents to navigate it faster. SAGA's authors argue this misdiagnoses the bottleneck. For grand challenges, the objective function itself is only an imperfect proxy for what we actually want — it is incomplete, problem-specific, and, worse, vulnerable to misalignment and reward hacking. An agent that flawlessly optimizes a bad objective produces a confident wrong answer, faster.

The consequence is that "creativity" in discovery is largely the creativity of defining the objective, not just exploring under it — and that is the part current systems leave to the scientist and freeze. Automating objective design therefore isn't a nice-to-have; it is the unmet need that gates open-ended discovery.

This generalizes the reward-hacking problem from training to discovery. Since Does self-consistency reliably reward correct answers during training?, any fixed proxy invites Goodharting once you optimize hard against it — SAGA's answer is not a better static proxy but a moving one that gets rewritten when failure modes appear. And since Can rubrics and dense rewards work together without hacking?, both point the same direction: robustness against reward hacking comes from restructuring the objective, not from trusting a single fixed reward to be complete.