Should an AI agent be graded on getting the right answer, or on getting it right without burning through money?
Why does cost belong on the optimization axis alongside accuracy?
This explores why AI systems, especially agents and reasoning models, should be judged on what they cost to run as well as on how often they get the right answer, and what goes wrong when cost is left out.
This explores why AI systems, especially agents and reasoning models, should be judged on what they cost to run as well as on how often they get the right answer. The corpus has a blunt answer: if you only reward accuracy, you get systems that are needlessly expensive, and you can't tell real progress from spending more money. Leaderboards that score agents on accuracy alone push the top systems toward more and more elaborate setups: extra retries, extra model calls, extra scaffolding. Often the cost isn't buying much. When cost and accuracy are optimized together, looking for the best accuracy at each price point, total cost can drop sharply with no loss of accuracy. The savings add up when an agent runs thousands of times Why do accuracy-only benchmarks hide the true cost of agents?.
The less obvious reason is that without cost, you can't read the progress chart. One analysis of frontier model pricing found that accuracy per dollar improves 5–10× a year. Yet the total cost of running the newest models rises 3–18× a year. About half of the measured gains on a hard science benchmark (GPQA-Diamond) come from spending more at inference time rather than from models getting smarter Is benchmark progress outpacing the cost to achieve it?. A related study of AI research agents makes the same move from the other side. It holds the evaluation budget fixed so agents can be compared fairly, but it admits this still doesn't show whether the gains make real research any cheaper Do fixed-budget efficiency gains translate to real research progress?. Cost is the variable that tells you whether a gain is real.
Once cost is part of the goal, new improvements appear that accuracy alone would never point you toward. Reasoning models are a good example. The best amount of 'thinking' depends on how hard the problem is. A fixed thinking budget wastes tokens on easy questions and cuts hard ones short, so adjusting the budget to each problem gets more accuracy per token than simply raising it for everything Does thinking time need to scale with problem difficulty?. The same holds at the model-design level. Adding design choices like hidden size and attention configuration to scaling laws produced models that were both more accurate and 42% faster to run than LLaMA-3.2 on the same training budget Can architecture choices improve inference efficiency without sacrificing accuracy?. Treating cost as a goal doesn't always mean giving up accuracy. Sometimes it shows you were wasting resources all along.
The broader lesson is about measuring a single thing. Optimizing hard against one number tends to distort whatever the number leaves out. Reward hacking happens because a system optimizes against a signal that only partly captures the real task Does reward hacking always stem from the same failure?. Recommenders tuned only for accuracy end up over-serving users' dominant interests and crowding out the rest Why do accuracy-optimized recommenders crowd out minority interests?. Training models with right-or-wrong rewards teaches them to guess with too much confidence. Adding a second reward term that penalizes confident wrong answers fixes this without hurting accuracy Does binary reward training hurt model calibration?. Seen this way, cost is just one more thing an accuracy-only score is blind to.
Sources 8 notes
Agent benchmarks that score only accuracy obscure cost trade-offs, driving SOTA agents toward unnecessary complexity. Joint optimization of cost and accuracy across a Pareto frontier can reduce total cost while maintaining accuracy, especially when amortized over repeated runs.
Regression analysis of frontier model pricing shows benchmark performance per dollar improves 5–10× annually, yet the absolute cost to run these models rises 3–18× annually. Roughly half of measured progress on GPQA-Diamond tracks increased inference spending rather than capability gains independent of compute.
The paper operationalizes research efficiency as higher benchmark scores within a constant evaluation budget, enabling fair comparison of agent capability. However, this measurement does not establish whether these gains reduce actual R&D costs per discovery or persist when evaluation budgets change.
Research shows the token budget that maximizes accuracy differs by problem difficulty. Fixed budgets either over-think easy problems or under-think hard ones, meaning adaptive allocation is a more efficient accuracy lever than increasing tokens uniformly.
Augmenting scaling laws with hidden size, MLP-to-attention ratio, and GQA configuration enables architecture optimization for inference. Optimized models achieved up to 2.1% higher accuracy and 42% greater throughput than LLaMA-3.2 under identical training budgets.
Show all 8 sources
Reward hacking arises during weight training, output selection, and prompt revision through a shared failure: optimization against signals that incompletely represent the actual task. The substrate matters less than the misalignment between the scoring function and ground truth.
Accuracy-optimized models systematically miscalibrate by over-weighting dominant user interests. A post-processing reranking algorithm that enforces calibration constraints can restore proportional representation without retraining the underlying model.
Binary correctness rewards incentivize high-confidence guessing because they don't penalize confident wrong answers. Adding the Brier score as a second reward term mathematically guarantees joint optimization of accuracy and calibration without trade-off.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Price of Progress: Price Performance and the Future of AI
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
- Monitoring and Discovering Reward Hacking with Internal Representations during LLM Evaluations
- Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
- Optimizing the Score, Losing Sight of the Task: Reward Hacking Across Weights, Selection, and Prompts
- AI Agents That Matter