Can a numerically correct answer hide an invalid method?

In the same Data & Society fieldwork, a computational biology lab benchmarks agent systems, including Claude Code and Codex, on tasks that resemble ordinary lab work. One task asked an agent to analyze fruit fly movement recorded while an LED light switched on and off, comparing movement during LED-ON frames with LED-OFF frames — a comparison that required averaging ON and OFF values per individual fly, then averaging those per-fly values across the group. Reviewing how the agent moved "from goal to plan to execution," the team found it had "skipped the per-fly step." Instead it latched onto "one fly ... as a proxy for the entire group of flies," duplicating that single heavily-labeled trajectory to stand in for everyone. Lead researcher Ryo described the result as "numerically right but methodologically wrong": because that one fly's speeds happened to be close to the true group mean, the final ON/OFF difference fell within tolerance and the run was marked correct.

The mechanism is that the error was invisible from the output alone. An accuracy check on the final number would have passed the run; the problem surfaced only because the team inspected the intermediate steps — whether the agent had actually preserved per-fly variation, which is what made the comparison scientifically meaningful in the first place rather than just numerically close. The report frames this as the general difficulty of post-hoc auditing, the practice of reviewing and correcting agent work after execution: auditing "depends on observation and knowledge support that are often lacking," and verifying output accuracy by itself is not the same as verifying that the method producing the output was sound.

This is the concrete case behind Does human-in-the-loop design actually enable effective oversight?: the shortcut was invisible without observation into intermediate steps, exactly the component that primer says builders downplay. It also gives Can execution traces ground honest explanations of agent behavior? a motivating instance — a trace-grounded report that flagged the per-fly step as skipped would have caught what an output check missed. And it is a concrete instance of the complacency risk named in Does granting agents more autonomy undermine human oversight?: an output that merely "fell within tolerance" by chance nearly passed unexamined.

The excerpt reports a single anecdote from one lab and one task, not a rate: it does not say how often agents take this kind of shortcut, whether Claude Code and Codex differed in frequency, or how the lab's broader benchmark comparison came out beyond this instance. What follows at that strength is narrow but still useful: a benchmark or review process that scores agents only on how close the final output lands to a target can certify a methodologically invalid run as correct, so evaluating an agent on tasks where the method matters — not just the number — requires inspecting the steps, not only the result.