Why can brilliant researchers look at the same AI output and disagree on whether it's truly original?

This explores why expert judgment splits on whether a frontier reasoning model like o3 comes up with ideas that are actually new. To be clear up front, the collection has no notes about o3 itself or about particular scientists' public statements on it. What it does have is a clear account of why reasonable experts can look at the same AI output and disagree. The short version: the disagreement may not come from anyone being wrong. It comes from the fact that 'novel' isn't something you can measure the way you measure accuracy. Parker argues that claims of AI discovery will stay contested for years, because whether an idea counts as novel and useful is decided by a research community, not by an objective test. Mathematics may be the one exception, because a computer can check a proof Will we ever agree on whether AI makes real discoveries?.

The experimental evidence shows that both sides can be right at once. In a large study of more than 100 NLP researchers, experts rated LLM-generated research ideas as significantly more novel than ideas written by human experts Do language models generate more novel research ideas than experts?. The explanation is surprising. Experts are held back by what they know won't work, while models freely combine concepts across fields without that restraint Can LLMs generate more novel ideas than human experts?. But when 43 researchers spent over 100 hours each actually carrying out randomly assigned ideas, the AI ideas lost far more quality than the human ones. Evaluation plans turned out to be impractical, and technical groundwork was missing Do LLM research ideas actually hold up when experts try to execute them?. So a scientist who judges ideas on paper may see real novelty, while one who judges by what survives in the lab may see mostly noise.

There is a second split: the models generate well but judge poorly. They produce surprising combinations but avoid committing to whether an idea is feasible or valid. Automated evaluation overestimates idea quality by about 60% Why do LLMs generate more novel research ideas than experts?. That means much of the 'is it real?' work falls on human experts, and each brings their own standards. Even favorable evidence often comes from the systems' own builders. Co-Scientist's team reports that hypotheses get better as the system is given more compute, but replication so far comes mostly from their own validation Does more thinking time improve AI-generated research hypotheses?. Meanwhile, MIT's withdrawal request for an influential AI-and-science preprint shows how a claim can shape the debate long before anyone has checked it Can unreviewed preprints shape scientific debate before peer review?.

Two deeper points help explain why the debate is so hard to settle. First, matching outputs don't prove matching understanding. Networks can produce identical results while their internal representations are fractured in ways that block creative recombination in new settings Can identical outputs hide broken internal representations?. A striking idea might come from real generalization, or from lucky surface-level recombination, and the text alone can't show which. Second, in science, much of an idea's weight comes from who proposes it: their track record and their stake in the field. Models work without that social context Can language models distinguish expert arguments from common assumptions?. Part of what top scientists disagree about may be whether an idea can count as a discovery when no one stands behind it.