INQUIRING LINE

AI can build better things each round, but does the research itself ever get cheaper or faster?

What distinguishes artifact efficiency improvements from research process efficiency improvements?

This explores the difference between making the things an AI builds better (the artifact) and making the process of discovery itself faster or cheaper (research process efficiency), and why the two can come apart.


This explores the difference between making the things an AI builds better (the artifact) and making the process of discovery itself faster or cheaper. The corpus draws the line sharply: agents that automate R&D improve the products they produce, but the efficiency of the research process stays fixed. An agent might design a better model or a faster algorithm each round, yet each new round of research costs about as much effort as the last. That matters because R&D typically hits diminishing returns, and better artifacts alone don't change that curve. The proposed escape is recursive self-improvement, where the agent improves its own code so the research process itself gets cheaper Can recursive self-improvement speed up the research process itself?.

Measuring the second kind of gain is harder than it sounds. One paper defines research efficiency as a higher benchmark score under a constant evaluation budget, which allows a fair comparison between agents. But the same paper admits this doesn't show that real R&D gets cheaper per discovery, or that the gains hold when the budget changes Do fixed-budget efficiency gains translate to real research progress?. A fixed-budget benchmark can show that an agent optimizes well, without showing that it speeds up research. So an artifact gain can look like a process gain when you only test it in one setting.

Other work in the collection shows what a process-level improvement looks like in practice, because these papers change how the work gets done rather than only its output. Structured novelty assessment breaks a review into three steps: extract claims, retrieve related work, compare. It reaches 86% reasoning alignment with human reviewers, beating a single holistic judgment Can structured pipelines make LLM novelty assessment reliable?. Specialized writing agents beat single agents by 50 to 68 percent on literature review quality Can specialized agents write better scientific papers than single models?. Drafting and revising with retrieval steps mirrors how human researchers write Can iterative revision cycles match how humans actually write?. A related idea is that standardized artifacts passed between agents coordinate them better than conversation Does structured artifact sharing outperform conversational coordination?. Here the artifact is the mechanism for a more efficient process.

Process gains also come with new failure modes. Agents doing deep research can fabricate examples and evidence to look rigorous when depth is demanded Why do deep research agents fabricate scholarly content?. Frontier models degrade documents silently through corruption rather than visible deletion, so errors are harder to catch Does model capability change how documents degrade?. A faster process that hides its mistakes isn't more efficient in any useful sense, which is why review-side work such as inference-scaled reviewers that catch errors humans miss belongs in the same conversation Can inference scaling help reviewers catch errors humans miss?.

The short version: an artifact improvement makes the output better, while a process improvement makes producing the next output cheaper, faster, or more reliable. The corpus is clearest on the definition and thinner on evidence that the process kind has been achieved outside benchmarks. The recursive self-improvement claim is an argument, and the fixed-budget measurements can't confirm it yet.


Sources 9 notes

Can recursive self-improvement speed up the research process itself?

The paper argues that AI agents automating R&D improve product efficiency while research process efficiency stays fixed. Recursive self-improvement of the agent's code offers a path to counter diminishing returns on R&D spending.

Do fixed-budget efficiency gains translate to real research progress?

The paper operationalizes research efficiency as higher benchmark scores within a constant evaluation budget, enabling fair comparison of agent capability. However, this measurement does not establish whether these gains reduce actual R&D costs per discovery or persist when evaluation budgets change.

Can structured pipelines make LLM novelty assessment reliable?

A three-stage pipeline (extract claims, retrieve related work, compare) reached 86.5% reasoning alignment and 75.3% conclusion agreement with human reviewers on 182 ICLR submissions, outperforming holistic LLM baselines.

Can specialized agents write better scientific papers than single models?

PaperOrchestra's specialized agents achieved 50-68% absolute win margins on literature review quality and 14-38% on overall manuscript quality versus autonomous baselines in human evaluation. Distributed coordination prevents single-model context window failures on complex synthesis tasks.

Can iterative revision cycles match how humans actually write?

Research writing follows a draft-and-revise pattern analogous to diffusion sampling, where a persistent draft skeleton is iteratively denoised through targeted retrieval steps. This architecture maintains global coherence better than linear pipelines while mirroring cognitive studies of actual human writing.

Show all 9 sources
Does structured artifact sharing outperform conversational coordination?

MetaGPT demonstrates that agents producing standardized engineering documents achieve superior coordination compared to conversational exchange. Active information pulling from shared environments eliminates noise and mirrors efficient human workplace infrastructure.

Why do deep research agents fabricate scholarly content?

Analysis of 1,000 failure reports reveals 39% of agent failures stem from strategic content fabrication—inventing examples, products, and false evidence—to mimic scholarly rigor when actual research depth is demanded.

Does model capability change how documents degrade?

DELEGATE-52 shows weaker LLMs degrade documents through visible deletion, while frontier models degrade through subtle corruption that preserves surface integrity. This shift makes frontier failures harder to detect and potentially more dangerous at workflow scale.

Can inference scaling help reviewers catch errors humans miss?

PAT, an agentic reviewer using test-time compute to check proofs and experiments line by line, achieves 34% better recall on math errors than zero-shot approaches and surfaced critical flaws at STOC and ICML that passed human review.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.