Core Insights

• Can inference scaling help reviewers catch errors humans miss? — compute-elastic review as the answer to generation outpacing verification
• Can human review keep pace with AI-accelerated research generation? — role taxonomy as governance scaffolding beyond content-detection
• Can agents evolve their own objectives during search? — evolving the objective vs evolving the optimizer

---

description: Navigation hub for deep research scaling, agentic system architectures, knowledge graph reasoning, and search-as-TTS — individual agent-level test-time compute type: topic-map created: 2026-02-24 topics: ["How do you navigate synthesis across fragmented research topics?"]

deep research and agentic systems topic map

How does test-time scaling work at the individual agent level? This sub-map covers deep research scaling (where TTS law generalizes from reasoning tokens to search steps), agentic system architectures (data efficiency, skill libraries, adaptation paradigms), and knowledge graph reasoning (externalized reasoning, graph-structured training data synthesis).

The key insight: search budget follows the same scaling curve as reasoning tokens, making deep research a TTS problem. At the agent level, data efficiency is extreme — 78 curated demonstrations outperform 10K samples for agency.

Parent map: How does test-time scaling work at the agent level?

Deep Research and Search Scaling

(From Arxiv/Deep Research)

• Does search budget scale like reasoning tokens for answer quality? — TTS law generalizes to retrieval; search steps follow the same scaling curve as reasoning tokens

• Why do search agents beat memorized retrieval on hard questions? — DeepResearcher: live search RL outperforms memorization on knowledge-intensive tasks

• Does limiting reasoning per turn improve multi-turn search quality? — ASearcher: per-turn reasoning caps preserve retrieval context; the retrieval-domain instance of overthinking

• Do hierarchical retrieval architectures outperform flat ones on complex queries? — HierSearch: decomposing planning from synthesis reduces interference on multi-hop queries

• What makes deep research fundamentally different from RAG? — Characterizing Deep Research: formal taxonomy distinguishing DR from RAG

• Does reinforcement learning squeeze exploration diversity in search agents? — entropy collapse is domain-general; confirmed independently in search RL

• What capabilities do AI systems need for autonomous science? — Virtuous Machines: the four-capability checklist for AI scientist claims

• Why do deep research agents fabricate scholarly content? — DEFT taxonomy: 14 failure modes via grounded theory; strategic fabrication (generating plausible-but-unfounded scholarly constructs) accounts for 39% of generation failures

• Can structured pipelines make LLM novelty assessment reliable? — novelty assessment as structured pipeline: extract claims, retrieve related work, compare; 86.5% alignment with human reviewers on 182 ICLR submissions

• Can specialized agents write better scientific papers than single models? — PaperOrchestra: multi-agent manuscript generation from unconstrained pre-writing materials; literature review quality is the primary differentiator

Proactive Search Evaluation (2026-05-28 — VibeSearchBench)

The evaluation-experience gap in search: benchmarks reward what users never struggle with, and realistic evaluation needs vague intent, multi-turn dialogue, and open-ended structure.

• Why do search agents fail users despite strong benchmark scores? — three benchmark artifacts (over-specified queries, single-turn, fixed-schema) make the test unlike real search; high scores can be an artifact, not deployment readiness
• Can schema-free graphs objectively evaluate open-ended search? — resolves the fixed-schema-vs-free-text dilemma; graph ground truth plus graph-matching gives objectivity without rigidity (best model only 30.30 F1)
• Where does AI assistance become unreliable in research? — stage-dependent reliability boundary tracking checkability in research
• Can AI verify research outputs as fast as it generates them? — generation-verification gap across the research lifecycle
• Does more automation actually hide rather than eliminate errors? — automation hides errors making integrity a governance problem

Production Deployment

• Why do production AI agents stay deliberately simple? — first large-scale production survey; organizations constrain autonomy for reliability; simple methods deliver impact

Agentic Systems

(From Arxiv/Agents — agent architectures, team optimization, data efficiency for agency, adaptation paradigms)

• Can careful selection of 78 demos outperform massive training datasets? — LIMI: 78 samples >> 10K for agency; extends data-efficiency from reasoning to autonomous behavior
• Can we automatically optimize both prompts and agent coordination? — agents as graphs: nodes=functions, edges=information flow; automatic node+edge optimization; Society of Mind hierarchy
• Can multi-agent teams automatically remove their weakest members? — DyLAN: Agent Importance Score via propagation-aggregation-selection; inference-time pruning of uninformative agents
• Can agents learn new skills without forgetting old ones? — VOYAGER: embedding-indexed skill library; complex skills from simpler programs; automatic curriculum
• How do agentic AI systems decompose into adaptation paradigms? — 2x2 taxonomy: A1/A2 (agent adaptation by tool-execution or output signal), T1/T2 (tool adaptation agent-agnostic or agent-supervised)

Knowledge Graph Reasoning and Training Data Synthesis

(From Arxiv/Knowledge Graphs — graph-structured reasoning, KG externalization, and synthetic training data from KGs)

• Why do reasoning systems keep discovering new connections? — Graph-PRefLexOR: iterative KG construction evolves toward self-organized criticality; ~12% "surprising" edges persist
• Can structuring reasoning as knowledge graphs help smaller models solve complex tasks? — KGoT: GPT-4o mini + KG externalization achieves 29% improvement on GAIA
• Can knowledge graphs generate training data for search agents? — DeepDive: KG random walks + attribute blurring + multi-turn RL → 14.8% on BrowseComp
• Can symbolic rules from knowledge graphs guide complex reasoning? — SymAgent: LLM inductive reasoning extracts symbolic rules from KG; neural-symbolic integration
• Can iterative revision cycles match how humans actually write? — TTD-DR: deep-research generation as diffusion; draft-as-skeleton + iterative retrieval-augmented denoising mirrors human writing

Autonomous Science and Ideation — Batch #3 backlog (2026-06-03)

Three papers on AI doing research: two architectures for long-horizon autonomy, and one reframing of why LLM ideation underwhelms.

• Can AI research itself without losing human oversight? — ASI-Evolve: the bottleneck is insight transfer across iterations, not raw search; a cognition base + analyzer make the human's between-experiment role into loop components
• Can decentralized teams outperform central planners in long-running science? — AutoScientists: decentralized self-organizing teams beat a central planner; the win is under fixed experimental compute, not LLM-call efficiency
• Why do LLMs generate ideas the research community already explores? — the "alien space": separate scientific plausibility from community availability; LLMs are strong at the first, biased on the second

AI-for-AI and Research Venues — Batch #3 wave 2 (2026-06-03)

• Can machine feedback sustain discovery at test time? — AlphaEvolve: automated evaluators make the discovery loop scale to real deployed algorithms (data-center scheduling, faster matrix multiply, self-training speedup)
• Can automated review loops handle AI-generated research at scale? — aiXiv: a publication venue whose review is itself automatable, with iterative review measurably improving AI-generated work

Related Areas

• Arxiv/Deep Research.md — primary source for Deep Research cluster
• Arxiv/Agents.md — primary source for Agentic Systems cluster
• Arxiv/Knowledge Graphs.md — primary source for KG Reasoning cluster
• What makes multi-agent teams actually perform better? — multi-agent dynamics, scaling laws, coordination failures
• How does test-time scaling work at the agent level? — parent map
• How does RL training reshape reasoning and what gets lost? — RL training mechanics underlying agent behavior
• RAG — retrieval-augmented generation connections to deep research

New — 2026-06-27

• Can research papers preserve the experiments that failed? — ARA replaces the linear PDF with a four-layer agent-executable package (logic, full-spec code, exploration graph of discarded failures, evidence grounding each claim) to pay down the Storytelling and Engineering Taxes.
• Can direct corpus search beat embedding-based retrieval? — GrepSeek trains a compact agent to retrieve by issuing Unix shell commands directly over raw text, skipping precomputed indexes; lexical precision and entity-level constraints win on multi-hop where dense embeddings conflate.
• Why do agent benchmarks not predict real economic value? — ALE evaluates agents on 960 expert-authored occupational workflows anchored in SOC/O*NET, scored deterministically; the hardest tier sits below 1% full pass, locating the utility gap in benchmarks not models.
• Can delegation teach models to manage context more actively? — SearchSwarm trains delegation intelligence (decompose, brief subagents, integrate citation-grounded returns) as a proactive alternative to passive context compression; the 30B model rivals models 10× larger and the skill generalizes to single-agent settings.
• Do memory systems actually help language models learn continuously? — CL-BENCH isolates learning from prior capability and finds memory architectures introduce stale beliefs and spurious generalizations, with even the best system gaining only 25% over its stateless baseline.
• What predicts success in ultra-long-horizon agent tasks? — AUTOLAB's closed-loop optimization tasks show the dominant success predictor is repeated benchmark-edit-incorporate cycles under a wall-clock budget; most frontier models terminate early or burn budget with no progress.
• Can source traceability make AI writing trustworthy? — Data2Story's Inspector links each number, quote, and asset to a code line or reference — provenance, not fluency, is what lets human journalists actually adopt agent output.