How do language agents become optimizable computational graphs automatically?
This explores how language agents can be re-cast as computational graphs that a system can tune on its own — and what the corpus says about when that automatic optimization actually buys you something versus when it hits a wall.
This explores how language agents become computational graphs that optimize themselves, rather than systems a human has to hand-redesign. The starting idea is deceptively simple: if you represent an agent as a graph where nodes are operations (a prompt, a tool call, a reasoning step) and edges are the flow of information between them, then famous techniques like chain-of-thought, tree-of-thought, and Reflexion stop looking like separate inventions and turn out to be the same kind of structure wearing different clothes Can we automatically optimize both prompts and agent coordination?. Once everything is a graph, two things become tunable that used to require manual craft: the text inside each node (the prompts) and the wiring between nodes (which step talks to which). 'Automatic' means a search process can rewrite both axes instead of an engineer redrawing the pipeline by hand.
The interesting part is what fills those nodes and edges once you can optimize them. Capability routing — deciding which node handles a given subtask — can itself be learned: versioned capability vectors let agents discover each other by semantic match rather than hand-wired routing tables, so the graph's edges form themselves as new agents appear Can semantic capability vectors replace manual agent routing?. And the nodes don't all need to be expensive: most agentic subtasks are repetitive and well-defined, so a sensible optimized graph defaults to small models and reserves the large ones for the few nodes that truly need them Can small language models handle most agent tasks?. Optimization here is as much about cost topology as accuracy.
But the corpus pushes back hard on the fantasy of a graph that improves without limit. Self-improvement is formally bounded by a generation–verification gap: a model can propose fixes, but every reliable improvement needs something external to validate it, so a graph can't bootstrap itself out of its own blind spots through introspection alone What stops large language models from improving themselves?. Reflexion is the honest version of this — agents do get better across episodes, but only because the environment hands them an unambiguous success/failure signal they store as episodic memory; the binary feedback is what stops them from rationalizing Can agents learn from failure without updating their weights?. Automatic optimization works when there's a real external verifier closing the loop, and stalls when there isn't.
There's also a ceiling on what the optimization can discover. On genuine constrained-optimization problems, LLMs plateau around 55–60% regardless of scale or reasoning training Do larger language models solve constrained optimization better?, and a deeper diagnosis shows why: models can't actually execute iterative numerical procedures in latent space — they pattern-match a problem to something memorized and emit plausible-but-wrong values Do large language models actually perform iterative optimization?. Even RL fine-tuning often sharpens memorization rather than installing a real procedure, which out-of-distribution tests expose immediately Do fine-tuned language models actually learn optimization procedures?. So 'optimizing the graph' is optimizing arrangement and prompting — it does not conjure a reasoning capability the underlying nodes lack.
The quietly hopeful counter-thread is that structure, not scale, is where the leverage lives. Graph-based reasoning self-organizes toward a critical state where roughly 12% of edges stay semantically surprising, which is what keeps it discovering new connections instead of converging and going dead Why do reasoning systems keep discovering new connections?; learned traversal policies via MCTS and RL beat exhaustive reading by navigating a graph selectively within context limits Can learned traversal policies beat exhaustive graph reading?; and curricula built from knowledge-graph paths can give a mid-sized model domain superintelligence that far larger models miss Can knowledge graphs teach models deep domain expertise?. The thing you didn't know you wanted to know: making an agent into an optimizable graph isn't mainly an automation trick — it's a bet that the topology of how reasoning steps connect matters more than how big each step is, and the corpus says that bet pays off right up until you ask a node to do something the model fundamentally can't.
Sources 11 notes
Language agents represented as computational graphs—where nodes are operations and edges define information flow—reveal that CoT, ToT, and Reflexion are formally equivalent structures. This unified view enables automatic optimization of both node prompts and edge connectivity without manual redesign.
Versioned Capability Vectors embedded in HNSW indices couple semantic matching with policy and budget constraints, making capability discovery a first-class operation that scales sub-linearly as agent heterogeneity increases.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
Self-improvement in LLMs is formally bounded by the generation-verification gap, meaning every reliable fix requires something external to validate and enforce it. Models cannot escape this constraint through metacognition alone.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
Across constrained-optimization tasks, LLMs converge to ~55–60% constraint satisfaction independent of architecture, parameter count, or training regime. Reasoning models do not systematically outperform standard models, suggesting a fundamental ceiling rather than a scaling gap.
Research shows LLMs cannot perform iterative procedures in latent space. They recognize optimization problems as template-similar and emit plausible-looking but incorrect values, a failure mode that persists across model scale and training approaches.
Even GRPO-trained models show sharp performance drops on out-of-distribution variants (N-1 test sets) compared to in-distribution problems, indicating RL optimizes template-matching rather than genuine problem-solving procedures.
Analysis shows iterative graph reasoning evolves toward a stable phase where semantic entropy persistently dominates structural entropy, with ~12% of edges remaining semantically surprising despite structural connection, fueling ongoing discovery.
Graph-O1 replaces whole-graph ingestion with step-by-step agentic navigation using Monte Carlo Tree Search and reinforcement learning. This approach fits within LLM context windows while learning domain-specific traversal policies, though it trades certainty about the full graph for decision-making under uncertainty.
Fine-tuning a 32B model on 24,000 reasoning tasks derived from medical knowledge graph paths produces state-of-the-art performance across 15 medical domains, demonstrating that structured knowledge composition matters more than scale.