Can we automatically optimize both prompts and agent coordination?
This explores whether language agents can be represented as computational graphs whose structure and content adapt automatically. Why it matters: current agent systems require hand-engineered orchestration; automatic optimization could unlock more capable multi-agent systems.
The computational graph representation of language agents resolves a fragmentation problem: diverse prompting techniques (CoT, ToT, Reflexion, Self-Consistency) and multi-agent frameworks (AutoGPT, CAMEL, ChatDev) all look different in implementation but share the same underlying structure.
Three levels of the hierarchy:
- Node — a fundamental operation: LLM inference, tool use, function call, embodied action
- Graph (Agent) — multiple nodes forming a coherent functional entity; edges define execution topology
- Composite Graph (Swarm) — agents whose collective capabilities exceed individual agents; inter-agent edges establish collaboration
This is not merely descriptive. The graph representation enables automatic optimization along two axes:
- Node optimization — each node adapts its prompts based on previous input and task feedback (analogous to prompt tuning)
- Edge optimization — the connectivity between nodes changes, allowing suboptimal agent organization to be overcome and prompting techniques to be automatically recombined
Because Can reasoning topologies be formally classified as graph types?, the insight here is that the graph representation extends beyond individual reasoning traces to entire agent systems. A single agent's reasoning is a graph; a multi-agent system is a composite graph. The same formalism covers both, enabling optimization at both levels.
The Society of Mind (Minsky 1988) framing is deliberate: higher-level intelligence emerges from combining simpler, modular cognitive components. This is the same principle that makes Can extreme task decomposition enable reliable execution at million-step scale? work — decomposition into graph nodes enables both parallelism and optimization.
The practical implication: instead of hand-engineering agent orchestration patterns, define the problem as a graph and let optimization discover the topology. This is the agent-systems analog of what Do reasoning cycles in hidden states reveal aha moments? found for reasoning traces — topology matters, and it's optimizable.
Inquiring lines that read this note 49
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can AI agents learn to improve performance from experience?- Can language agents be represented as optimizable computational graphs?
- How do language agents become optimizable computational graphs automatically?
- How do agents discover and select which tools to invoke?
- Should agent capability be optimized separately from general capability?
- What makes planning, tool use, and reasoning into jointly optimizable subsystems?
- How would a bi-level agent restructure objective functions during discovery?
- When should you optimize agent behavior versus tool performance separately?
- Does parallel task structure determine optimal multi-agent architecture?
- How should proportionality constraints be implemented in agentic systems?
- What makes composable abstractions emerge under performance pressure in agent systems?
- Why do rigid orchestration frameworks fail where generative environment specifications succeed?
- How do graph-based reasoning topologies map to multi-agent interaction patterns?
- How do multi-agent routers balance flexibility against interpretability in design?
- Can structured protocols outperform pure emergence in autonomous multi-agent coordination?
- Which task requirements does each graph view address in agent systems?
- How do context engineering limits relate to multi-agent coordination problems?
- Should parallel agents be specialized with graph abstractions instead of general subagents?
- Does encoding organizational structure into agent systems outperform natural language coordination?
- Should agent orchestration be treated as an explicit graph rather than emergent behavior?
- What organizational structures emerge when optimization pressure targets communication patterns?
- Can adaptive prompt-difficulty allocation compound with architectural efficiency improvements?
- Can compute-optimal scaling work without co-optimizing the prompt itself?
- Can architectural changes alone achieve compute-optimal per-prompt scaling?
- Can prompt optimization inject genuinely new knowledge into a model?
- Can prompt optimization for clarity automatically improve token efficiency?
- What knowledge can prompt optimization actually activate in trained models?
- Can gradient-free prompt optimization beat manual prompt engineering at scale?
- How does algorithmic control flow define computational graph structure in LLM programs?
- Can algorithmic control flow over prompts simulate traditional programming languages?
- Can optimization algorithms exploit the shift between procedural and planning bottlenecks?
- Can construction-time routing and runtime agent pruning be combined effectively?
- How do planning and memory compress agentic system costs?
- How do external prompt artifacts improve agent behavior compared to inline instructions?
- What structural constraints produce recursion costs in agentic systems?
- Can we design efficient agents by targeting constraints directly?
- Should new agent protocols replace existing ones or layer on top of them?
- How do postmortem convention-setting stages enable language evolution in agents?
- Why does structured protocol coordination outperform free-form agent-to-agent communication?
- Can structured artifacts replace natural language for multi-agent coordination?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can reasoning topologies be formally classified as graph types?
This explores whether Chain of Thought, Tree of Thought, and Graph of Thought represent distinct formal graph structures with different computational properties. Understanding this matters because the topology itself determines what reasoning strategies are possible.
the individual-reasoning analog; this note extends the graph formalism to agent systems
-
Do reasoning cycles in hidden states reveal aha moments?
What if the internal loops in model reasoning—visible in hidden-state topology—correspond to the reconsidering moments that happen during reasoning? This note explores whether graph cyclicity captures a mechanistic signature of insight.
topology optimization for reasoning; this note demonstrates topology optimization for agent orchestration
-
Can extreme task decomposition enable reliable execution at million-step scale?
Can breaking tasks into maximally atomic subtasks with voting-based error correction solve the fundamental reliability problem in long-horizon tasks? This challenges whether better models or better decomposition is the path to high-reliability AI systems.
MAKER as a specific composite-graph pattern
-
Can multiple LLMs coordinate without explicit collaboration rules?
When multiple language models share a concurrent key-value cache, do they spontaneously develop coordination strategies? This matters because it could reveal how reasoning models naturally collaborate and inform more efficient parallel inference.
emergent coordination as a graph property
-
Can algorithms control LLM reasoning better than LLMs alone?
Explores whether embedding LLMs within algorithmic control flow—where programs manage state and context filtering—enables complex task decomposition beyond what LLMs achieve through self-managed reasoning chains.
LLM Programs are computational graphs with predetermined topology: the algorithm defines the edges, each LLM call is a node, and information hiding enforces the graph structure; the difference is that LLM Programs fix the graph at design time while optimizable graphs discover it through optimization
-
Can modular cognitive tools unlock reasoning without training?
Can reasoning capabilities be elicited by structuring LLM calls as isolated cognitive operations—understanding, recalling, examining, and backtracking—rather than through reinforcement learning?
cognitive tools are specific node implementations within the computational graph: each tool (understand, recall, examine, backtrack) is a sandboxed function node that can be composed into agent-level reasoning graphs
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Language Agents as Optimizable Graphs
- AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems
- Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence
- FlowReasoner: Reinforcing Query-Level Meta-Agents
- Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
- Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Original note title
language agents as optimizable computational graphs unify prompting techniques and enable automatic optimization of both prompts and agent orchestration