Can we automatically optimize both prompts and agent coordination?
This explores whether language agents can be represented as computational graphs whose structure and content adapt automatically. Why it matters: current agent systems require hand-engineered orchestration; automatic optimization could unlock more capable multi-agent systems.
The computational graph representation of language agents resolves a fragmentation problem: diverse prompting techniques (CoT, ToT, Reflexion, Self-Consistency) and multi-agent frameworks (AutoGPT, CAMEL, ChatDev) all look different in implementation but share the same underlying structure.
Three levels of the hierarchy:
- Node — a fundamental operation: LLM inference, tool use, function call, embodied action
- Graph (Agent) — multiple nodes forming a coherent functional entity; edges define execution topology
- Composite Graph (Swarm) — agents whose collective capabilities exceed individual agents; inter-agent edges establish collaboration
This is not merely descriptive. The graph representation enables automatic optimization along two axes:
- Node optimization — each node adapts its prompts based on previous input and task feedback (analogous to prompt tuning)
- Edge optimization — the connectivity between nodes changes, allowing suboptimal agent organization to be overcome and prompting techniques to be automatically recombined
Because Can reasoning topologies be formally classified as graph types?, the insight here is that the graph representation extends beyond individual reasoning traces to entire agent systems. A single agent's reasoning is a graph; a multi-agent system is a composite graph. The same formalism covers both, enabling optimization at both levels.
The Society of Mind (Minsky 1988) framing is deliberate: higher-level intelligence emerges from combining simpler, modular cognitive components. This is the same principle that makes Can extreme task decomposition enable reliable execution at million-step scale? work — decomposition into graph nodes enables both parallelism and optimization.
The practical implication: instead of hand-engineering agent orchestration patterns, define the problem as a graph and let optimization discover the topology. This is the agent-systems analog of what Do reasoning cycles in hidden states reveal aha moments? found for reasoning traces — topology matters, and it's optimizable.
Inquiring lines that use this note as a source 32
This note is a source for these synthesized inquiries. Follow a line forward into its question, or open it to trace back to all of its sources.
- Can language agents be represented as optimizable computational graphs?
- When should you optimize agent behavior versus tool performance separately?
- Why do rigid orchestration frameworks fail where generative environment specifications succeed?
- Can prompt engineering fully prevent role flipping in LLM agents?
- Can adaptive prompt-difficulty allocation compound with architectural efficiency improvements?
- Can prompt optimization inject genuinely new knowledge into a model?
- How does algorithmic control flow define computational graph structure in LLM programs?
- Does parallel task structure determine optimal multi-agent architecture?
- Can construction-time routing and runtime agent pruning be combined effectively?
- How do graph-based reasoning topologies map to multi-agent interaction patterns?
- Can compute-optimal scaling work without co-optimizing the prompt itself?
- Can prompt optimization for clarity automatically improve token efficiency?
- How do language agents implement prompts as executable computational graphs?
- What knowledge can prompt optimization actually activate in trained models?
- Can algorithmic control flow over prompts simulate traditional programming languages?
- How do language agents become optimizable computational graphs automatically?
- How do multi-agent routers balance flexibility against interpretability in design?
- Can optimization algorithms exploit the shift between procedural and planning bottlenecks?
- How do agents discover and select which tools to invoke?
- Why do sequential derivation and parallel agent modeling conflict?
- Why does language ambiguity cause premature convergence in multi-agent systems?
- Should agent capability be optimized separately from general capability?
- How should proportionality constraints be implemented in agentic systems?
- Can multimodal agents use entity-centric graphs within this three-axis framework?
- What makes planning, tool use, and reasoning into jointly optimizable subsystems?
- What makes composable abstractions emerge under performance pressure in agent systems?
- How do planning and memory compress agentic system costs?
- How do external prompt artifacts improve agent behavior compared to inline instructions?
- What structural constraints produce recursion costs in agentic systems?
- Can we design efficient agents by targeting constraints directly?
- What components of agent scaffolding most impact domain-specific output quality?
- Should new agent protocols replace existing ones or layer on top of them?
Related concepts in this collection 6
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can reasoning topologies be formally classified as graph types?
This explores whether Chain of Thought, Tree of Thought, and Graph of Thought represent distinct formal graph structures with different computational properties. Understanding this matters because the topology itself determines what reasoning strategies are possible.
the individual-reasoning analog; this note extends the graph formalism to agent systems
-
Do reasoning cycles in hidden states reveal aha moments?
What if the internal loops in model reasoning—visible in hidden-state topology—correspond to the reconsidering moments that happen during reasoning? This note explores whether graph cyclicity captures a mechanistic signature of insight.
topology optimization for reasoning; this note demonstrates topology optimization for agent orchestration
-
Can extreme task decomposition enable reliable execution at million-step scale?
Can breaking tasks into maximally atomic subtasks with voting-based error correction solve the fundamental reliability problem in long-horizon tasks? This challenges whether better models or better decomposition is the path to high-reliability AI systems.
MAKER as a specific composite-graph pattern
-
Can multiple LLMs coordinate without explicit collaboration rules?
When multiple language models share a concurrent key-value cache, do they spontaneously develop coordination strategies? This matters because it could reveal how reasoning models naturally collaborate and inform more efficient parallel inference.
emergent coordination as a graph property
-
Can algorithms control LLM reasoning better than LLMs alone?
Explores whether embedding LLMs within algorithmic control flow—where programs manage state and context filtering—enables complex task decomposition beyond what LLMs achieve through self-managed reasoning chains.
LLM Programs are computational graphs with predetermined topology: the algorithm defines the edges, each LLM call is a node, and information hiding enforces the graph structure; the difference is that LLM Programs fix the graph at design time while optimizable graphs discover it through optimization
-
Can modular cognitive tools unlock reasoning without training?
Can reasoning capabilities be elicited by structuring LLM calls as isolated cognitive operations—understanding, recalling, examining, and backtracking—rather than through reinforcement learning?
cognitive tools are specific node implementations within the computational graph: each tool (understand, recall, examine, backtrack) is a sandboxed function node that can be composed into agent-level reasoning graphs
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Language Agents as Optimizable Graphs
- FlowReasoner: Reinforcing Query-Level Meta-Agents
- Graph-enhanced Large Language Models in Asynchronous Plan Reasoning
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
- FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
- Intelligent AI Delegation
- Demystifying Chains, Trees, and Graphs of Thoughts
Original note title
language agents as optimizable computational graphs unify prompting techniques and enable automatic optimization of both prompts and agent orchestration