How does algorithmic control flow define computational graph structure in LLM programs?
This explores how the algorithm wrapped around an LLM — the if/then branches, loops, and step ordering — becomes the actual computational graph: the nodes (LLM calls) and edges (information flow) that structure how a multi-step program reasons.
This reads the question as asking what happens when you stop treating an LLM as one big call and instead embed it inside explicit program logic — and how that logic literally becomes a graph. The corpus has a surprisingly clean answer: the control flow *is* the graph, not a metaphor for one. In LLM Programs, the surrounding algorithm manages control flow and state, handing each LLM call only the context relevant to its step Can algorithms control LLM reasoning better than LLMs alone?. The branching and sequencing you write in code is what decides which calls happen, in what order, and what each one can see.
The sharpest framing is that reasoning topologies classify as formal graph types. Chain-of-thought is a path graph, tree-of-thought is a tree, and graph-of-thought is an arbitrary directed graph — and this distinction is load-bearing, not decorative. Because GoT allows a node to have more than one incoming edge (in-degree > 1), it can express divide-and-conquer synthesis that a tree structurally cannot Can reasoning topologies be formally classified as graph types?. So your choice of control flow doesn't just organize the same computation differently; it changes what computations are even reachable.
The payoff of taking the graph view literally is that you can optimize it. When language agents are represented as computational graphs — nodes are operations, edges define information flow — CoT, ToT, and Reflexion turn out to be formally equivalent structures, just different wiring Can we automatically optimize both prompts and agent coordination?. Once that's true, you can automatically tune both the prompts inside nodes and the connectivity between them, instead of hand-redesigning the whole pipeline. Control flow becomes a thing you search over, not a thing you commit to up front.
Here's the part you might not have come looking for: a big reason this works is *information hiding*. The algorithm's job is partly to keep step-irrelevant context away from each call Can algorithms control LLM reasoning better than LLMs alone?, which sidesteps context-window and capability limits and makes each sub-task debuggable in isolation. A related move externalizes reasoning state into knowledge-graph triples, letting a small model (GPT-4o mini) jump 29% on hard GAIA tasks by building structure outside the model rather than holding it all in latent space Can structuring reasoning as knowledge graphs help smaller models solve complex tasks?. The graph isn't just routing — it's offloading memory and control the model is bad at holding internally.
Which is exactly why this whole approach earns its keep: the corpus is blunt that LLMs don't reliably do this structure on their own. They recognize graph data as a category but largely ignore the actual connections — shuffle the topology and performance barely moves Can language models actually use graph structure information? — and they pattern-match templates instead of executing genuine iterative procedures in latent space Do large language models actually perform iterative optimization?. Algorithmic control flow defines the computational graph precisely because the model won't construct or respect that structure unsupervised; the program supplies the rigor the network lacks.
Sources 6 notes
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
CoT, ToT, and GoT map precisely to path graphs, trees, and arbitrary directed graphs respectively. The topology is not metaphorical but defines actual computational structure—GoT's in-degree > 1 enables divide-and-conquer synthesis that trees cannot express.
Language agents represented as computational graphs—where nodes are operations and edges define information flow—reveal that CoT, ToT, and Reflexion are formally equivalent structures. This unified view enables automatic optimization of both node prompts and edge connectivity without manual redesign.
Knowledge Graph of Thoughts (KGoT) achieves 29% improvement on GAIA Level 3 tasks using GPT-4o mini by externalizing reasoning into iteratively constructed KG triples. The approach improves transparency, reduces bias, and enables quality control over reasoning steps.
LLMs develop attention shifts toward node tokens after training, but randomly shuffled topology barely affects performance. Models treat graph data as a category to recognize rather than as structured relationships to use.
Research shows LLMs cannot perform iterative procedures in latent space. They recognize optimization problems as template-similar and emit plausible-looking but incorrect values, a failure mode that persists across model scale and training approaches.