What graph structures would enable transformational creative reasoning in LLMs?
This explores what kinds of graph structure could push LLMs past recombining known ideas into genuinely *transformational* creativity — reshaping the idea space itself, not just searching within it — and the corpus turns out to disagree about whether structure should be imposed or allowed to emerge.
This explores what graph structures might enable *transformational* creative reasoning — and the first thing the corpus does is sharpen the target. Creativity isn't one thing: research splits it into combinational, exploratory, and transformational modes, and notes that today's LLM reasoning methods address only conventional problem-solving, leaving the creative paradigms — especially transformation, where the rules of the space themselves get rewritten — almost entirely unaddressed Can LLMs reason creatively beyond conventional problem-solving?. So the question isn't 'can a graph help reasoning' but 'what graph lets a model break out of its own conceptual frame.'
The most direct answer comes from work showing that the right graph isn't a fixed scaffold but a *self-organizing* one. When reasoning is run as an iteratively grown graph, the system drifts toward a critical state where semantic surprise persistently outruns structural connection — roughly 12% of edges stay semantically surprising even after they're structurally linked, and that residual surprise is what keeps fueling new discovery Why do reasoning systems keep discovering new connections?. That's a concrete candidate for transformation: a graph that never fully settles, where new edges keep reframing the meaning of existing nodes rather than just filling in gaps.
But there's a sharp catch the corpus surfaces: LLMs are bad at actually *using* graph structure. Models develop attention toward node tokens after training yet barely change behavior when the topology is randomly shuffled — they recognize 'this is a graph' as a category but ignore the connections themselves Can language models actually use graph structure information?. So any structure meant to drive creative reasoning can't just be handed to the model as data; it has to be made operative. Two notes show ways to do that: deriving explicit symbolic rules from a knowledge graph's topology to create navigational plans that bind language to structure Can symbolic rules from knowledge graphs guide complex reasoning?, and externalizing reasoning into iteratively built KG triples so each step is inspectable and steerable Can structuring reasoning as knowledge graphs help smaller models solve complex tasks?.
There's a deeper tension worth following. One line argues the real reasoning happens in latent-state trajectories, not the surface text Where does LLM reasoning actually happen during generation? — which suggests transformational moves might live in hidden geometry a triple-graph never captures. The counterweight is that fully formalizing reasoning *loses* information: partial symbolic augmentation, enriching natural language with selective structure rather than replacing it, beats both pure language and full formalization Why does partial formalization outperform full symbolic logic?. Read together, they imply the productive graph is a *partial* one — structured enough to constrain wandering, loose enough to preserve the semantic ambiguity that creative leaps feed on.
That 'wandering' is the failure mode to design against: reasoning LLMs explore unsystematically, lacking validity, effectiveness, and necessity, so success collapses exponentially with problem depth Why do reasoning LLMs fail at deeper problem solving?. The unintuitive synthesis here is that transformational creativity and rigorous search want *opposite* things from a graph — search wants pruning and necessity, transformation wants persistent surprise and unsettled edges. The corpus hints the resolution is layered: an algorithmic or modular control structure handling the systematic part Can algorithms control LLM reasoning better than LLMs alone?, wrapped around a critically self-organizing semantic graph that's deliberately kept from converging. Structure for rigor, criticality for novelty — not one graph, but two regimes coupled together.
Sources 9 notes
Research identifies combinational, exploratory, and transformational reasoning as distinct creative modes grounded in cognitive science. Existing LLM reasoning methods address only conventional problem-solving, leaving creative paradigms unaddressed and potentially explaining diversity collapse in ideation.
Analysis shows iterative graph reasoning evolves toward a stable phase where semantic entropy persistently dominates structural entropy, with ~12% of edges remaining semantically surprising despite structural connection, fueling ongoing discovery.
LLMs develop attention shifts toward node tokens after training, but randomly shuffled topology barely affects performance. Models treat graph data as a category to recognize rather than as structured relationships to use.
SymAgent derives symbolic rules from KG structure using LLM reasoning to create navigational plans that align natural language with graph topology. This approach captures structural reasoning patterns explicitly, outperforming retrieval methods that rely on semantic similarity alone.
Knowledge Graph of Thoughts (KGoT) achieves 29% improvement on GAIA Level 3 tasks using GPT-4o mini by externalizing reasoning into iteratively constructed KG triples. The approach improves transparency, reduces bias, and enables quality control over reasoning steps.
Evidence from CoT faithfulness tests, feature steering, and layer analysis suggests latent-state dynamics drive reasoning, while surface chain-of-thought serves as a partial interface. Hidden reasoning processes should be the default focus of study.
QuaSAR and Logic-of-Thought both achieve 4-8% accuracy gains by enriching natural language with selective symbolic elements rather than replacing it. Full formalization loses semantic information; pure language lacks structure. Augmentation preserves both.
Current reasoning models lack the three properties of systematic exploration: validity, effectiveness, and necessity. This causes success probability to drop exponentially with problem depth, making medium problems solvable but deep problems catastrophically harder.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.