Can query-time graph construction replace pre-built knowledge graphs?
Does building dependency graphs from individual queries at inference time offer a more flexible and cost-effective alternative to constructing knowledge graphs over entire document collections upfront?
GraphRAG builds a knowledge graph over the entire corpus before any query is served. The graph captures relationships between entities in documents, enabling multi-hop reasoning by traversal. Performance on complex tasks is strong. But the pre-building cost is prohibitive: token overhead for graph construction, latency for updating as the corpus evolves, and the graph being LLM-generated may include irrelevant or redundant relationships.
Worse: the pre-built graph is static. Real-world queries vary in type and complexity, requiring different logic structures for accurate reasoning. A graph built for financial reporting queries may not support the traversal patterns needed for medical diagnosis queries on the same corpus.
LogicRAG inverts this: instead of building a graph over the corpus, build a graph over the query at inference time. Decompose the query into subproblems, construct a directed acyclic graph (DAG) encoding logical dependencies between subproblems, topologically sort for execution order, resolve each subproblem via retrieval conditioned on previously resolved subproblems.
The result: retrieval plans that match the logical structure of the specific query. A multi-hop question about "which policies introduced by this figure were later reversed?" generates a DAG with specific dependency edges that no pre-built graph would have pre-encoded. Context pruning (LLM-based summarization of retrieved content) and graph pruning (merging semantically similar subproblems) reduce token overhead.
This generalizes Do hierarchical retrieval architectures outperform flat ones on complex queries? — the same separation principle, implemented dynamically at the query level rather than architecturally.
Inquiring lines that read this note 50
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do knowledge graphs enable efficient multi-hop reasoning over alternatives?- How does LLM-PKG compare to mining product relations directly from interaction data?
- How do community summaries and selective traversal differ as graph scaling strategies?
- What graph structures better support multi-hop reasoning than pairwise edges?
- Why does community detection in knowledge graphs outperform pure retrieval or pure summarization?
- Can hierarchical entity extraction from books enable both textual and visual reasoning?
- How do LLMs and knowledge graphs work together in different integration patterns?
- How do community-based summaries differ from retrieval-based traversal in knowledge graph RAG?
- How does map-reduce over communities compare to flat multi-hop retrieval architectures?
- Can inference-time query decomposition replace pre-built knowledge graph structures?
- Why do binary edges lose information when representing multi-entity relations?
- What is the computational cost of constructing and traversing hypergraphs?
- How does hypergraph accumulation differ from single-pass graph retrieval?
- Which knowledge structure types best fit different query types?
- How does knowledge graph structure enable multi-hop reasoning in recommendations?
- How does GraphRAG differ from HippoRAG despite both using knowledge graphs?
- Can query-time logic graphs match the efficiency of pre-built knowledge graph indexing?
- What makes graph traversal superior to vector embeddings for relational reasoning?
- How does graph structure amplify poisoning compared to flat document retrieval?
- When should you use knowledge graphs instead of semantic vector retrieval systems?
- How do graph databases address the relational query failures that LLMs encounter?
- How do hierarchical knowledge graphs solve similar multimodal retrieval problems in books?
- How does upfront graph construction trade off against retrieval performance over time?
- Can small edits to source text compromise entire knowledge graph reliability?
- Can graph-based retrieval with knowledge graphs scale to multi-hop reasoning?
- How can knowledge graphs improve over pure embedding retrieval?
- Can knowledge graphs externalize and validate reasoning steps during inference?
- How do review-augmented systems compare to knowledge graph approaches?
- How do taxonomy-based retrieval scaffolds improve model performance at inference time?
- Can knowledge graph structure be exploited for efficient multi-hop retrieval?
- What makes graph-matching more faithful than fixed-schema evaluation methods?
- How do knowledge graphs scale as training data for open-ended search tasks?
- Can knowledge graphs built at inference time outperform pre-built retrieval augmented generation?
- Why do fixed-schema outputs fail to capture real knowledge relationships?
- How do hierarchical research architectures improve multi-hop query accuracy?
- How do hierarchical research architectures handle multi-hop queries better?
- Can fixed heuristics like PageRank match learned traversal policies on graphs?
- Why does Personalized PageRank naturally discover concepts multiple hops from query seeds?
- Can cyclic aggregation relationships enable fully inductive graph-based recommendation?
- What makes web retrieval more effective than static knowledge bases?
- How do time-based and entity-based queries differ from semantic similarity retrieval?
- How does query decomposition reduce retrieval costs at inference?
- When is vector embedding retrieval actually faster and cheaper than graph databases?
- When should relational graph traversal replace vector embedding retrieval?
- Do graph databases outperform embeddings for relational retrieval tasks?
- What makes graph databases better than embeddings for relational queries?
- How do vector embeddings fail to capture task-relevant document relationships?
Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do hierarchical retrieval architectures outperform flat ones on complex queries?
Explores whether separating query planning from answer synthesis into distinct architectural components improves performance on multi-hop retrieval tasks compared to unified single-pass approaches.
LogicRAG is a concrete mechanism for the "query planning" step; DAG construction makes the planning structure explicit and executable
-
When do graph databases outperform vector embeddings for retrieval?
Vector similarity struggles with aggregate and relational queries that require traversing multiple entity connections. Can graph-oriented databases with deterministic queries solve this failure mode in enterprise domain applications?
pre-built graph approach; LogicRAG offers the relational reasoning benefit without the build cost by constructing query-specific logic at inference time
-
Can structuring reasoning as knowledge graphs help smaller models solve complex tasks?
Can externalizing LLM reasoning into structured knowledge graph triples enable smaller, cheaper models to match the performance of much larger ones? This explores whether making reasoning explicit and inspectable improves both capability and transparency.
KGoT and LogicRAG are complementary inference-time graph construction approaches for different purposes: LogicRAG builds query-dependency DAGs for structured retrieval planning, KGoT builds reasoning-trace KGs for externalized computation; both demonstrate that graph structure adds value over flat attention-based reasoning at different points in the pipeline
-
Can routing queries to task-matched structures improve RAG reasoning?
Does matching retrieval structure type to task demands—tables for analysis, graphs for inference, algorithms for planning—improve reasoning accuracy over uniform chunk retrieval? This explores whether cognitive fit principles from human learning transfer to AI systems.
LogicRAG's query DAG is the "graph" option in StructRAG's five-structure routing space; cognitive fit theory provides the theoretical grounding for when DAG-structured retrieval outperforms chunk-based retrieval
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures
- Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation
- Can Language Models Solve Graph Problems in Natural Language?
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Multi-hop Question Answering via Reasoning Chains
- RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
- Talk like a Graph: Encoding Graphs for Large Language Models
- Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research
Original note title
inference-time query logic graphs avoid the cost and inflexibility of pre-built knowledge graph rag