A small AI tuned to a specific field built a better knowledge graph than a giant one — can it also write the answers?

This explores whether the approach that worked for building GraphRAG graphs, a small domain-tuned model instead of a big general one, also works for the last step, where a model reads the retrieved graph material and writes the answer. The corpus tests only the first half. It has strong evidence for extraction and none for a specialized small answerer, so the answering case below is inference from neighboring results.

The extraction result is the anchor. A 7B domain-adapted model beat Qwen2.5-32B at building the graph, because extraction leans on language comprehension, which scales weakly with size, and not on stored world knowledge (Can smaller language models outperform larger ones at graph extraction?). The answering stage looks similar on this axis. The facts have already been retrieved and placed in the prompt, so the model mostly has to read them faithfully and phrase a response. If that is the job, a small model has less to give up. The broader argument for small models points the same way: most agent work is repetitive, well-defined language tasks, and small models handle them at 10–30× lower cost (Can small language models handle most agent tasks?).

Structure also seems to help small models. Knowledge Graph of Thoughts had GPT-4o mini externalize its reasoning into knowledge-graph triples and gained 29% on hard GAIA Level 3 tasks (Can structuring reasoning as knowledge graphs help smaller models solve complex tasks?). Once the information is laid out as explicit relations, the small model no longer has to hold the whole chain in its head. A GraphRAG answerer reading graph paths gets similar scaffolding. For the specialization itself, DPO on a large teacher's correct and incorrect examples lets small models match big ones on rigid, format-sensitive tasks (Can small models match large models on function calling?). The same recipe could be tried with grounded and ungrounded answers as the pairs. Separately, student models trained on pruned reasoning chains beat those trained on frontier-model compression (Which tokens in reasoning chains actually matter most?), which suggests reasoning traces can be shrunk without losing what matters.

The main risk is that answering is not always reading. When a question needs several hops across graph paths, the small model has to reason as well as read. Reasoning models keep beating non-reasoning ones at any inference budget, because training gives them a protocol that extra tokens can use (Can non-reasoning models catch up with more compute?). A small answerer therefore has to be trained for graph reasoning. Tuning it on language alone would not be enough. A second limit is that a better answerer cannot repair a bad graph. Editing under 0.05% of source words dropped GraphRAG accuracy from 95% to 50% by corrupting the extracted structure (How vulnerable is GraphRAG to tiny text manipulations?). Most of the risk therefore sits in the extraction stage, and that is where the small specialized model has the clearest evidence.

So a small answerer is plausible, most clearly for lookup-style questions and least clearly for multi-hop ones, and the corpus has not tested it directly. The open question is whether GraphRAG answering behaves like extraction, where language skill is enough, or like reasoning, where training regime decides the outcome.