Can knowledge graphs teach models deep domain expertise?

Language models acquire general abstractions through top-down self-supervised learning on vast corpora, but this approach captures surface-level regularities rather than deep domain expertise. Bottom-up curriculum learning from knowledge graphs offers an alternative: KG paths naturally encode compositional reasoning chains where atomic triples (e.g., "Methane Contains Element Carbon") compose into multi-hop paths that build toward higher-order understanding (e.g., methane's bonding structure through C-H bonds → sigma bonds → single covalent bonds).

The pipeline synthesizes 24,000 reasoning tasks from a medical KG, paired with structured thinking traces derived from diverse medical primitives. Fine-tuning QwQ-32B on this curriculum produces QwQ-Med-3, which significantly outperforms state-of-the-art open-source and proprietary reasoning models across 15 medical domains on the ICD-Bench evaluation suite.

The key architectural insight: KG topology naturally induces the bottom-up curriculum — beginning with atomic relations and composing them into increasingly complex reasoning chains. This mirrors how human students build expertise through pedagogical structure (foundational → advanced chapters), not encyclopedic browsing. Previous neuro-symbolic and probabilistic graph inference approaches attempted similar hierarchical reasoning from primitives but failed to generalize beyond synthetic regimes; LMs provide the generalization capability that symbolic systems lacked.

The broader implication challenges the AGI-as-breadth paradigm: domain-specific superintelligence may be achievable through relatively small models (32B) fine-tuned on structured domain knowledge, composing into broader intelligence through interacting specialist agents — analogous to how human society acquires expertise through collaborative specialization.

This connects to:

• Can organizing knowledge structures beat raw training data volume? — both demonstrate that knowledge structure matters more than knowledge volume; StructTuning is taxonomy-based, KG curriculum is path-based
• Does gradually tightening token budgets beat fixed budget training? — curriculum design principle applies to both RL token budgets and KG complexity progression
• Why does reasoning training help math but hurt medical tasks? — bottom-up KG curriculum may work precisely because it provides both domain knowledge (lower layers) and compositional reasoning (higher layers) simultaneously
• Does procedural knowledge drive reasoning more than factual retrieval? — KG paths encode procedural/relational knowledge, not just factual triples