Teaching an AI a brand-new fact goes smoothly, but rewriting something it already believes turns unpredictable. Why?
What distinguishes new associations from existing ones at the computational level?
This explores what actually differs, inside a model or system, between picking up a brand-new association and using one it already has. It isn't asking about the brain. It's asking how the two are stored, added and used.
This explores what actually differs, inside a model or system, between picking up a brand-new association and using one it already has. The corpus has no single note that defines the difference, so what follows is stitched together from several angles. The clearest one is an asymmetry in training.
Synthetic-document fine-tuning can insert new facts into a model predictably. When the new documents contradict something the model already believes, the result becomes unpredictable, and a stronger push doesn't make it controllable Can training data edits reliably override what models already believe?. Adding to empty space and rewriting something already wired in are different operations. Existing associations also aren't inert storage. LLMs reason by leaning on parametric commonsense and token associations, and when the meaning is stripped out of a task, performance collapses even with the correct rules sitting in context Do large language models reason symbolically or semantically?. Old associations are part of the machinery doing the reasoning, which is probably why they resist editing.
What makes a connection count as new? In iterative graph reasoning, systems settle into a state where semantic entropy stays above structural entropy. Roughly 12% of edges link things that are structurally connected yet still semantically surprising Why do reasoning systems keep discovering new connections?. On that view, a new association isn't one that lacks a link. It's a link between things that weren't already close in meaning, and the system keeps producing them.
Architecture may draw the same line. Engram pairs constant-time N-gram lookup with mixture-of-experts computation, and a balanced split beats either alone. The gains show up most in reasoning and code, not pure retrieval Can lookup memory and computation work together better than either alone?. The note doesn't frame this as new versus old, so this part is my inference. Familiar patterns are cheap to fetch, and unfamiliar combinations seem to need real computation.
Recommendation systems show the bookkeeping most plainly, because every new item needs an identity. Hashed embedding tables put collisions on the most frequent items, and they get worse as new IDs keep arriving Why do hash collisions hurt recommendation models so much?. Multi-facet identifiers combine an ID, a title and attributes, so a new item is distinct and also carries meaning Can item identifiers balance uniqueness and semantic meaning?. One caution comes from a note on internal structure. A model can pass every test while its internals are fractured Can models be smart without organized internal structure?. So a newly learned association that behaves correctly may still not be integrated the way an old one is, and standard evaluation wouldn't show it.
Sources 7 notes
Synthetic documents add novel information to models predictably, but contradict and revise existing associations unpredictably. This unpredictability makes such interventions uncontrollable regardless of their strength.
When semantic content is decoupled from reasoning tasks, LLM performance collapses even with correct rules in context. Models rely on parametric commonsense and token associations rather than formal logical manipulation, constraining reasoning to training distribution semantics.
Analysis shows iterative graph reasoning evolves toward a stable phase where semantic entropy persistently dominates structural entropy, with ~12% of edges remaining semantically surprising despite structural connection, fueling ongoing discovery.
Engram combines O(1) N-gram lookup with Mixture-of-Experts routing, revealing a U-shaped scaling law where balanced allocation to both mechanisms outperforms either alone. Gains appear largest in reasoning and code rather than pure retrieval.
Monolith's empirical work shows that real recommendation systems have power-law distributed frequencies, causing collisions to accumulate precisely on the entities models need most accurate. Fixed-size hashed tables worsen this over time as new IDs arrive.
Show all 7 sources
TransRec shows that combining numeric IDs, titles, and attributes into structured identifiers solves three problems simultaneously: distinctiveness from IDs, semantics from text, and generation grounding from structural constraints. Neither pure IDs nor pure text alone achieves all three.
Models trained with SGD can contain all the linearly decodable features needed for a task while maintaining fundamentally broken internal organization. This makes them vulnerable to perturbation and distribution shift invisible to standard evaluation metrics.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Self-Organizing Graph Reasoning Evolves into a Critical State for Continuous Discovery Through Structural-Semantic Dynamics
- Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
- Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
- Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
- Monolith: Real Time Recommendation System With Collisionless Embedding Table
- GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning