How does model weight freezing across users affect virtual instance individuation?
This explores a puzzle in how AI is deployed: when one set of frozen model weights serves every user, what actually makes your conversation or agent feel like a distinct individual rather than the same generic model — and the corpus answer is that individuation lives in context and memory, not in the weights.
This explores how a single frozen model — identical weights shared across millions of users — nonetheless produces what feels like a personalized, individuated instance for each one. The short version from the corpus: nothing in the weights individuates you, so everything that does must come from what surrounds them. The most direct frame is that an LLM isn't a single character at all but holds a probability distribution over many possible characters at once, narrowing as the conversation accumulates Does an LLM commit to a single character or maintain many?. Your 'instance' is that narrowing — a path carved through a shared distribution by your specific context, not a separate entity. This also explains why two users with the same model get different personalities, and why regenerating can yield a different one: the weights are frozen, but the sample isn't.
If individuation can't come from weight updates, the interesting question becomes how far context and memory alone can carry it. Here the corpus is surprisingly optimistic. Frozen agents that build structured, causal-form memory — recording not just what worked but the conditions under which it applied — continually improve across trials and even transfer gains to new environments, outperforming generic reflection by a wide margin Can frozen language models continually improve through memory structure alone?. Similarly, frozen models that extract reusable natural-language 'skills' from their context reason measurably better without touching a single parameter, and those skills port across model backbones Can frozen models learn better by extracting context into skills?. So a virtual instance individuates by accumulating its own memory and skill scaffolding on top of shared weights — the shape of that memory matters more than the weights it sits on.
There's a deeper reason this works, which reframes the whole question. Evidence suggests RL post-training teaches a base model *when* to deploy capabilities it already latently contains, not *how* to do new things — reasoning strategies pre-exist as activation patterns before any training Does RL post-training create reasoning or just deploy it?. If the shared frozen model is already a vast reservoir of latent behaviors, then per-user context isn't teaching it anything new — it's selecting which latent behaviors to surface. Individuation is a routing-and-selection phenomenon, not a learning one. That's why freezing weights across users costs surprisingly little: the diversity was always in the model, waiting to be conditioned out.
The catch is that the same context that individuates can also corrupt. Because each instance is defined by its accumulated history, errors in that history compound: once a model's own mistakes fill its context, performance degrades non-linearly, and scaling the model doesn't rescue it Do models fail worse when their own errors fill the context?. Your individuated instance is only as good as the trajectory it has accumulated — a frozen model gives you a clean substrate, but the per-user path through it can drift somewhere bad. And there's a quieter failure: a per-user instance can hit identical benchmark numbers while its internal representations are fractured and fragile, so two 'instances' that look equally competent may behave very differently under pressure Can models be smart without organized internal structure?.
What you didn't know you wanted to know: individuation across a shared frozen model is closer to *casting* than to *building*. The model is a fixed superposition; your context is the spotlight that picks one simulacrum out of the crowd and the memory that keeps it standing there. Freezing the weights doesn't flatten everyone into sameness — it just relocates where individuality is allowed to live.
Sources 6 notes
Research shows LLMs don't commit to a single character but instead maintain a probability distribution over many consistent simulacra. Each response samples from this distribution, explaining why regenerations can yield different personalities while remaining consistent with prior context.
Agents using causal-form memory (preserving applicability conditions) outperform generic reflection by 23 points on repeated trials and gain 4-17 points transferring to new environments, showing memory shape matters more than parameter updates.
Extracting natural-language rules from context into reusable skills improves frozen model reasoning without weight updates. On CL-bench, this lifts GPT-4.1 from 11.1% to 16.5%, with skills transferable across model backbones.
Evidence shows base models already contain reasoning capability in latent form; RL training optimizes deployment timing rather than capability creation. Hybrid models recover 91% of performance gains by routing tokens only, and activation vectors for reasoning strategies pre-exist before any RL.
Error accumulation in context causes non-linear performance degradation in long-horizon tasks. Model scaling does not fix this; only test-time compute through thinking models reduces the effect by preventing error-contaminated context from biasing reasoning.
Models trained with SGD can contain all the linearly decodable features needed for a task while maintaining fundamentally broken internal organization. This makes them vulnerable to perturbation and distribution shift invisible to standard evaluation metrics.