A tiny, fixed-size nudge to training can reshape a model's behavior — but does that trick work on every AI model, or just a couple?

This explores whether the 'O(1)-bit signal' result, the idea that a tiny, fixed-size training signal can produce large behavior changes, holds for model families beyond Llama and Qwen. The short answer is that this corpus can't tell you. None of the retrieved notes discuss the O(1)-bit theory, and none test it on other model families. Treat what follows as nearby evidence about why the question is worth asking, not as an answer to it.

The most relevant lesson is that the same cheap signal can behave differently across model families. Take difference-of-means vectors, a nearly free way to detect reward hacking from a model's existing internal activations. On the same coding benchmark, they caught 3.1% more hacks than a full LLM monitor in Kimi K3 but 7.9% fewer in GLM 5.2 difference-of-means-vectors-are-similarly-effective-to-llm-monitors-but-virtually. The method was identical, but the result flipped direction between families. If a lightweight signal can swing like that for monitoring, a theory about lightweight training signals needs testing family by family before anyone calls it general.

There is also a deeper reason for caution. Two models can score the same on every benchmark while organizing their knowledge in very different ways inside. One can be cleanly structured and the other 'fractured' and fragile Can models be smart without organized internal structure?. If a small signal works by nudging an internal structure that is already there, how well it works may depend on how a given model family was pretrained. Llama and Qwen sharing a pattern doesn't guarantee that Mistral, Gemma, or a mixture-of-experts model would. Architecture differences matter too. Looped or recurrent designs Can looped models beat parameter-matched standard Transformers? and byte-level models that skip tokenization Can byte-level models match tokenized performance with better efficiency? process information differently enough that results from standard transformers may not carry over.

The takeaway you may not have expected: evidence from two families is much weaker than it sounds when those families have similar architectures and overlapping pretraining recipes. The detector example shows that a method's apparent advantage can disappear or reverse when the model family changes. To test the O(1)-bit theory properly, you'd want at least one family with a different architecture or training pipeline. This corpus doesn't yet contain that test.