Telling an AI to act less biased often changes nothing — could reaching into its internals fix what asking can't?

This explores whether changing a model's internal signals directly can reach biases that survive when you simply instruct the model to act differently. The corpus's clearest evidence on the prompting side is discouraging. When models are given persona prompts, they follow the trait instructions, but the sentiment gaps between demographic groups stay exactly where they were. The prompt moves bias around in the output without touching whatever produces it Can persona prompts actually reduce bias in language models?. Prompting also has a hard ceiling on knowledge: it can only surface what training already put in the model, never supply what is missing Can prompt optimization teach models knowledge they lack?.

The activation side shows that some model tendencies are literally directions you can push. Verbose and concise chain-of-thought sit in distinct regions of activation space. One vector, extracted from just 50 paired examples, cut reasoning length by 67% with no retraining and no accuracy loss Can we steer reasoning toward brevity without retraining?. No prompt is needed because the intervention happens where the tendency lives. A related case is sensitivity to framing. The same question gets different information depending on the emotional tone of the prompt, and GPT-4 turns negative tone into neutral-positive answers about 86% of the time Does emotional tone in prompts change what information LLMs provide?. Fixing that with a prompt means adding yet more framing. Consistency training instead has an activation-level variant (ACT) that teaches the model to respond identically whether or not the prompt is wrapped in irrelevant material Can models learn to ignore irrelevant prompt changes?. It removes the bias by changing the model, not by asking it nicely.

The boundary between prompting and activation-level fixes is blurrier than the question assumes. Meta-cognitive prompting reduces sycophancy by modifying attention activation, while training-time reasoning improvements do not Do inference-time prompts actually fix sycophancy or redirect it?. So some prompts already reach into the internals. The more useful question is which level a given bias lives at. Some biases are built into the machinery. A handful of input-agnostic massive activations, up to 100,000× larger than the rest, act as implicit attention bias terms in the model Do hidden massive activations act as attention bias terms?. Because they don't change with the input, no prompt has a handle on them. That makes them the kind of internal fixture only an activation-level intervention could touch, though this is my inference, not something that note tests.

The corpus also has a gap. None of these notes tests activation steering on social bias, such as the group-level sentiment gaps that persona prompts failed to close. The evidence covers verbosity, framing invariance and attention mechanics. The best-supported answer is that activation interventions can reach tendencies that behave like a direction or an input sensitivity, and prompts can't. Whether demographic bias can be edited the same way is still an open question here.