An AI persona can ace a personality quiz, then act like a different character once a real conversation gets emotional.

This explores why a personality questionnaire given to an LLM persona can look fine while the persona acts differently in a real conversation. No note in the collection tests inventories head-to-head against in-context behavior, so this is an inference from neighboring work. That work points to three gaps: a questionnaire is a snapshot, it's usually a single pass, and it records what the persona says about itself rather than what it does.

The first gap is that an inventory asks fixed questions at one calm moment, but persona behavior moves over the course of a conversation. Mapping hundreds of character archetypes shows that emotional and meta-reflective conversations predictably pull a model away from its default Assistant persona, and the pull is strong enough that researchers have to cap activations along that axis to stop harmful shifts (How stable is the trained Assistant personality in language models?). A neutral questionnaire never triggers those situations. Drift also comes in different kinds: local wobble within a turn, slow global drift across a conversation, and outright factual contradictions. The team that trained simulators against drift needed three separate metrics to catch all of them (Can training user simulators reduce persona drift in dialogue?). A single inventory score averages these differences away.

The second gap is that what a persona says about itself may not be what it does. In one study, persona prompts made models follow the trait instructions, yet the underlying bias gap between groups stayed the same, because the prompt only changed the output layer (Can persona prompts actually reduce bias in language models?). A persona could answer "I'm agreeable" on a questionnaire and still behave like the base model on a task. Post-training also installs stable dispositions that persist under adversarial pressure, which is what separates them from role-play that collapses (Are RLHF personas performed characters or realized dispositions?, Are LLM personas realized or merely simulated through training?). Read together, the notes suggest the questionnaire answers may reflect that trained baseline as much as the persona you prompted.

The third gap is noise. When the same persona prompt was run repeatedly on annotation tasks, the variation between runs matched or exceeded the variation between different personas (Why do LLM persona prompts produce inconsistent outputs across runs?). That study looked at annotation rather than inventories. But if run-to-run noise can swamp the persona signal there, a one-shot inventory score could easily mistake noise for a stable trait.

The notes that do show personas working validate them on behavior in a situation rather than on self-report. Personas built from real behavioral data predicted A/B test direction 75–90% of the time, though they were unreliable when the true effect was near zero (Can behavior-based personas predict A/B test outcomes?). Persona simulations replicated 76% of published marketing-experiment effects, with success tracking how strong the original evidence was (Can AI personas reliably replicate human experiment results?). A layered persona with a scripted structure held its dialogue consistency better than a shallow description (Can layered persona architecture sustain coherent character behavior?). The practical lesson is to test the persona in the situations you care about, repeatedly and over many turns, instead of trusting a trait score.