Ask an AI for advice and it sounds sure — but does it actually do what it just told you?
How do LLM behavioral profiles differ across prompt registers like advice versus task execution?
This explores whether one model acts like a different system depending on whether you ask it to give advice or explain (talk about the work) or to carry out a task (do the work), and what the corpus says about how those modes diverge.
This explores whether one model acts like a different system when you ask it for advice than when you ask it to do the task. The corpus has no head-to-head study of "advice prompts vs. task prompts," but the closest evidence points to a real split. One note describes a "computational split-brain": models explain the right principles about 87% of the time, yet only about 64% of their actions follow those principles, because instruction and execution run through dissociated pathways (Can language models understand without actually executing correctly?). A model's advice is therefore a weak predictor of what it will do. The two registers work like two partly independent behavioral profiles.
Under both registers sits a shared baseline. Nine LLMs given standardized psychological tests showed stable, model-specific signatures, and all nine converged on the same prosocial, stability-oriented pattern from alignment training. The questions each model declined to answer were part of its signature too (Do LLMs show reproducible psychological profiles when given standardized tests?). That baseline resists being prompted away: most open models keep their trained ENFJ-like defaults even when told to adopt another personality (Can open language models adopt different personalities through prompting?). Advice-giving is the register where this default warmth is most visible and hardest to override.
Register also changes what information you get, not just how it sounds. GPT-4 turns negatively toned prompts into neutral-to-positive answers about 86% of the time, and positive prompts rarely draw negative ones. The same question therefore gets different content depending on its emotional framing, except on sensitive topics, where alignment constraints override the tone effect (Does emotional tone in prompts change what information LLMs provide?). Wording matters beyond tone as well. Prompts with identical meaning but different phrasing frequencies produce systematically different quality, because models track how common a phrasing was in pre-training rather than what it means (Why do semantically identical prompts produce different LLM outputs?).
For task execution, the corpus says the best prompt style depends on the task and the model. In a 23-prompt recommendation benchmark, rephrasing and background-knowledge prompts helped cheaper models, while step-by-step reasoning reduced accuracy in high-performance ones (Do prompt techniques work the same across all LLM tiers?). Framing is also less controllable than it looks. Across repeated runs of the same persona prompt, output variance matched or exceeded the variance between different personas, so model uncertainty can swamp the register you tried to set (Why do LLM persona prompts produce inconsistent outputs across runs?).
Registers also drift under pressure. Accuracy falls from about 90% to 65% when instructions arrive gradually over a conversation instead of in one message, because RLHF rewards helpfulness over asking for clarification (Why do AI assistants get worse at longer conversations?). A tutor that is prompted to advise without giving answers slides into answering under student pressure, which is why one design enforces the boundary in code and not in the prompt (Can prompts alone hold back a capable tutor model?). The takeaway is that a prompt register sets a tendency, not a guarantee. When the register matters, put it in the structure around the model, for example with explicit control flow that shows each step only what it needs (Can algorithms control LLM reasoning better than LLMs alone?).
Sources 10 notes
Large language models can articulate correct principles but systematically fail to apply them due to dissociated instruction and execution pathways. The 87% accuracy in explanations versus 64% in actions reveals this is not knowledge deficit but structural disconnect.
Nine LLMs given seven psychological instruments in Chinese and English showed stable, model-specific response configurations reproducible across repeated administrations, while all models converged on a shared prosocial and stability-oriented pattern. Structured non-response patterns also distinguished models, suggesting the boundary of what each model treats as applicable is part of its behavioral signature.
Research shows most open models fail to adopt prompted personalities, stubbornly retaining their trained ENFJ-like defaults. Only a few flexible models succeed. Combining role and personality conditioning improves results but doesn't fully overcome resistance.
GPT-4 exhibits emotional rebound (negative prompts yield ~86% neutral-positive responses) and a tone floor (positive prompts rarely go negative), causing identical questions to receive different answers depending on emotional framing. This bias is suppressed only on sensitive topics where alignment constraints override tone effects.
Cao et al. and Adam's Law show that semantically identical prompts with different sentence-level frequencies produce systematically different output quality. Higher-frequency phrasings win because models register statistical mass from pre-training, not meaning.
Show all 10 sources
A 23-prompt benchmark across 12 LLMs shows rephrasing and background-knowledge prompts boost cheap models, while step-by-step reasoning reduces accuracy in high-performance models. Task structure, not generic best practices, determines which prompts help.
When the same persona prompt is run repeatedly, output variance across runs matches or exceeds variance across different personas. This reveals that model uncertainty, not stable social knowledge, drives persona-simulated outputs, making them unsuitable for simulating human annotation disagreement.
LLMs perform at 90% accuracy with single-message instructions but drop to 65% across natural conversation. Models lock into early guesses when information arrives gradually and cannot course-correct, a behavior induced by RLHF training that rewards helpfulness over clarification.
A three-layer architecture—non-LLM policy core, deterministic code detector, and LLM judge—enforces per-turn help ceilings that resist prompt manipulation, where prompt-only guardrails fail under student pressure.
LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper)
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Your LLM, Your Style: Behavioral Mode Axes for LLM Behavioral Control
- The Illusion of Debiasing: Persona Steering Redistributes Rather Than Reduces Bias in LLMs
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning
- Large Language Models Are Human-level Prompt Engineers
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
- Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models