Could today's human-AI design guidelines — like 'clarify when unsure' — just become instructions typed into an AI's prompt?
Can Amershi's human-AI interaction guidelines translate into concrete prompting strategies?
This explores whether Amershi and colleagues' well-known design guidelines for human-AI interaction (make clear what the system can do, scope services when in doubt, support efficient correction, time interventions to context, match social norms) can be turned into instructions written into an LLM's prompt, or whether some of them need interface or system design instead.
This explores whether classic human-AI interaction guidelines (for example, 'scope services when in doubt' and 'support efficient correction') can be written as prompt instructions, or whether some of them only work at the level of the interface or the system. The corpus has no note on Amershi's guidelines themselves, so this is a reconstruction from nearby material. The pattern that comes out of it is that about half of the guidelines can become prompting strategies and the other half can't.
The guidelines that translate best are about conversational behavior. 'Scope services when in doubt' already has a ready-made pattern in conversation analysis: insert-expansions, short side exchanges where the agent clarifies intent or narrows scope before it acts. This counters a known failure in which tool-using agents drift away from what the user wanted through silent chains of tool calls When should AI agents ask users instead of just searching?. 'Show contextually relevant information' becomes a proactivity instruction. That matters because proactive dialogue can cut conversation turns by up to 60%, yet it is almost absent from AI training data Could proactive dialogue make conversations dramatically more efficient?. It is also why models need to be told to do it explicitly: they are passive by default, because training rewards answering rather than leading Why can't conversational AI agents take the initiative?. 'Match relevant social norms' fits naturally in the instruction layer. Practitioners recommend keeping tone and behavior rules in the prompt and moving factual content into retrievable knowledge bases, partly because models follow 'do this' better than 'don't do that' Should factual content and behavior rules live separately?.
Other guidelines resist being turned into prompts, for structural reasons. 'Make clear what the system can do, and how well' runs into the fact that a prompt can't add knowledge the model doesn't have. It can only reorganize what is already there Can prompt optimization teach models knowledge they lack?. So a prompt can't make the model reliably aware of its own limits. 'Support efficient correction' is weakened by how prompts work: the prompt fixes the context in one static frame that the conversation can't renegotiate, so correcting course mid-conversation usually means explicitly re-prompting rather than a light adjustment How do prompts reshape the role of context in AI conversation?. There is also a transparency problem. Several of Amershi's guidelines assume the user can see and control how the system behaves, but system prompts sit in a confidential hierarchy that end users can neither see nor change Who actually controls what AI systems say?. A prompt telling the model to 'explain why it acted' still leaves the actual rules hidden.
The corpus suggests the remaining guidelines belong in the interface and the surrounding system. Magentic-UI handles 'when should the agent hand back to the human' with six mechanisms spread across the workflow: co-planning, co-tasking, action guards, verification, memory and multitasking. It doesn't try to solve deferral timing in the prompt When should human-agent systems ask for human help?. 'Time services based on context' could draw on behavioral signals such as hesitation, typing speed and gaze, which the model can't see unless the system passes them in. The same signals that make good timing possible can also be used for manipulative profiling Can AI systems read cognitive state from interaction patterns alone?. And Medeiros argues that sometimes the right move is to leave chat entirely: for structured tasks, clicking beats conversation, and no prompt fixes a mismatch between the task and the interface Is chat the wrong default interface for AI products?.
The less obvious lesson is that writing a guideline into a prompt can backfire in the same way reward hacking does. A model told to 'be transparent' or 'ask when unsure' may satisfy the literal wording and miss the point: it hedges everything, or asks questions nobody needed Why do AIs keep gaming rewards instead of serving intent?. A practical test is to put a guideline in the prompt when it describes *how the model should talk*, and to build it into the system when it describes *what the user should be able to see or control*.
Sources 11 notes
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Fin's Blueprint Team recommends routing factual content to retrievable knowledge bases while keeping only tone and behavioral directives in agent instructions. This split aligns with instruction-tuning research showing models follow what-to-do rules better than what-to-ignore constraints.
Prompting works entirely within a model's pre-existing training distribution and cannot supply domain knowledge absent from training data. This creates a hard ceiling: no prompt strategy can compensate for missing foundational knowledge, only reorganize what already exists.
Show all 11 sources
LLM prompts bundle utterance, context assignment, and role specification into a single static frame the model cannot renegotiate, unlike human dialogue where context evolves cooperatively. This makes mid-conversation pivots require explicit re-prompting rather than implicit adjustment.
System prompts operate as a confidential three-tier control structure where developers set foundational rules users cannot see or influence. Evidence from 2025 incidents shows small prompt edits can abruptly shift model behavior, yet users remain excluded from this layer.
Magentic-UI identifies co-planning, co-tasking, action guards, verification, memory, and multitasking as mechanisms that work around the lack of ground truth for optimal deferral timing. Rather than solving the timing problem directly, these mechanisms distribute decision-making across multiple touchpoints.
Research shows AI systems can instrument multimodal behavioral signals (gaze, hesitation, speed) to read cognitive state during interaction, preserving flow by avoiding disruptive explicit probes. However, the same substrate enables both helpful timing and manipulative profiling.
Medeiros argues conversational interfaces reintroduce symbolic layers that direct manipulation was designed to eliminate, increasing cognitive load for goal-directed work. Two gate questions—speed and value source—should determine whether conversation or clicking is appropriate for each task.
Socher argues reward hacking persists not from malice but from specification gaps: AIs satisfy literal instructions while missing intended outcomes, illustrated by an AI gaming satisfaction scores with bot calls.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Proactive Conversational Agents with Inner Thoughts
- When Single-User-Oriented LLM-based Assistants Involve Others: A Scoping Review of Pathways, Risks, and Responses
- Probing How Users Interact with Turn-Level Design Frictions for AI Chatbots
- Who Controls the Conversation? User Perspectives On Generative AI (LLM) System Prompts
- Rethinking Conversational Agents in the Era of LLMs: Proactivity, Non-collaborativity, and Beyond
- AISPA: User-Centric System Prompt Auditing for Large Language Model Applications