Chat forces you to type out what you want — what other screen designs could skip that step and still get AI to do real work?
What UI patterns beyond chat complement AI for task-oriented workflows?
This explores which interface designs besides a chat box help people get concrete work done with AI, and when those designs beat conversation or fall short of it.
This explores which interface patterns besides chat help people get concrete work done with AI, and where each one pays off or costs something. The corpus starts with a diagnosis. For goal-directed work, chat makes people describe their intent in typed words, and that is slow. Direct manipulation (clicking, dragging, editing the thing itself) was designed to remove exactly this kind of translation step. Is chat the wrong default interface for AI products? calls chat-by-default a regression rather than an innovation for structured tasks. It offers two test questions: does this task need speed, and where does the value come from? Can task-oriented UI patterns reduce the friction of AI chat interfaces? turns this into five design levers: capturing intent, shaping output, refining results, running agentic tasks, and embedding AI in existing workflows. Concrete examples include node diagrams that show what the AI thinks you asked for, 'style lenses' for adjusting output, and canvases you work on directly instead of describing changes to.
The strongest evidence comes from generative interfaces, where the AI builds a task-specific UI on the fly (a dashboard, a small tool, an interactive view) instead of replying with a block of text. Users preferred these over chat in more than 70% of cases, especially for information-dense tasks Do generated interfaces outperform text-based chat for most tasks?. There is a catch, though. The TaskArtisan study found that generated analysis widgets make results clearer but harder to change halfway through a task. Customizing them pushes non-programmers toward thinking like engineers Do generated analysis UIs really work better than chat?. Chat is weak at structure but easy to bend, and structured UIs show the reverse. The best designs probably let you switch between the two rather than picking one.
One lateral finding: structure beats conversation even when no humans are involved. In MetaGPT, AI agents coordinated better by producing and reading standardized documents (specs, designs, plans in a shared workspace) than by chatting with each other Does structured artifact sharing outperform conversational coordination?. For human-facing design, this suggests a shared artifact that both person and AI can see and edit may be the real complement to chat. That fits Atlassian's Rovo guidelines, which treat visible system state, preserving the user's flow, and multiplayer human-AI collaboration as concrete UI commitments How should AI design connect individual work to team knowledge?. Visible state matters especially for AI. Its working context (the prompt, the history, retrieved data) keeps shifting in ways users can't keep in their heads. That makes showing context a design job in its own right How does AI context differ from conventional software context?.
Some patterns change conversation itself rather than replacing it. A 'full-duplex' frontend lets you hand off a task to run in the background while the conversation keeps going, then folds the result back into the same timeline Can frontends handle delegation while staying conversationally engaged?. Other work targets chat's passivity. LLMs mostly react, because training rewards answering questions, not leading toward a goal Why can't conversational AI agents take the initiative?. In simulations, offering relevant information before being asked cut dialogue turns by up to 60% Could proactive dialogue make conversations dramatically more efficient?. Conversation analysis gives a formal rule for when an agent should stop and ask a clarifying question instead of quietly chaining tool calls away from what the user meant When should AI agents ask users instead of just searching?.
The surprising last turn: interfaces now have a second audience. Computer-use agents also have to read and operate UIs. Adding agent-specific cues for perception, element targeting, and control improved agent task completion with no measurable cost to human usability Can interface design help agents complete computer tasks better?. So the question may not be 'what replaces chat' but 'what interface serves both the person and their agent working on the same task.'
Sources 12 notes
Medeiros argues conversational interfaces reintroduce symbolic layers that direct manipulation was designed to eliminate, increasing cognitive load for goal-directed work. Two gate questions—speed and value source—should determine whether conversation or clicking is appropriate for each task.
Friedman identifies five design levers—intent capture, output shaping, refinement, agentic tasks, and workflow embedding—that sidestep slow typed prompts. Examples include node-based intent visualization, style lenses for output, and canvas manipulation, shifting AI interfaces away from conversational chat toward direct interaction.
Research shows users strongly prefer LLM-generated interactive interfaces—dashboards, tools, animations—over text blocks, especially for structured and information-dense tasks. Structured representation and iterative refinement reduce cognitive load.
TaskArtisan found that GUI widgets improve clarity and presentation in LLM-assisted analysis but introduce rigidity and prompting overhead. This trade-off between malleability and specification appears unavoidable: easier-to-use UIs are harder to customize mid-workflow, while flexible UIs demand engineering-style thinking from non-programmers.
MetaGPT demonstrates that agents producing standardized engineering documents achieve superior coordination compared to conversational exchange. Active information pulling from shared environments eliminates noise and mirrors efficient human workplace infrastructure.
Show all 12 sources
Atlassian's published design guidelines for its Rovo AI product specify that AI should connect individual work to team knowledge through context-awareness, flow preservation, transparent system state, and fluid human-AI multiplayer collaboration, operationalizing abstract collective productivity principles as concrete UI commitments.
AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.
Realtime-Venus demonstrates that delegated requests, results, and intervening dialogue can share one ordered record, letting foreground interaction continue while background tasks execute. A dual-loop runtime keeps conversation flowing and folds results back in naturally.
Research shows LLMs including ChatGPT cannot initiate topics, plan strategically, or lead conversations because their training optimizes for responding to queries, not creating dialogue from agent goals. This passivity is reinforced by alignment objectives and masked by fluent-sounding outputs.
Simulations show proactivity—providing relevant information without being asked—cuts dialogue turns by 60% in medium-complexity domains. This behavior mirrors human conversation and Grice's maxims but is almost entirely absent from AI datasets and research benchmarks.
Tool-enabled LLMs drift from user intent through silent tool chaining. Conversation analysis reveals insert-expansions—clarifying intent, scoping responses, enhancing appeal—as a formal framework for proactive user consultation that prevents misunderstanding instead of recovering from it.
UI-Verse benchmark testing found that augmented heuristics targeting agent perception, grounding, and control consistently improved agent task completion and efficiency. Human studies confirmed these designs preserved usability for human users with no observable regressions.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- When Single-User-Oriented LLM-based Assistants Involve Others: A Scoping Review of Pathways, Risks, and Responses
- Proactive Conversational Agents in the Post-ChatGPT World
- DiscussLLM: Teaching Large Language Models When to Speak
- Probing How Users Interact with Turn-Level Design Frictions for AI Chatbots
- Design Theater: Evaluating the Gap Between User-Facing Design Reasoning and Implementation in Generative UI Tools
- What does Generative UI mean for HCI Practice?
- Proactive Conversational Agents with Inner Thoughts
- TaskArtisan: Designing Composable Generative Widgets for LLM-Assisted Analysis