Could an AI interface that mixes 'tell it your goal' with hands-on clicking and dragging actually solve the tension between the two?

This explores whether stacking an intent layer (telling the AI your goal) on top of a direct-manipulation layer (clicking, dragging, editing) resolves the friction between them. One caveat first: the collection has no material on Nielsen's three-layer model. So this answer can't judge his specific proposal. What it can do is show what the corpus says about the tension his model is trying to resolve, and that turns out to be the more interesting story.

The sharpest version of the tension comes from Is chat the wrong default interface for AI products?. Direct manipulation was invented to remove a layer of symbols between people and their work. You drag the file instead of typing a command. Chat-first AI puts that layer of symbols back. For goal-directed work, that is a step backward. Medeiros doesn't settle this by layering. The proposal is a routing rule instead: two gate questions (how fast does this need to be, and where does the value come from?) decide whether a given task should be talked through or clicked through. Can task-oriented UI patterns reduce the friction of AI chat interfaces? gets closer to a layered answer. Friedman names five places where interfaces can avoid slow typed prompts: capturing intent, shaping output, refining, running agentic tasks, and embedding in workflows. The examples are hybrids, such as node graphs that make intent visible, 'style lenses' applied to output, and canvases you can manipulate directly. In those designs, intent and direct manipulation are mixed together, not stacked as separate layers.

The part you might not expect is that the tension isn't only about speed. People often don't know what they want until they see something. Why can't users articulate what they want from AI? argues that intent develops through interaction. A pure intent layer assumes the user arrives with a finished goal. Here, the fix is for the system to show options, so the user evaluates choices instead of describing a goal from a blank page. A VR study gives concrete evidence. Do spatially-anchored previews stabilize VR geometry editing tasks? found that pairing clarifying questions with 3D previews placed directly in the scene made progress steadier and cut the number of conversation rounds. Peak performance didn't change, though. The combination made results less erratic, not better at the top end. That is a realistic benchmark for what any layered design can promise.

There's also a twist the question doesn't anticipate. The person at the interface may be an AI agent, not a human. Can interface design help agents complete computer tasks better? found that adding cues aimed at agents improved agent success without hurting human usability. Why do vision-only GUI agents struggle with screen interpretation? shows that agents also struggle when one layer has to do two jobs at once, in this case working out what is on screen and deciding what to do. Splitting those jobs apart helps. So the case for layers may be strongest where humans and agents share one interface: each audience gets the layer that suits it.

If you want a map for judging any layered proposal, Nielsen's included, How should designers structure human-GenAI interaction systems? lays out the dimensions to check, including user control, model feedback and engagement level. A three-layer design is one way of choosing settings on those dimensions. Whether it actually removes the tension is not something this corpus can answer yet.