If an AI tool shows you every design decision it's making as you work, does that keep you in charge — or just busier?
Can architectural design choices resist the automation of design work itself?
This explores whether the way an AI system is built (its architecture) can keep human design judgment necessary, even as AI starts automating design work itself.
This explores whether how an AI system is built can keep human designers in the loop, or whether design work gets automated no matter what. The corpus gives two answers that pull against each other. One camp says architecture can keep design judgment in play. The other says the parts of design we call irreplaceable are exactly what gets formalized next. Read together, they suggest that architecture doesn't stop automation. What it can decide is where human judgment ends up.
The most direct case for resistance comes from Lam and colleagues' 'just-in-time objectives' approach Does visible design structure keep HCI expertise necessary as AI advances?. Their system doesn't hide how it generates and evaluates outputs. It makes those decisions visible at every interaction, so design judgment becomes an ongoing part of how the system runs rather than a step someone finishes before launch. A related line of work treats the LLM as a material designers can shape directly. The Canvil tool lets designers adjust model behavior through prompts inside Figma, with no engineering background needed Can designers shape LLM behavior without deep technical knowledge?. Another note adds that AI context (the prompt, the conversation history, retrieved data) keeps shifting in a way fixed software screens never did. It argues this calls for a new kind of design work, context engineering, rather than less of it How does AI context differ from conventional software context?.
The opposing case is blunt. One note traces a pattern across past automation waves. Practitioners name some human capacity as irreplaceable, and then that capacity gets written down as a process a machine can run. Designer taste is next, already being captured in evaluation rubrics and preference data. The designer's job moves from making things to writing evals Will AI automation eventually formalize designer taste?. Anthropic's planner-generator-evaluator setup shows this happening in practice. It turns a question like 'is this beautiful?' into concrete criteria a model can grade consistently, calibrated with a few examples and running for hours per build Can subjective design judgment be turned into consistent grading criteria?. So the 'visible design decisions' approach may not be resistance at all. It may be the first step toward formalizing those decisions.
Here's the twist the corpus adds. Automated design is less reliable than it looks. One benchmark found that generative UI tools fail to carry out about a quarter of the design rationales they say they're following. For functional requirements the figure rises to 34%, and the tools recognized only half the UX principles written into their prompts Do generative UI tools actually implement their stated design rationales?. Splitting design work across many agents creates its own problem. Cognition's Walden Yan argues that parallel subagents each make unstated design choices and end up contradicting each other Why do parallel subagents produce conflicting design decisions?. Work on AI companions finds that ethics and UX can't be separated, and each principle has to be weighed by context, which resists clean rubrics Why don't design recommendations transfer cleanly to AI companions?.
The takeaway you may not have expected: the most useful architectural choice may not be about keeping designers busy. It may be about designing the handoff. Holldack, Banh and Strobel describe agentic systems that delegate tasks back to humans Can AI systems delegate tasks back to humans?. That suggests a future where the system itself decides when a design call needs a person. The IMF makes a parallel argument for payments. It splits probabilistic AI decisions from deterministic settlement, because some layers need guarantees a model can't give Should AI agents make payment decisions independently?. Design may need the same kind of split: let AI generate, and build in fixed points where human judgment is required by the structure rather than left optional. The corpus doesn't yet have a study testing whether that holds up over time.
Sources 10 notes
Lam and colleagues argue that just-in-time objectives architecture makes design decisions between AI and interfaces visible, ensuring HCI expertise remains critical. By surfacing generation-and-evaluation steps at every interaction rather than hiding them, the architecture structures ongoing design judgment into the system itself.
Canvil demonstrates that designers can effectively shape LLM behavior via a low-barrier Figma widget for prompt authoring and testing, bringing user-centered judgment directly into model adaptation without requiring engineering expertise.
AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.
Historical automation waves follow a pattern: practitioners identify a core human capacity as irreplaceable, then that capacity gets formalized into processes machines can execute. Taste is already being formalized through evaluation rubrics and preference data that AI applies, shifting the designer's role from executor to eval author.
Anthropic's three-agent system rephrases subjective questions like 'is this beautiful?' into concrete criterion-based assessments that a model can grade consistently. Running up to four hours per build with live browser interaction, the evaluator reduced score drift through few-shot calibration and penalized generic outputs.
Show all 10 sources
A benchmark of 24 tasks across five tools found roughly 25% of design rationales go unimplemented, rising to 34% for functional requirements. Tools recognized only half the UX principles embedded in prompts.
Walden Yan argues multi-agent parallelism is fundamentally fragile: subagents embed unwritten assumptions in every action, so agents working independently converge on conflicting choices. The fix is reverting to single-threaded linear agents where context remains continuous.
A multi-method study of design principles found that borrowed recommendations create tension because ethical and UX concerns cannot be separated. Each principle—safety, engagement, empathy, consistency—requires context-sensitive application that weighs both dimensions differently depending on user and situation.
Holldack, Banh and Strobel argue that agentic information systems capable of independent decision-making can transfer rights and responsibilities to humans, reversing the traditional one-directional delegation model. This requires moving beyond agency-primacy frameworks to delegation theory that accounts for bidirectional task assignment.
An IMF analysis argues that agent-mediated payments must split probabilistic AI decision-making (Layer 1) from deterministic authorization and settlement (Layers 2–3) because core payment infrastructure requires predictability and legal enforceability that probabilistic reasoning cannot guarantee.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Design Theater: Evaluating the Gap Between User-Facing Design Reasoning and Implementation in Generative UI Tools
- Just-In-Time Objectives: A General Approach for Specialized AI Interactions
- What does Generative UI mean for HCI Practice?
- Guidelines for Human-AI Interaction
- Prototyping Multimodal GenAI Real-Time Agents with Counterfactual Replays and Hybrid Wizard-of-Oz
- Design Principles for Generative AI Applications
- A Framework of User Experience Principles for Human-AI Agent Interaction in the Workplace
- The human-authorship halo: attribution bias in literary style evaluation by humans and AI