When an AI tool maker writes design rules, does anyone actually check that the AI followed them?

This explores how design rules handed down by AI tool makers and platform vendors compare with design frameworks built up from studying real users. The collection has no study that tests the two against each other, so treat what follows as a picture put together from nearby evidence. That evidence points one way: a written design principle, whoever wrote it, matters less than whether anyone can check that it was actually carried out.

The sharpest warning about doctrine comes from a benchmark of generative UI tools. When the tools were given design rationales to implement, about a quarter went unimplemented, rising to 34% for functional requirements. The tools recognized only half of the UX principles built into the prompts Do generative UI tools actually implement their stated design rationales?. That benchmark measured whether tools carry out instructions, not what vendors claim. Still, the lesson carries over to vendor rules: a principle that is stated is not the same as a principle that is followed. The researchers call this gap 'design theater.' Moran pushes the other way, arguing that generative UI is valuable because the AI decides what interface to build, since most users can't describe what they want Can generative UI serve users who cannot specify interfaces?. Put those two notes side by side and the tension is clear. The more design judgment is handed to the tool, the more it matters whether that judgment actually holds up.

Frameworks built from research have their own weakness: they don't move well between settings. One study of AI companion design found that recommendations borrowed from other areas clashed with the new setting. In companion apps, ethics and UX can't be separated, so a principle like 'engagement' or 'empathy' has to be weighed again for each user and situation Why don't design recommendations transfer cleanly to AI companions?. A review of 291 papers found six recurring dimensions of human–GenAI interaction design, including model feedback, user control, and how systems are evaluated How should designers structure human-GenAI interaction systems?. That is a map of where design choices get made. It doesn't tell you what to choose, which is why it stays useful across settings and also why it gives you no specific instructions.

Pattern libraries like the AI UX Playground sit between the two. Its six trust patterns (citations, visible chain-of-thought, a source browser, fine-grained consent, failure disclosure, data ownership) aren't justified by appeal to authority. They are justified because each one lets users check what the system is doing or take back control Do UI patterns for verifiability actually build user trust?. Lam and colleagues build the same idea into the system itself. Their 'just-in-time objectives' architecture shows the AI's design decisions at every interaction instead of fixing them ahead of time Does visible design structure keep HCI expertise necessary as AI advances?. Canvil, a Figma add-on, lets designers write and test the prompts that shape a model's behavior, bringing their user-centered judgment into the model without any engineering Can designers shape LLM behavior without deep technical knowledge?.

So the line that matters may not run between doctrine and research. It may run between fixed rules and judgment that can be inspected. AI context keeps shifting with each prompt, conversation history, and retrieved data, and users can't learn it the way they learn a stable interface How does AI context differ from conventional software context?. In that setting, any rulebook written in advance, from a vendor or from a study, goes stale faster than it would for traditional software. The approaches that seem to last make their design choices visible and testable at the moment they're used.