How should UI widgets coexist with conversational interfaces?

OpenAI's Apps SDK design guidelines state that third-party app UI embedded in ChatGPT should "extend what users can do without breaking the flow of conversation," and that developers should "use cards, carousels, fullscreen views, and other display modes only when visual interaction improves the workflow." The guidelines name three such display modes, each tied to a specific interaction type: inline cards are "lightweight, single-purpose widgets embedded directly in conversation" for "quick confirmations, basic actions, or visual aids," appearing before the model's generated response, with primary actions limited to two actions, placed at bottom of card; fullscreen is an "immersive" surface for "multi-step workflows or deeper exploration"; and picture-in-picture (PiP) is "a persistent floating window inside ChatGPT optimized for ongoing or live sessions like games or videos."

The guidelines' reasoning is that the conversational composer should never disappear, regardless of which display mode is active. In fullscreen, "the ChatGPT composer remains overlaid, allowing users to continue 'talking to the app' through natural conversation," and the guidance to developers is to "design your UX to work with the system composer" since it "is always present in fullscreen." In PiP, the same logic applies to live state: the window "can update dynamically in response to user prompts," so a developer must "ensure the PiP state can update or respond when users interact through the system composer." Across all three modes the card or widget is explicitly secondary to the conversation — a short "follow-up" response may suggest "edits, next steps, or related actions" after a widget, but developers are told to "avoid content that is redundant with the card."

This gives a concrete, shipped counterpart to the GUI-fallback layer How should AI interfaces handle the shift from doing to supervising? proposes in the abstract: where Nielsen's Direct-Manipulation Surface is reserved for edge-case editing, granular corrections, and emergency overrides, OpenAI's inline/fullscreen/PiP taxonomy is a production design space for when and how that manual surface should appear without displacing the conversational intent surface. It also narrows the debate behind Do generated interfaces outperform text-based chat for most tasks? and Do full web pages beat markdown chat for LLM responses?: those findings weigh generated UI against chat as competing response formats, while OpenAI's guidelines treat UI and chat as concurrent, not competing — the composer and the widget coexist in the same screen rather than one replacing the other.

The source is a vendor design-guidelines page, not an empirical study: it states principles and named modes without usability data, adoption figures, or named apps that measure whether composer-overlay actually prevents users from feeling the conversation was interrupted. It also does not say how developers should choose among the three modes when a task's workflow depth is ambiguous, beyond the general instruction to use visual interaction "only when" it "improves the workflow." The guidelines establish a vocabulary and a design constraint (composer always present) but not evidence that the constraint improves outcomes over a design where visual UI can fully occlude conversation.