Prototyping with Prompts: Emerging Approaches and Challenges in Generative AI Design for Collaborative Software Teams

Generative AI models are increasingly being integrated into human task workflows, enabling the production of expressive content across a wide range of contexts. Unlike traditional human-AI design methods, the new approach to designing generative capabilities focuses heavily on prompt engineering strategies. This shift requires a deeper understanding of how collaborative software teams establish and apply design guidelines, iteratively prototype prompts, and evaluate them to achieve specific outcomes. To explore these dynamics, we conducted design studies with 39 industry professionals, including UX designers, AI engineers, and product managers. Our findings highlight emerging practices and role shifts in AI system prototyping among multistakeholder teams. We observe various prompting and prototyping strategies, highlighting the pivotal role of to-be-generated content characteristics in enabling rapid, iterative prototyping with generative AI. By identifying associated challenges, such as the limited model interpretability and overfitting the design to specific example content, we outline considerations for generative AI prototyping.

Introduction. The shift from predictive AI to generative AI, driven by advances in large language models (LLMs) [73], is transforming how we design and prototype AI applications. Unlike traditional AI development, where task-specific models are fine-tuned and trained with narrowly defined objectives, generative AI enables teams to rapidly experiment with pre-trained models with more generalized capabilities across tasks using prompt engineering. This approach empowers multidisciplinary teams — composed of user researchers, designers, product managers, and machine learning engineers — to quickly assess model capabilities, explore design possibilities, and iterate on solutions before committing to full-scale software development. Prompt engineering plays a pivotal role in this new workflow, allowing teams to shape AI outputs by crafting effective input prompts. This ability to experiment quickly and interactively with AI models introduces a more dynamic design process. User researchers, for example, can use generative AI to generate diverse user personas [25], while designers can rapidly prototype user interfaces and create multiple design variations [16]. Product managers can explore feature ideation and market scenarios using AI-powered analysis [32], and machine learning engineers can generate code snippets or debug algorithms by interacting with AI models in real time [33]. However, the adoption of generative AI also introduces new challenges to design workflows [35, 51]. While there is more flexibility in prototyping model outputs, there is a reduced degree of freedom in shaping the underlying AI systems, as most teams must work within the constraints of pre-trained foundational models [9, 57]. For designers prototyping with pre-trained models, they need to develop a designerly understanding of the pre-trained model [44], find effective ways to support user interactions by minimizing the gulfs of execution, evaluation, and envisioning [30, 65], and build correspondence between interfaces, prompts, and underlying generative AI capabilities [53]. Further prompt engineering has its own challenges for software teams and end-users [87, 88], and strategies for effectively and collaboratively designing prompts are largely lacking. While new tools for prompt engineering and prototyping, such as AI Studio [22], ChainForge [4], and Canvil [16] are being developed, we lack an understanding of how to prototype generative AI applications with them effectively. Specifically, in collaborative software teams, this new approach to design prototyping can be a complex process. The varying degree of freedom and the critical role of prompt engineering necessitates a collaborative approach to these tasks to ensure the prompt instructions and model fine-tuning are aligned with the user interfaces, task workflows, and contexts. Furthermore, the process of iteratively designing and evaluating choices during prompt prototyping remains unclear. Given these shifts in both the nature of AI design and the collaborative processes involved in prototyping, this paper explores how prompt engineering can be effectively used as a tool for rapid iteration and experimentation in generative AI application design. Specifically, we investigate the following questions:

(1) How do teams approach the iterative prompt-based prototyping and evaluation of generative AI applications? (2) How do different roles within collaborative teams contribute to the prototyping process of generative AI features? (3) What unique challenges arise in the prototyping of generative AI applications, and how do they differ from prior AI design approaches?

We conducted a design study with 39 industry practitioners from the United States and India across 13 sessions. In each session, a UX designer, an engineer, and a product manager collaboratively worked on a given generative AI design problem and iteratively prototyped potential solutions using the AI Studio [22] prototyping tool. We observed that content– both example (ground truth) content and the generated content–played a central role in the iterative design-test-analyze prototyping process, which we call content-centric prototyping. More specifically, we observed a range of strategies in approaching prompt prototyping from templates to agent personas, how teams collaboratively engaged in divergent and convergent thinking of formulating instructions and curating few-shot examples, and how different roles contributed to the design and evaluation tasks. We also identified key challenges to prototyping due to the high sensitivity of generative models to prompt phrases, lack of interpretability, and other human-introduced challenges, including overfitting designs to specific content examples and potential transparency issues for end-users that may be hidden at the prompt instruction level.

Related work. The shift from designing applications for task-specific ML models to generative AI models introduces new layers of complexity and flexibility to the human-centered AI design process [87]. Task-specific models are traditionally engineered through a deliberate process of defining data and model behavior, and designers have some level of control in aligning those specifications for target user needs and interactions that are direct and predictable [67]. In contrast, generative AI, with its broader scope for content creation and autonomous decision-making, demands a more dynamic iterative design process with inherent uncertainties in design decision-making. More specifically, when prototyping generative AI applications, software teams have to balance designing in ways that harness the model’s creative potential while contending with the reduced determinacy in how these models interpret and respond to user inputs, ultimately impacting the predictability of user interactions and the fidelity of generated content to user expectations [65]. Here, we characterize this shift in design prototyping and identify knowledge gaps by integrating insights from previous studies on AI prototyping, prompt engineering in generative AI, and the roles of expertise and collaboration in implementing human-centered AI strategies.

2.1 Prototyping AI Experiences In recent years, work across academia and industry has formulated design guidelines for task-specific ML models including designing interactions [2, 20, 27], explainability [75], privacy [26, 36], transparency [7, 29, 43], etc. To operationalize these guidelines, prior research has considered how software practitioners work with AI as a design material [44, 71, 82], and has developed processes [67] and tools [50, 66] to prototype application experiences with AI models. Across this body of work, the basis for prototyping is that UX teams need to first understand the capabilities and limitations of ML models, including the data the model has been trained on, discerning the behaviors the model has adopted from that data, comprehending the relationships between inputs and outputs, and identify uncertainties and edge cases to the model behavior [14, 15, 83, 84]. Further, to facilitate this understanding, approaches such as model-informed prototyping [66] and failure-driven design [50] support opening up the ML black box and allow designers to engage in model sensemaking using real data and model outputs that ultimately inform their design choices in prototyping. Other work has looked at deeper collaboration with engineering teams who understand ML to make design and prototyping more efficient [64, 67].

Method. As outlined in the research questions, our objective was to gain a deeper understanding of collaborative prototyping strategies deployed by multistakeholder teams comprising User Experience Researchers / User Experience Designers ( UXR ), Product Managers ( PM ), and Software Developers ( SWE ). To this end, we conducted multi-site design workshops (n=13) with thirty-nine participants and observed their experiences in using a Generative AI prototyping tool (AI Studio [22]) to design an application for marketing content creation. We selected this domain because of general exposure and familiarity with marketing content across a wide range of demographics, ensuring a diverse set of perspectives and creative inputs. Consequently, the problem itself embodies the challenge of capturing and engaging diverse audiences with compelling narratives that require both creative and strategic thinking. We conducted the study over a three-month period from June to August 2023. Each design session involved three stakeholders – PMs, SDEs, and UXRs and one of the authors to facilitate the sessions. The sessions lasted 105 minutes each and were conducted over Google Meet (video conferencing software). We obtained approval for this research from the Institutional Review Board of an anonymized US university. All participants signed standard consent forms. Given the observational and experimental nature of our study, our data collection involved a combination of handwritten notes and video recordings. Each design workshop was video recorded with consent. The participants are identified using a combination of their role and group number to enhance clarity in the findings. For example, ‘PM1’ denotes the product manager in Group 1.

3.2 Tool Description: AI Studio Our study used AI Studio [22] (formerly AI Studio) to investigate the prototyping process with generative models. AI Studio is a browser-based IDE that enables rapid prototyping with generative models, utilizing a version of the large language model described in [23]. AI Studio offers two interfaces for prompting the language model: (1) Freeform and Chat Prompts, which allow natural language interaction for building conversational experiences, and (2) Structured Prompts, where users can specify example requests and expected replies for more controlled outcomes. Participants in our study primarily focused on structured prompts due to their higher control over the model’s output, though they occasionally utilized freeform and chat prompts. The typical workflow involved crafting a prompt to achieve a specific objective, running the model by clicking the "Run" button, and reviewing the output. Participants could then iterate on their prompts by refining the wording, adding more test examples, or adjusting the temperature parameter, which ranges from 0 to 1 and controls the randomness of the generated text. This allowed participants to fine-tune the creativity of the model’s responses.

3.3 Study Protocol We anchored our design activity on a real-world application of Generative AI in an organizational setting – marketing content creation (detailed in Section 3.3.1). We used existing professional content creation and copyright workflows based on information on the web and one of the author’s own experiences as a starting point to develop instructions for the protocol. We identified introductory topics to guide participants toward common goals. The questions corresponded to (1) Product Goals, (2) User Personas, (3) Prompt Structure, (4) Expected Outputs, and (5) Guardrails. However, these were intended as suggested thinking tools and we did not mandate their usage while running the design activity. Participants used AI Studio [22], to prototype the application. We should note that our intent is not to assess the tool itself. AI Studio is one of the first publicly available prototyping tools for generative AI applications. In addition to the conversational affordances of tools such as ChatGPT, AI Studio has specific interface features to provide fewshot examples in a tabular format and also design conversational interactions.

Discussion. 5.1 Prompt Design We observed several unique considerations in the prototyping process. First, the iterative prompt design process resembles the process of curating ‘gold examples’ (high-quality input-output pairs that improve model performance). Our study provides an initial insight into the curation process of such gold examples in the prototyping process. Participants engaged in strategies in ‘ablations’, removing parts of the prompt to study model behavior and acquiring an overview of model behavior through convergent and divergent thinking. Second, the design choices that our participant teams made during the prototyping process were guided by the control knobs they had access to – curating gold examples, prompt refinement, model parameters (eg: temperature). Our study showcases the opportunity for designers to support decision-making across such knobs.

Based on how teams authored the prompts, we identified four main sub-components of a prompt that include (1) the input context, (2) the system instructions, (3) the output constraints, and (4) few-shot examples (Figure 4). The input context provides the generative model with the necessary background or situational context to understand the task at hand. It encompasses specific details that the end-user might provide when using the prompt to tailor the generated response. The system instructions are commands or requests detailing what the model is expected to do and specify the task, action, or outputs desired from the AI. The output constraints provide guidelines or boundaries for the generated response. In our sessions, these constraints included stylistic aspects such as format and length of response as well as high-level guardrails, e.g., asking the model not to generate harmful content. Finally, the few-shot examples provided the model with illustrative content that implicitly indicated the desired nature of the output or behavior expected of the generative model. In addition to the ‘anatomy’ of a prompt, our findings revealed how teams strategically approached thinking about these components and how different roles contributed to the prompt components. Designers who have expertise in crafting interfaces and designing input/output flows found themselves particularly drawn to utilizing prompts as a metaphorical ‘template’ during the prototyping process. Conversely, certain design participants encountered challenges in understanding the concept of fewshot examples within prompts, a technique fundamentally rooted in engineering principles. Given the interconnected nature of these sub-components, and as shown in our findings– prototyping involving refining information across the various sub-components–led to blurred lines between roles and design representations. This merging of roles and responsibilities has obscured individual agency and ownership, challenging the conventional division of tasks, such as modular design principles within project teams. Further, in identifying issues with separation of concerns, Subramonyam et al. advocate for the concept of “leaky abstractions” which allow for some overlap across boundaries while preserving distinct design artifacts specific to each role [64]. However, in the context of generative AI prototyping, these separations become even more indistinct, with shared artifacts and process integration further blurring the lines between different roles. It remains an open question how this evolving dynamic will impact the definition of roles, the nature of design representations, and the methodologies employed in prototyping generative AI applications. Future research should look at how distinct roles and expertise can effectively contribute to crafting instructions for generative AI.

Conclusion. Generative AI models, as a design material, offer dynamic capabilities but also constrain software teams working with pre-trained models. In this study, we explored how collaborative teams prototype generative AI applications through prompt engineering, requiring alignment with human-centered values while addressing diverse user needs. Our findings revealed a shift toward contentcentric prototyping, where design and computation are intertwined, and prompts become essential tools for shaping both AI behavior and user experience. However, challenges arose, such as implicit assumptions about user behaviors, overfitting models to narrow content, and the sensitivity of generative AI to prompt variations. These issues highlight the risks of exclusion and misalignment between system design and user expectations. We also observed the ethical concerns of using public content for prototyping, emphasizing the need for guidelines. The shifts in prototyping practices call for a broader exploration of design space and new evaluation frameworks that balance both technical feasibility and user experience in generative AI applications.

Limitations. 5.5 Limitations Given the novelty of generative AI in software products and constraints to gathering process-related data in real-world teams, we focused on a controlled design study with a pre-determined application context, i.e., marketing content creation. While this setup allowed us to glean insights into prompt prototyping and collaboration, it may not be representative of real-world teams. The complexity and variability inherent in different projects, industries, and team structures are likely to influence the process and outcomes of generative AI design in ways not fully explored in our study. Second, we did not include domain experts, in our case, marketing content creators, copyright specialists, etc.