GhostWriter: Augmenting Collaborative Human-AI Writing Experiences Through Personalization and Agency

Large language models (LLMs) are becoming more prevalent and have found a ubiquitous use in providing different forms of writing assistance. However, LLM-powered writing systems can frustrate users due to their limited personalization and control, which can be exacerbated when users lack experience with prompt engineering. We see design as one way to address these challenges and introduce GhostWriter, an AI-enhanced writing design probe where users can exercise enhanced agency and personalization. GhostWriter leverages LLMs to learn the user’s intended writing style implicitly as they write, while allowing explicit teaching moments through manual style edits and annotations. We study 18 participants who use GhostWriter on two different writing tasks, observing that it helps users craft personalized text generations and empowers them by providing multiple ways to control the system’s writing style. From this study, we present insights regarding people’s relationship with AI-assisted writing and offer design recommendations for future work.

Introduction. With the rapid advances and increasing ubiquity of large language models (LLMs), there is growing interest in exploring their impressive text generation capabilities [6], particularly in the context of writing. Many systems have emerged that leverage LLMs to assist with various writing tasks, from brainstorming ideas and drafting content [25, 38, 56] to summarizing documents [11, 57, 58] and refining existing text [46, 56]. This example illustrates two core challenges that can arise when using LLMs for AI-assisted writing, which internal teams in our organization have consistently observed. First, the output can be too general, reflecting a lack of personal- ization in the generated text [9, 16, 28, 38, 56]. The LLM likely did not write in Sarah’s own writing style, making the output feel, to those who know Sarah and Sarah herself, as coming from somebody else and overly formal.

Discussion / Conclusion. In this work, we explore how we can design AI-assisted writing systems that allow control over personalization and champion user agency. Based on a set of research questions and design principles, we designed and implemented GhostWriter, which we use as a design probe to study the potential of LLMs in crafting personalized writing experiences through style and context. Our evaluation revealed that GhostWriter is effective in preserving user agency when interacting with a system with AI-fueled writing assistance. Participant feedback also illustrated the value in offering both implicit and explicit mechanisms to teach the system about one’s writing preferences. Guided by these findings, we present a series of design lessons to shape the future of collaborative human-AI writing. As a whole, we hope our work can inspire others looking to leverage LLMs to augment and complement human capabilities, providing a reference for exploring the new challenges and opportunities that arise when designing and using these emerging technologies.