A Design Space for Intelligent and Interactive Writing Assistants

In our era of rapid technological advancement, the research landscape for writing assistants has become increasingly fragmented across various research communities. We seek to address this challenge by proposing a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants. Through a large community collaboration, we explore five aspects of writing assistants: task, user, technology, interaction, and ecosystem. Within each aspect, we define dimensions (i.e., fundamental components of an aspect) and codes (i.e., potential options for each dimension) by systematically reviewing 115 papers. Our design space aims to offer researchers and designers a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants, and aid in the envisioning and design of new writing assistants.

Introduction. A writing assistant is a computational system that assists users with improving the quality and effectiveness of their writing, from grammar and spelling checks to idea generation, text restructuring, and stylistic improvement. In our current era of rapid technological advancement, however, the research landscape for writing assistants is becoming increasingly fragmented across various communities. While numerous writing assistants have emerged in recent years, quite disparate research communities like Natural Language Processing (NLP), Human-Computer Interaction (HCI), and Computational Social Science (CSS) study writing assistants with different emphases, such as model performance, user interaction, and social phenomena. There are even more specific areas like creativity support, second language acquisition, and disability studies, each of which may struggle to stay up to date with work happening across other communities. This splintering poses a significant challenge for researchers and designers seeking a holistic view, making it essential to bridge these gaps for effectively navigating the complexities of sociotechnical systems [145, 245]. In this paper, we contribute a design space to provide a structured way to explore the multidimensional space of intelligent and interactive writing assistants. Design spaces are taxonomies that present critical aspects of a design [32, 33, 166, 178, 214] with three main uses. The first is to establish a shared vocabulary that can help streamline communication and collaboration between researchers, designers, and other stakeholders. The second use case is to provide support for understanding existing designs. They allow one to reflect on why certain design choices were made, and why a given design may succeed in certain ways and fail in others. The third is to support envisioning new designs. By thinking about regions of a design space that may be “empty” (i.e., given a set of dimensions, no design considers all of them), we can think about what a design in that space would look like, and whether it might be worth pursuing. Through a large community collaboration, we create a design space based on five key aspects of writing assistants—task, user, technology, interaction, and ecosystem (Figure 1)—based on the sociotechnical systems perspective. Within each aspect, we identify dimensions (i.e., fundamental components of an aspect) and codes (i.e., potential options for each dimension) by systematically reviewing 115 papers and employing an iterative coding process. As a result, our design space contains 35 dimensions and 143 codes (Table 1–7), which includes writing contexts (e.g., academic, creative, and journalistic) and users’ relationships to a system (e.g., agency, ownership, trust, and privacy) that are closely related to interaction

Related work. Technology has had a significant impact on the way we write. Some technologies long precede the appearance of computers, such as clay tablets known as cuneiform writing [184, 219], and in the 19th and 20th century, typewriter technologies [194], each of which dramatically changed the way we produce written artifacts. Technical advances introduced computer-powered text-entering systems, which further developed into word processors. Word processors allowed users to flexibly edit texts with functions like deletion, copypaste, and find-and-replace [194]. These systems further evolved to support the writer’s cognitive process, or the distinctive thought processes the writers go through when composing writings [73, 101]. Examples of such support include providing text analysis [78, 164] and text-planning support [50] (see Appendix C for more examples). While supporting the cognitive process of writing can help people improve their writing proficiency and efficiency, how to do so consistently and persistently remains a challenge. For example, what kinds of support should we provide to the human writing process and with which interactions? How can we technically enable such support? And when should we “fade” this support as a temporary “scaffolding” [193] to encourage independent writing? One thread of work is understanding the human writing process, such as the cognitive process theory of writing [73, 100]. Another thread has been on extending the “intelligence” of writing assistants, so that these assistants can provide a wider range of support, such as those that require “understanding” of long and complicated texts. For example, for a writing assistant that provides suggestions on filling in between already written story events [62, 86], the system would need to grasp what happened in the preceding and subsequent events in order to coherently connect them. Fortunately, relevant technologies have advanced rapidly within the fields of NLP, where most recently, language models (LMs) [22, 185, 186] and their prompt- and example-based usage showed impressive capabilities in generating coherent text.

Integrating scientific knowledge of human writing and technical advances, researchers designed and built many intelligent and interactive writing assistants in recent years. Some of these aimed to support existing human writing tasks and domains, such as help request writing in professional settings [115], instant message writing in affectionate relationships [135], or tweetorial writing for scientific communication [85]. Researchers also studied how writers interact with these new writing assistants and how they influence human writing. For example, Buschek et al. [27] studied how generative writing suggestions can influence email writing behaviors.

Method. To develop a design space for intelligent and interactive writing assistants, we first decide on the scope (Section 3.1) and core aspects of writing assistants (Section 3.2). Then, we perform a systematic literature review while employing an iterative coding process and create the design space (Section 3.3, Figure 2). We highlight that this process was a collaborative effort consisting of a large team of researchers from a variety of disciplines, including HCI, NLP, Information Systems (IS), and Education.

We define the scope of our work by specifying what we consider as intelligent and interactive writing assistants. Note that these are working definitions that align with the objectives of our study, and are not intended to be imposed as a universally accepted definition.

• Intelligent: We consider systems to be intelligent if they are capable of autonomous decision-making and/or text generation, with a special focus on modern AI, such as language models (LMs). • Interactive: We consider systems to be interactive if they reflect human input and/or output (as opposed to generating text without human involvement) and facilitate an iterative process to produce a written artifact. • Writing: We consider systems to be relevant to writing if at some point human users translate their thoughts into written language, perhaps via system support, and produce text in a natural language (e.g., English) as a final artifact.

To have a stronger and tighter focus on intelligent and interactive writing assistants, we exclude the following types of work: programming tools (to focus on writing as a means to communicate with humans as opposed to machines), collaborative tools (to focus on the dynamics between a user and a system as opposed to multiple users), text-entry tools (to focus on tools that facilitate the cognitive process of writing, rather than input methods, such as gestural, hand-writing, or speech-to-text). Throughout the paper, we interchangeably use “interactive and intelligent writing assistants” and “writing assistants” to refer to the writing assistants that fall within our scope (see Appendix B for our definitions of similar terms like “technology,” “system,” and “model”).

3.2 Five Aspects of Writing Assistants Writing assistants are sociotechnical systems that involve interaction between both technical (e.g., AI and hardware infrastructure) and social (e.g., user behaviors and societal norms) components. The sociotechnical systems perspective is an interdisciplinary approach [19, 145, 217], widely used in various research fields like HCI [140, 151] and IS [39, 259] to analyze complex interactions involving technology, workflow design, and organizational changes. A core assumption of the sociotechnical systems perspective is that technology does not exist in a vacuum and is always interconnected with people and other systems. The perspective considers “task,” “user,” “technology,” and “structure” as integral and interdependent parts of a complex system [145]. We adopt this point of view with the following modifications: We first split “technology” into two—“technology” and “interaction”—to be more aligned with current research in NLP and HCI, respectively, and rename “structure” as “ecosystem” to account for the broader context in which writing assistants operate. As a result, we designate these five key aspects of writing assistants.

• Task: Writing stages, contexts, and purposes that writing assistants aim to support. This involves understanding the purpose of writing, the constraints imposed on writing, and the intended audience for the written content. • User: Characteristics and preferences of users of writing assistants, providing insights into how different user groups may prioritize different attributes in their interactions with these systems. • Technology: Building blocks of developing underlying models that power writing assistants, involving the quality and quantity of training data, modeling problems and techniques, and evaluation methods.

Discussion. In this section, we share use cases for our design space, our reflections on the current literature and ethical implications, the challenges and limitations we encountered in creating the design space, and our plans for future work.

In this section, we present two use case scenarios for our design space. These scenarios illustrate how the design space can be utilized (e.g., generative and analytical) and demonstrate its value for a range of stakeholders (e.g., researchers and policymakers). They also underscore the interdependencies and trade-offs between different dimensions and codes.

Generative scenario. Suppose a group of researchers aims to create a writing assistant, specifically designed to aid non-native English writers in choosing the best paraphrase among multiple paraphrases generated by AI.6 Motivated by previous work, the researchers plan to build a prototype of the writing assistant to improve the writing quality of the target user group by offering support features. Upon examining the design space, the researchers find they have already factored in numerous dimensions, such as the writing process [revision], writing context [academic], demographic profile [language & culture], system output preferences [explainability], and digital infrastructure [usability consistency]. However, they realize that they overlooked certain key dimensions like data - source and evaluation - focus of the foundation model they intend to use, as well as dimensions like relationship to system [trust, transparency] and interaction metaphor that could influence how the user perceives the system. These insights prompt them to do investigations into their options for foundation models that they may not have previously considered, take into account user concerns of trust and transparency, and think about various ways to frame and present the system to users. Overall, the researchers find that the design space ensures they do not overlook important design decisions, resulting in a richer and more thoughtful design.

Analytic scenario. Suppose a group of policymakers is concerned about the unintended consequences of AI-powered writing assistants swaying public opinions. This worry stems from a research report suggesting that co-writing with opinionated language models (LMs) can influence writers’ views.7 To gain a comprehensive understanding of the context from which these findings originate, the policymakers refer to the research paper as well as our design space. By mapping the writing assistant used in the paper to the design space, they gain a nuanced understanding of the experimental context (e.g., writing stage [drafting], specificity [general direction], and model - type [foundation model]). More importantly, they identify several factors that could potentially alter the findings. For example, the writing assistant in the study automatically provided suggestions to users (UI - initiative [system-initiated]), rather than allowing users to request suggestions when needed. Furthermore, users had no way to explicitly control or guide the system’s output, and had limited implicit control; even though the underlying model took user text as input to account for the user’s writing style and existing contents, it was consistently prompted by the system to output text in favor of a pre-determined stance on the topic that the user was asked to write about (user - steering the system [implicit]). Recognizing these factors, the policymakers realize they could potentially introduce regulations to make user-initiated interactions mandatory and to allow users to explicitly steer the system’s output. They could also recommend that designers visually differentiate between user-generated and AI-generated text (UI - visual differentiation).

Conclusion. In this work, we present a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants. Through community collaboration and systematic literature review across multiple disciplines, we define 35 dimensions and 143 codes exploring five key aspects of writing assistants. We hope that this design space provides researchers and designers with a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants, and aids them in the design of new writing assistants.

Limitations. One limitation of our work is the coverage of sampled and filtered papers, restricted by the search criteria. Using “write” in titles or keywords may not be particularly suitable for NLP papers. While expanding the search to include abstracts would increase the corpus, this approach could also lead to a diminishing return rate. Consequently, the focus remained on developing useful dimensions and codes, rather than striving for an exhaustive collection of papers. Another limitation was not explicitly including commercial writing assistants (without research papers) in our search. In our coding process, it is possible that we may have misinterpreted authors’ intentions, which may have resulted in errors in our annotation. Despite these limitations, we argue that this design space serves as a reference for examining existing assistants and developing new ones, while preventing implicit assumptions or overlooked considerations, thereby facilitating a holistic understanding of the factors that drive design considerations and choices. For future work, we hope to continue to refine our dimensions and codes and code commercial writing assistants to further understand the gaps and opportunities in the current research landscape and suggest possible directions for future research and development.