Show Me Your Prompts! How Writers Feel About Sharing Prompts in Collaborative Text Editors

Generative AI writing assistants are becoming integrated into collaborative text editors; however, it is unclear how much information about a user’s prompting activities should be shared with collaborators. We explore the effects of different levels of prompt information sharing within collaborative text editors: not sharing anything, sharing a placeholder to indicate AI use; sharing details about how the resulting text was generated; and sharing everything, including how the prompt was formulated, in real-time. Sixteen participants wrote persuasive essays in pairs using all four techniques. Results suggest a strong preference for techniques that share more information about prompting activities for increased awareness. Our work shows that collaborative text editors should share more information among writers on when, how, and where AI is used.

Introduction. Different writing activities, such as cursor movements, text selections, and text edits, are immediately shared with others in collaborative text editors like Google Docs and Overleaf. This can improve awareness and collaboration [5], but it can also distract writers and make them feel self-conscious [13–15, 22]. Many people use large language models (LLMs) when writing, for example, to brainstorm, change the tone of an e-mail, or proofread a paper. This is typically done by writing prompts in isolation using systems like ChatGPT and pasting the output into a text editor. More recently, LLMs have become integrated into collaborative text editors [12, 17, 18], allowing writers to prompt, and insert generated text into a shared document without changing interfaces. However, these systems vary greatly in how much prompting activity is shared with collaborators. Despite being collaborative, text editors like Google Docs and Overleaf do not share details of a writer’s prompting activities, such as the prompt that was issued and where it was used [12, 18].

Discussion / Conclusion. In summary, our results suggest that participants preferred techniques that shared more prompting activities. Participants noted multiple perceived benefits related to increased awareness that came from sharing more information. Specifically, they wanted to know when, how, and where AI was being used. Participants could more easily tell what their collaborators were doing when they knew when prompts were being formed in realtime. Sharing the prompt and original text selection gave insights into how the resulting text was formed. This helped participants understand each other’s thought process more, built trust, and provided opportunities to learn from each other. Leaving a comment signalled where AI was used, which was important as participants wanted to indicate and know parts of the document that needed additional verification. That said, there were some perceived cons to sharing more information about prompting activities. Some participants noted that formulating a prompt should be “private,” felt uncomfortable when everything was shared, and worried the quality of their prompts would be judged. Having awareness of other’s prompts may have influenced each other’s thinking in ways they did not appreciate and found distracting and annoying.