Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platforms

Abstract—The widespread adoption of conversational AI platforms has introduced new security and privacy risks. While these risks and their mitigation strategies have been extensively researched from a technical perspective, users’ perceptions of these platforms’ security and privacy remain largely unexplored. In this paper, we conduct a large-scale analysis of over 2.5M user posts from the r/ChatGPT Reddit community to understand users’ security and privacy concerns and attitudes toward conversational AI platforms. Our qualitative analysis reveals that users are concerned about each stage of the data lifecycle (i.e., collection, usage, and retention). They seek mitigations for security vulnerabilities, compliance with privacy regulations, and greater transparency and control in data handling. We also find that users exhibit varied behaviors and preferences when interacting with these platforms. Some users proactively safeguard their data and adjust privacy settings, while others prioritize convenience over privacy risks, dismissing privacy concerns in favor of benefits, or feel resigned to inevitable data sharing. Through qualitative content and regression analysis, we discover that users’ concerns evolve over time with the evolving AI landscape and are influenced by technological developments and major events. Based on our findings, we provide recommendations for users, platforms, enterprises, and policymakers to enhance transparency, improve data controls, and increase user trust and adoption.

Introduction. Large language models (LLMs) [1], [2] have revolutionized artificial intelligence (AI), driving the widespread adoption of conversational AI platforms (e.g., ChatGPT [3], Gemini [4], and Claude [5]). These platforms allow users to interact with LLM-based AI systems through different modalities, such as text and voice. They are being integrated into a diverse range of applications, including customer service [6], [7], [8], personal assistants [9], [10], healthcare [11], [12], and finance [13], [14], due to their advanced natural language abilities, enabling more intuitive user interactions. However, as conversational AI platforms become more popular, concerns surrounding their security and privacy (S&P) have also intensified. These platforms present unique S&P risks as their human-like conversational abilities can inadvertently encourage users to share sensitive information more freely than with traditional interfaces [15]. Prior work has also demonstrated that LLMs can memorize and reproduce sensitive information from their training data in response to malicious prompts [16], [17]. High-profile data leaks [18], corporate restrictions on platform usage due to suspected sensitive data exposures [19], [20], [21], and the drive for regulations (e.g., the White House Executive Order on AI adoption [22]) further highlight the critical nature of these risks. While prior research has predominantly focused on the technical aspects of securing LLMs [23], [24], [25], [26], and in turn these platforms, there is a notable gap in understanding how users perceive and respond to these S&P risks. Preliminary studies have leveraged mixed methods (e.g., surveys, semi-structured interviews) to investigate users’ S&P concerns about conversational AI platforms [27], [28]. These studies provide valuable initial insights into privacy harms and risks associated with AI systems, including common types of personal information users disclose and factors that influence trust in these platforms. However, these studies are conducted in controlled contexts with limited participant diversity and capture user concerns at a single point in time. Consequently, a comprehensive understanding of users’ S&P concerns, behaviors, and preferences in interactions with conversational AI platforms across diverse user bases and how they change over time is needed. To address this gap, we complement prior works by conducting a large-scale analysis of real-world, organic discussions on Reddit [29] to investigate the following research questions:

• RQ1 What are users’ S&P concerns related to conversational AI platforms?

• RQ2 What are users’ S&P attitudes toward conversational AI platforms?

• RQ3 How do users’ S&P concerns evolve over time, and how do major events in the AI ecosystem influence users’ S&P concerns and attitudes?

To answer these questions, we collect and analyze a dataset comprising ∼2.5M posts (∼180K submissions and ∼2.35M comments) from r/ChatGPT, the largest subreddit dedicated to conversational AI platforms, with over 7.4M members. Using keyword filtering and stratified sampling, we select and manually annotate a subset of 1, 200 posts from the entire corpus to fine-tune a RoBERTa-based binary classifier [30] for identifying S&P-related content. Applying this classifier to the entire dataset yields 30,240 S&P-related posts. We iteratively sample and code a subset of these posts through qualitative analysis to develop a codebook. We then leverage an LLM-based multi-class classifier for post-hoc data labeling, assigning thematic codes to the entire S&P dataset. Finally, we conduct an interrupted time series regression analysis [31] to investigate how users’ S&P concerns evolve over time, specifically in response to major events in the AI ecosystem. Our qualitative and quantitative analysis reveals that users are primarily concerned about what personal and proprietary data these platforms collect and why (43.7%). They also worry about how the collected data is used (22.5%), particularly for training models or third-party sharing, and its retention (9.5%). Users also seek assurance in platform security and security of apps integrating conversational AI platforms (28.9%), compliance with privacy laws (9.6%), and desire transparent data handling practices and clear control over their data (11.1%). We observe that users exhibit varied S&P behaviors and preferences when interacting with conversational AI platforms. Some are proactive in safeguarding their data, adjusting privacy settings, and limiting the information they share. Others are inquisitive, seeking information about how their data is collected, used, and stored. Some prioritize convenience and the benefits these platforms offer, often downplaying or dismissing privacy risks. Meanwhile, others feel resigned to data sharing as an unavoidable aspect of modern digital interactions.

Related work. Users’ S&P Concerns and Attitudes. Prior research has explored users’ security and privacy (S&P) concerns [34], behaviors [35], preferences [36], [37], and attitudes [38]. Studies have also explored users’ S&P attitudes and concerns toward specific platforms and technologies. For example, studies on smart homes and IoT devices have highlighted user apprehensions about data collection and device security [39], [40], [41], [42], [43]. Similarly, studies on web browsers and online social networks have investigated users’ views and satisfaction with privacy settings, uncovering a range of concerns about data sharing and control [44]. While prior research has examined general S&P attitudes or concerns toward specific technologies, we focus on the unique challenges and concerns that arise from the human-like interactions facilitated by conversational AI. Online Discussions on S&P. Online discussion platforms (e.g., Reddit, X, StackOverflow) offer spaces for open dialogue, mutual learning, and peer support, providing rich, real-world data that reflects user concerns and behaviors. Researchers have leveraged online discussions to study S&P from different perspectives [45], [46], [47], [48], [39], [49], [50], [51]. For example, Li et al. [39] analyzed Reddit posts to investigate users’ S&P considerations when adopting smart home technologies. Similarly, Tahaei et al. [47] conducted a qualitative analysis of privacy-related discussions on Stack Overflow to explore privacy challenges faced by developers. Wei et al. [46] studied user perceptions of targeted advertising by analyzing Twitter data. Several works have also investigated user privacy feedback by analyzing app reviews on app stores (e.g., Google Play Store) [52], [53], [54], [55], [56]. These studies illustrate the potential of online discourse to surface concerns and behaviors not easily captured through traditional surveys or interviews. Our work builds on this approach by analyzing discussions on the r/ChatGPT subreddit to capture nuanced and evolving user S&P perspectives unique to conversational AI platforms. S&P in Conversational AI Platforms. Large Language Models (LLMs) [1], [2] are increasingly being leveraged to power AI systems that engage users in natural language across multiple modalities (e.g., text and voice). We refer to these AI systems as conversational AI platforms. These platforms vary in complexity, ranging from basic chatbots to intelligent agents with task completion capabilities, and have unique S&P challenges. LLMs’ human-like conversational abilities can encourage users to share sensitive information more freely than with traditional interfaces [15].

Method. To understand users’ security and privacy (S&P) concerns and attitudes toward conversational AI platforms, we conduct a multi-stage analysis of discussions on Reddit [29]. Reddit is an online discussion forum organized into topic-specific communities called subreddits, where users share, discuss, and vote on content in threaded structures. Each thread consists of a submission (the original post) and a series of comments. Reddit discussions represent large-scale, organic user-generated data that offer valuable insights into users’ concerns, perceptions, behaviors, and information-seeking practices in the real-world [39], [49], [50]. Figure 1 presents an overview of our methodology, which involves four main steps. We begin by collecting a large dataset of Reddit posts and annotating a subsample to train a classifier that identifies content related to S&P 1 . Next, we conduct a qualitative analysis to identify users’ S&P concerns and attitudes towards conversational AI platforms 2 . Building on this, we leverage an LLM-based multi-class classifier to label all S&P-related posts with thematic codes derived from our qualitative analysis 3 . Finally, we perform a regression analysis to investigate how users’ S&P concerns shift in response to major events in the AI ecosystem 4 .

To understand users’ concerns and attitudes, we first compiled a list of popular subreddits related to conversational AI platforms using Reddit’s search engine [29], as shown in Appendix A Table 1. From this list, we selected r/ChatGPT for our analysis based on several reasons. First, it is the largest and most active subreddit focused on conversational AI with 7.4M members - more than 4 times the size of the next largest subreddit (r/OpenAI) - and ranks 86th among all Reddit communities. Second, it has high user engagement and activity (∼300 daily posts and ∼4,000 comments). Third, despite its branding, r/ChatGPT regularly features discussions that extend beyond ChatGPT. For example, over 7,500 posts mention local LLMs (e.g., LLaMA [61]), and more than 43,000 reference other commercial platforms like Gemini [4] and Claude [5]), indicating a broader topical scope. Lastly, our qualitative analysis shows that the subreddit hosts diverse user attitudes (Section 5), ranging from cautious and curious to privacy-dismissive and resigned. Therefore, selecting r/ChatGPT allows us to capture substantial data covering diverse S&P concerns across different conversational AI platforms. To conduct our analysis, we gathered all posts from r/ChatGPT from its inception in December 2022 till July 2024, using publicly available Reddit dumps [62]. After removing deleted posts, bot-generated content 1, and duplicate entries, our final dataset included ∼2.5M posts (∼180K submissions and ∼2.35M comments, collectively termed “posts”). Given the dataset size, identifying S&P-related discussion threads is a significant challenge. An intuitive approach is to perform a search with keywords associated with S&P to surface relevant posts [49]. However, this approach can suffer from low coverage, missing relevant posts that use alternative phrasing, and high false positives, especially in communities like r/ChatGPT where users frequently share generative content (e.g., fictional stories, essays, or poems) that may include keywords in unrelated contexts. To address these limitations, we adopt a hybrid approach combining keyword filtering with machine learning. Keyword Filtering and Sampling. We began by reviewing the literature on S&P issues in conversational AI and AI risk taxonomies [16], [27], [28], [64], [65], [66], [67], [68], [69] to compile an initial list of candidate keywords. We refined this list iteratively by searching Reddit posts, reviewing matched results, identifying new keywords, and updating the list accordingly. Using this iterative approach, we compiled a list of 118 S&P-related keywords and expressions, presented in Appendix B Table 3. To ensure balanced representation and reduce bias toward frequently mentioned keywords, we grouped our keywords into 6 thematic categories based on prior works [66], [70], [71].

Discussion. Our analysis of Reddit posts from the r/ChatGPT community provides valuable insights into users’ security and privacy (S&P) concerns toward conversational AI platforms. These findings reveal parallels with concerns about other emerging technologies while highlighting unique challenges posed by conversational AI platforms. In this section, we discuss the implications of our findings, situate them within the broader context of technology adoption and privacy concerns, and provide recommendations for key stakeholders.

Although users’ concerns and attitudes toward conversational AI platforms mirror those of other computing platforms, such as smartphones and IoT devices, we identify unique factors inherent to conversational AI platforms that escalate or reshape these concerns. Moreover, we find that users’ S&P concerns evolve over time, influenced by platform updates, new features, and reported security incidents.

7.1.1. Parallels with Other Technologies. Our study shows how users worry about extensive data collection practices, echoing longstanding concerns in the S&P community. Several of the S&P concerns identified in our analysis parallel those found in other computing platforms and emerging technologies. For instance, concerns about personal data collection, such as PII, location data, and activity tracking, are not unique to AI platforms [96], [97]. Similarly, the unease over data sharing with third parties, the potential misuse of data for targeted advertising, and inadequate transparency and control mechanisms are common themes in S&P research on devices such as smartphones [98] and IoT devices [99] and platforms such as social media [100] and VR [101].

7.1.2. Concerns Unique to Conversational AI Platforms. Despite these similarities, conversational AI platforms present critical differences that exacerbate users’ S&P concerns, posing obstacles to secure adoption. First, the conversational format and LLMs’ ability to emulate human empathy often lead users to disclose more sensitive or personal information than they would on other platforms (e.g., confidential business plans, proprietary code, or deeply personal struggles). Second, conversational AI platforms use user data for model training, which can lead to the inadvertent memorization of sensitive information by AI models, a phenomenon distinct from traditional data-storage models. Memorized data can be regurgitated during inference, raising ethical and regulatory questions about a “right to be forgotten”. Lastly, conversational AI has seen an unprecedented and rapid scale of adoption. This widespread popularity has driven its integration into high-stakes sectors such as finance, healthcare, and enterprise services, where sensitive data and critical operations are involved. In these contexts, the natural language interface becomes a gateway to high-stakes data, creating novel security risks like prompt injections or jailbreaks that can expose secrets or allow unauthorized access to sensitive functionality.

7.1.4. Evolution of Users’ Concerns. Our findings indicate that users’ S&P concerns evolve over time, often in response to new features and reported incidents. Initially, many users express excitement about conversational AI platforms’ capabilities but simultaneously exhibit caution due to underlying S&P concerns. We find that the introduction of new functionalities (e.g., custom GPTs), tends to reignite discussions about privacy, primarily due to a lack of transparency and detailed information about these features. Moreover, users’ concerns are exacerbated by the perceived inadequacy of transparency and control mechanisms. Confusion over privacy settings, policy changes, and data control options contributes to a sense of mistrust.

Conclusion. We conduct a large-scale analysis of online user posts to study users’ S&P concerns and attitudes toward conversational AI platforms. Our qualitative analysis shows that users are concerned with all stages of the data lifecycle—collection, usage, and retention—and seek better security, regulatory compliance, transparency, and control over their data. We also highlight how users’ concerns evolve over time in response to major events. Based on our findings, we provide recommendations for different stakeholders involved.

Limitations. 7.1.3. Validity of User Concerns. As our analysis is based on user-posted content from Reddit, many concerns reflect users’ perceptions of platform behavior. While these concerns are not always technically accurate, they offer valuable insights into user sentiment and potential trust or usability issues. We validate user concerns by analyzing platform documentation, feature updates, privacy policies (e.g., OpenAI’s), and existing literature, categorizing them as presently valid concerns, resolved issues, or misconceptions. Presently Valid Concerns. We find that several concerns raised by users remain pressing and unresolved. For instance, LLM-generated code has been shown to contain security vulnerabilities, such as susceptibility to SQL injection and crosssite scripting attacks [102].