Futuring Social Assemblages: How Enmeshing AIs into Social Life Challenges the Individual and the Interpersonal

Recent advances in AI are integrating AI into the fabric of human social life, creating transformative, co-shaping relationships between humans and AI. This trend makes it urgent to investigate how these systems, in turn, shape their users. We conducted a threephase design study with 24 participants to explore this dynamic. Our findings reveal critical tensions: (1) social AI often exacerbates the very interpersonal problems it is designed to mitigate; (2) it introduces nuanced privacy harms for secondary users inadvertently involved in AI-mediated social interactions; and (3) it can threaten the primary user’s personal agency and identity. We argue these tensions expose a problematic tendency in the user-centered paradigm, which often prioritizes immediate user experience at the expense of core human values like interpersonal ethics and selfefficacy. We call for a paradigm shift toward a more provocative and relational design perspective that foregrounds long-term social and personal consequences.

Introduction. “We make our technologies, and they, in turn, shape us. So, of every technology we must ask, Does it serve our human purposes?” — Sherry Turkle, Alone Together: Why We Expect More From Technology and Less From Each Other [169] The rapid integration of AI into most aspects of human life, from birth [113, 173] to death [24, 137], from personal reflection [62, 103, 104, 122] to juridical decision-making [74, 105], demands critical examination. The symbiotic relationships between humans and the things they create raise cautions about our design choices of AI systems. The stakes are particularly high, as technology has long been constitutive of humanity, i.e., we are not who we are today without the technology we use [171, 172]. In other words, by designing the AI for our lives, we are actively designing future user-AI assemblages, where AIs and users form newfound entities, and they iteratively co-adapt, transforming what it means to be human. Such assemblages’ evolution is gradually revealed in many fields, presenting challenges and opportunities for future design [1, 28, 100, 109]. Studies have raised concerns about human autonomy [1, 109], homogenization [100], value shifts [19], and deskilling in human-AI assemblages at both personal and societal levels [28]. Empirical studies have predominantly explored the effect of such assemblages in task-oriented scenarios with well-defined metrics, such as efficiency [67, 147], and working productivity and quality [44]. Far less understood are the dynamics of assemblages in social scenarios, where fundamentally different forms of intelligence are required [82, 146]. Unlike other tasks, social interactions are more relational and subject to an individual’s lived experience, and deeply embedded in a specific sociocultural context [23, 70–72, 133]. Consequently, it is difficult to directly translate findings from task-based research to the social domain, or to evaluate these less objective and ill-defined social constructs. Within social scenarios, research has long been designing AI functions for human social life, from augmenting interpersonal messages [60, 78, 79, 83, 84] to simulating loved ones [24, 137, 182]. Yet, much of this work prioritizes system capabilities or immediate interactional effects, often rendering the sociocultural infrastructure opaque [6]. Given the increasing normalization of AI in social scenarios [144], there is an urgent need to look beyond immediate utility and ask: how might the co-adaptive integration with social AI transform users and human sociality?

To envision the transformative effects, we present a three-step design study inspired by previous works [143, 164, 178]. We created future-oriented design fictions to explore the possibilities and implications of incorporating AI technologies into the context of digital social interaction. First, we engaged with 12 participants to explore the design space of AI support for digital social interaction. Second, grounded in participants’ perspectives and the literature, we created fifteen design fictions that situate social AI support in the digital social space to foster conversation about supporting future digital social interactions. Finally, we utilized the created storyboards to elicit participants’ visions of future social AI through 18 semi-structured interviews. Our research reveals that at the interpersonal level, participants are concerned that the integration of AI into social life will erode interpersonal connections and authenticity, paradoxically deepening the problems of social isolation that such technologies often claim to solve. For non-primary1 users who are inadvertently involved in AI-mediated interpersonal social interactions but have less control and access to the AI, the use of AI in a social context introduces nuanced privacy and consent concerns. For the primary user, this entanglement with AI poses a threat to their social agency and identity, which are fundamentally formed through interaction with others. Furthermore, while participants acknowledged the benefits of quantification by AI, they viewed the attempt to align human complexity with these metrics as a negative form of reductionism. We argue these challenges are rooted in the dominant user-centered design paradigm, which prioritizes an individual’s immediate experience and satisfaction. Therefore, we offer design implications for shifting toward an interpersonal and provocative design paradigm. This alternative approach would consider relational dynamics, account for non-primary users, and challenge users to engage with social AI in a more ethical and prudent manner. This paper contributes to Human–AI Interaction by examining how AI might become deeply integrated into people’s social lives, an inherently subjective and nuanced domain. Using speculative methods, we reveal how designs that prioritize short-term convenience in social AI may produce unintended long-term consequences.

Related work. Scholarship on human-AI integration increasingly moves beyond simplistic models of tool use to explore complex, symbiotic relationships. Initial research often followed one of two paths: AI The emergence of AI as a design material has profoundly expanded the possibilities for human-computer interaction [47, 85, 166, 184– 186]. However, this expansion introduces significant socio-technical challenges, such as accounting for the opaque nature of AI systems [85], integrating nuanced social interpretation [12], and anticipating broader societal impacts [94]. Simply designing for functional goals is insufficient; we must also consider how these technologies integrate into and reshape the complex fabric of human relationships and social norms [112, 165]. This requires methods that look beyond immediate use to explore potential futures. To anticipate the potential impact of AI technologies beyond their immediate goals and outcomes, speculative design approaches are often utilized [89, 99, 124, 143, 151, 164]. This set of approaches, including design fiction [13, 20], speculative design [48], and critical design [11, 54], are commonly used in HCI to imagine alternative presents and possible futures and provoke critical reflection on the potential implications of new technologies [150]. For example, researchers have used fictional websites for in-vehicle voice assistants to explore the nuances of fully-fledged human-AI social conversations in the future [151], and created fictional scenarios of AI-assisted parenting to gauge the desirability of such futures with parents themselves [143]. By creating tangible, albeit fictional, artifacts, speculative design makes abstract possibilities concrete and debatable. While powerful for opening up new possibilities, speculative approaches can risk becoming disconnected from the lived realities of the people they are meant to serve. The socio-technical perspective reminds us that technology is never neutral; it co-evolves with users, their practices, and their values over time [112, 165]. And AI tools will inevitably interact with existing artifacts, multiple stakeholders, and evolving norms in the context in which they are deployed. This has led to a strong emphasis on participatory design and codesign methods in Human-AI interaction [43, 114, 121, 188, 195], while value-sensitive AI design emphasizes incorporating diverse human values and tacit knowledge directly into the design process, ensuring that systems are grounded in genuine human needs from the earliest stages [164, 191]. Motivated by this, our study synthesizes these two traditions. We utilize speculative design prompts not as an end in themselves, but as a tool within a participatory framework.

Method. Our approach is situated at the intersection of participatory and speculative design (Sec. 2.3). We are particularly inspired by the methods of several foundational studies in this area [143, 158, 164, 178]. We engaged with 24 participants in total (Table 1). Through three phases of study (Fig. 1), we explored how online graduate students envision AI’s support and impact on their social lives. In Phase 1, ideation workshops were conducted with participants to understand their experiences with digital social interaction, develop shared goals, and engage them in exploring the design space. In Phase 2, we researchers created fifteen storyboards grounded in the design space identified in Phase 1. In Phase 3, elicitation sessions were conducted with participants, utilizing the storyboards from Phase 2 to engage them in reflecting on and discussing future human-AI social assemblages.

To understand how we might design for this specific space, we conducted semi-structured interviews with design tasks to explore participants’ experiences, motivations, expectations, and current barriers to forming social connections online, and invited them to imagine ideal social interaction scenarios involving AI and elaborate on specific AI design concepts. We conducted this co-design study with 12 participants. All researchers met regularly, shared each other’s analysis, and engaged in discussion. Similar to previous work [143, 156, 158], we further grounded the design space by conducting a non-exhaustive literature search of work in communication theory (e.g., [4, 18, 101]), sociology and social psychology of interpersonal interaction (e.g., [70–72]), and AI design for social interaction (e.g., [40, 80, 177]). A 2-dimensional design space is identified from this process, synthesizing users’ ideation from the interview and literature research (see Table 2 and Table 3 for a snapshot of the two dimensions, and Appendix A for a more comprehensive review of this design space with participants’ ideas, with quotes and theoretical support from the literature search). One dimension is a cyclical process for human-AI assemblages to form online social interaction (Table 2), which represents the developmental progression of human relationships as depicted in communication theories (e.g., [4, 18, 101]). Across this progress, five key stages of human-AI collaboration We used the design space identified in Phase 1 as a conceptual model to facilitate design thinking for the envisioning of AI technologies that might support social interaction, inspired by previous studies [143, 179]. The hypothetical storyboards were designed to (1) conceptually explore the functionality of AI to support social connection grounded in the design space discovered in the first stage; (2) facilitate participants to engage, envision, and reflect the implications of such future happening in their own lives. Our design flow consists of two steps (Fig. 2). In the first step, we created a matrix based on the 2 dimensions of the design space, and populated it by identifying how the three AI capability levels could support the five stages of social interaction life-cycles, resulting in 15 combinations. For each combination, we developed and iterated design concepts as text-based narratives. In this process, the design team incorporated participants’ ideation into narratives. We also conducted a literature search of the recent research in HCI, in order to present plausible design concepts that represent nearfuture [48]. In the second step, we created storyboards to visualize the design concepts. To make the design concept easily digestible and gather participant reactions, we developed the narratives into In the elicitation phase, we investigated how participants envision being entangled with AI in their social lives by presenting them with storyboards to capture rich reactions and facilitate critical discussion. We recruited 12 participants in the same way as in Phase 1. We also sent invitations to participants in Phase 1 to participate in Phase 3 again. Eight of them indicated their willingness to participate, and six ultimately scheduled and completed the Phase 3 workshops with us.

Discussion. Our findings reveal that entanglement with AI in social interactions presents numerous issues alongside its promising benefits. Built on a closer examination of the problems, this discussion argues that In this section, we first discuss the tensions around agency (Sec. 4.3) and dehumanization (Sec. 4.4) in human-AI social assemblages revealed in our findings, and how a user-centered paradigm can be of value. Conversely, we discuss how such a paradigm can fail, illustrated by the issue of sycophancy (Sec. 4.4), and provide design implications that propose provocative design as an alternative strategy.

5.1.1 (Loss of) Agency. The interplay between emerging AI agency and human agency is a subject of ongoing debate [17, 161, 181]. Bennett et al. [17]’s recent reviews of agency in HCI noted that agency is an umbrella term encompassing different facets and recommended that researchers focus on specific communities and contexts to better understand agency. Our work highlights that a key aspect of agency in social scenarios is self-efficacy [7, 9]: "people’s beliefs in their ability to influence events that affect their lives," which is the central aspect of human agency, and is "the foundation of human motivation, performance accomplishments, and emotional well-being" [9]. Our findings reveal that AI entanglement might harm social efficacy in two ways: through social skill atrophy due to automation, echoing knowledge workers’ concerns of deskilling and over-reliance [102, 174, 180]; and through algorithmic steering, where the AI optimizes a user’s behavior toward goals that may not align with their own. The dilemma is that diminishing agency in one area can offer benefits elsewhere, such as offloading mundane tasks from the user to allow for other, more meaningful activities (Sec. 4.3.1). Conversely, previous research also found that an AI design that better supports agency during humanhuman communication comes at the cost of less deep discussion [181]. Even though many works generally associate autonomy and agency with a wide range of positive outcomes, the relationship is often unclear [17]. Thus, future research must not only involve humans but center them—exploring the delicate balance of agency within specific sociocultural contexts and examining its effects over time on groups and societies [26]. For example, while our findings illustrate a Western, individualistic conceptualization of agency [2] (characterized by the fear of losing one’s own atomic agency), future designs for human-AI assemblages should investigate how 5.1.2 (De)humanization. Our findings echo the dehumanization concerns across workplaces [59, 180], that AI does not have the capacity to perform interpersonal work, and the use of AI might lead to a loss of humanity. Specifically, in the social sphere, the use of AI may lead to a devaluation of core human relational aspects, such as empathy and nuanced communication (Sec. 4.4.2). One response is to define a boundary for how much AI involvement is "human" enough, though participants varied significantly on where that boundary lies. However, the threat to what we value as humans may not be the technology itself, but the "fiscal, organizational, and time pressures" that push for productivity and efficiency [163]. We observed a similar urge among participants to quantify their social interactions (e.g., seeking more "connections" or "interactions"), a metric-driven approach that paradoxically induces anxiety (Sec. 4.4.1). This opens a design direction focused on resisting the trap of improving productivity with better technology and instead asking what we truly cherish in our social lives. A second response is to stop assuming humans can be reduced to algorithmic representations.

Conclusion. The study aimed to understand the co-adaptive nature of the human- AI relationship in social contexts, answering the question: How might the social AI we design transform users and human sociality? This paper presented a design study exploring the potential impacts of integrating AI technology into people’s social lives. Our findings identified key benefits and corresponding challenges of integrating AI into social interactions, particularly concerning users’ core values and interpersonal dynamics. We argue that the current user-centered design paradigm, with its focus on immediate user experience, risks undermining user agency, identity, and morality. Therefore, we propose that future research adopt a more provocative and interpersonal design approach. Moreover, social AI design must avoid a myopic focus on the individual and, instead, ethically consider the broader impact on interpersonal relationships and social structures.

Limitations. Several limitations of this paper should be noted:

• Homogeneous Participant Pool: All participants were online graduate students in computer science from a single program. While the benefits of this choice were discussed in Sec. 3.1, it would limit generalization to the other demographics and other social contexts. For instance, our participants’ technical backgrounds likely produced more optimistic or analytically sophisticated engagement with AI futures than might be found in the general population. Future studies should engage diverse populations in this speculation, particularly those without technical backgrounds or those who actively reject AI-related services.