Unraveling Human-AI Teaming: A Review and Outlook

Paper · arXiv 2504.05755 · Published April 8, 2025
Human-Centered Design

Abstract. Artificial Intelligence (AI) is advancing at an unprecedented pace, with clear potential to enhance decision-making and productivity. Yet, the collaborative decision-making process between humans and AI remains underdeveloped, often falling short of its transformative possibilities. This paper explores the evolution of AI agents from passive tools to active collaborators in human-AI teams, emphasizing their ability to learn, adapt, and operate autonomously in complex environments. This paradigm shifts challenges traditional team dynamics, requiring new interaction protocols, delegation strategies, and responsibility distribution frameworks. Drawing on Team Situation Awareness (SA) theory, we identify two critical gaps in current human-AI teaming research: the difficulty of aligning AI agents with human values and objectives, and the underutilization of AI’s capabilities as genuine team members. Addressing these gaps, we propose a structured research outlook centered on four key aspects of human-AI teaming: formulation, coordination, maintenance, and training.

Our framework highlights the importance of shared mental models, trust-building, conflict resolution, and skill adaptation for effective teaming. Furthermore, we discuss the unique challenges posed by varying team compositions, goals, and complexities. This paper provides a foundational agenda for future research and practical design of sustainable, high-performing human-AI teams.

Key words: Artificial Intelligence; AI Agent; Human–AI Teaming; Situation Awareness; Research Outlook

Introduction. Artificial Intelligence (AI) is advancing at an unprecedented pace, transforming industries and reshaping daily life. From healthcare and finance to education and manufacturing, AI-based systems are increasingly supporting humans acrossvariousfields(Berenteetal.2021,LouandWu2021,LuandZhang2025).Yet,despiteAI’spotential, collaborations between humans and AI remain ad hoc, often resulting in underwhelming performance (Teaming 2022). The promise ofAI lies in itsability to integrate intoteams to augment humancapabilities, improve decisionmaking, enhance productivity, and drive innovation. Realizing the benefits, however, requires a more systematic examination of the novelties associated with human-AI teaming.

In traditional human-human teaming, researchers have extensively studied shared awareness, communication, and collaboration, establishing frameworks around communication frequency, shared mental models, task complexity,andtraining.Similarly,human-systemteaminghasfocusedontheuseofcommunicationandcollaboration tools, which support team coordination but remain distinct from the tasks themselves (Dennis and Wixom 2002, DeSanctis and Gallupe 1987, Jerry Fjermestad 2000). Implicit in all team-based interactions is the assumption that team members share common values and work together toward shared goals and objectives. The perspective that technology is a tool to enhance human efforts needs a refresh with the emergence of AI agents. Recent studies have shown that AI agents are evolving rapidly, with their capabilities doubling approximately every seven months since 2019 (Kwa et al. 2025).1 In this regard, human-AI teaming introduces a new era of team science, requiring revised interaction protocols for AI agents, and new approaches to delegation, task execution, and responsibility sharing.

In this paper, we examine the emerging landscape of human-AI teaming, where AI has moved beyond simple automation to become an intelligent, agentic artifact (Dennis et al. 2023, Gao et al. 2024b). The rapid rise of large language models (LLMs) such as Generative Pre-trained Transformer (GPT) has accelerated this transformation.

AI-enabled agentic artifacts are no longer simply embedded into workflows; they are now intelligent agents capable ofactivelytakingonmission-criticalrolesandtaskswithinteams(Alavi2024,McKinsey2023,2024).Thesesystems learn from interactions, adapt to changing contexts, and make independent decisions, transforming AI into an 1 The concept of AI agents is not new. Russell and Norvig (2016) define them as entities that perceive their environment and act rationally. Thispaperbuildsonandmodernizesthatfoundationalviewtoreflecttheadvancedcapabilitiesoftoday’sAI.Today’sAIagentshaveevolved into autonomous, iterative, and intelligent team members. They learn, reflect, and adapt within complex and dynamic environments. Moreover, they are no longer passive tools but active collaborators in human-AI teams in sustained interaction, emotional intelligence, and autonomous problem-solving. This marks a paradigm shift beyond the classical notion of rational action. autonomous partner in collaborative efforts. The agentic shift highlights some gaps in the current information systems (IS) literature, which mainly treats AI as a tool rather than a nearly full-fledged teammate (Dennis et al.

2023, Lu and Zhang 2025, Wang et al. 2019).

We identify two major research gaps. First, there is a significant challenge in designing AI agents that align with human values and objectives. This gap arises because humans themselves often struggle to explicitly articulate their goals and objectives. Effective team protocols and processes must account for these ambiguities to enable AI agents to augment human capabilities. Advancing human-AI collaboration requires strategies that bridge this gap and facilitate meaningful integration. The second gap stems from the limited integration of AI’s full capabilities into collaborative workflows. Many still view AI agents as static tools rather than adaptive, knowledge-enhancing partners. This narrow perspective restricts AI’s potential in dynamic decision-making and strategic reasoning while behavioral resistance and inertia further reinforce the issue (Polites and Karahanna 2012). Many individuals hesitate to accept AI as a genuine team member, leading to underutilization and even adverse outcomes. Bridging this gap requires reconceptualizing AI as an active partner in knowledge-intensive tasks, fostering deeper interaction between human and algorithmic intelligence to unlock AI’s full collaborative potential.

This paper aims to bridge these gaps by applying the well-established Team Situation Awareness (SA) theory to the context of human-AI teaming. We argue that integrating AI agents into teams not only requires designing systems that can work toward shared objectives but also requires a rethinking of how teams execute tasks alongside AI agents.

Related work. 2. Revisiting Human-AI Teaming 2.1. Emerging Agentic AI Systems Information systems are beginning their transformation into agentic artifacts that “are no longer passive tools waiting to be used, are no longer always subordinate to the human agent, and can now assume responsibility for tasks with ambiguous requirements and for seeking optimal outcomes under uncertainty” (Baird and Maruping 2021).

Modern AI systems, with their ability to “learn and act” independently of human instructions, are showing the promise to emulate human thought processes, behavior, and decision-making (Dennis et al. 2023, Durante et al.

2024). Although these systems may remain within the realm of Narrow AI, their capabilities now extend to many tasks that, until recently, required highly trained professionals.

Generative AI systems, such as ChatGPT, Midjourney, Claude, Llama, and Gemini, excel in tasks requiring nuanced judgment and adaptability — areas where classical AI systems often falter due to “brittleness” when encountering scenarios outside their predefined rules (Dhar 2023, McKinsey 2023). They now assist in domains previously dominated by human cognition and expertise (Susarla et al. 2023, Zhou and Lee 2024). For example, ChatGPT interprets ambiguous prompts to draft complex documents and hold meaningful conversations.

Similarly, Midjourney generates original artwork from simple text, demonstrating creative problem-solving skills that go beyond rule-based automation. Importantly, these capabilities pave the way for the emergence of agentic AI systems—systems that not only assist but also act autonomously and collaborate dynamically with humans (Durante et al. 2024). Agentic AI engages in creative, coordinated tasks, takes initiative in decision-making, and adapts through iterative feedback. By generating context-aware outputs and exhibiting independent reasoning, they challenge traditional models of human-AI interaction.

The key characteristics of these agents include:

• Planning: The ability to plan and execute multi-step processes to achieve requested goals. For example, an agent can outline an essay, conduct online research, and draft the essay in a sequential manner (Ng 2023).

• Self-explanation: A notable feature that allows the agent to communicate and reason effectively, possibly requiring a robust theory of mind for both itself and the human users (Bubeck et al. 2023).

• Sustained interaction: The facilitation of dynamic, prolonged engagements with users, moving beyond static, single-question interactions (Traumer et al. 2017).

• Reflection: The capacity to reflect on their work to identify and implement improvements, thereby continually enhancing their performance (Ng 2023).

Method. 2.3. Theoretical Framework: An Extended Model of Team Situation Awareness The classic team situation awareness (team SA) theory (Endsley 1995) provides a foundational framework for understanding the intricate dynamics (i.e., teaming process) that shape effective teamwork. As illustrated in Figure 1, a basic element of the team SA model is individual SA, which includes three levels: perception, comprehension, 2 This follows Gao et al. (2024a) who define the interaction flow of human-LLM co-working. and projection. Individual perception and comprehension SA enable team members to interpret and process information related to the collaborative environment. Comprehension SA enables understanding of teammate’s task contributions. Projection involves anticipating future actions of team members and consequences (Endsley 1995).

In addition to individual SA, the team SA model emphasizes the critical concept of “shared SA” (Teaming 2022), defined as “the degree to which team members possess the same SA on shared SA requirements” (Endsley 2001).

Shared SA is based on shared objectives among team members and goes beyond individual SA by encompassing the collective understanding of team members regarding the shared SA requirements. This requires cognitive investmentandthusconstitutesa“sharedmentalmodel”(Blickensderferetal.1997).Suchamodeliscrucialforsuccessful collaboration. Specifically, in a teaming process, shared SA is crucial to foster effective and creative idea/planning generation, and streamline the team work process for task implementation (Straus 1999). Shared SA underscores the cohesive fabric that binds team members into a synchronized unit and minimize the potential for miscommunication and misinterpretation (Bubeck et al. 2023).

Recognizing the participation of intelligent agents in a team Section 2.2, we extend the team SA model to incorporate role specification and role fluidity. Role specification ensures that all team members, including AI, clearly understand their specific tasks while effectively accommodating themselves with other’s expertise. Moreover, teamwork thrives on adaptability, open communication, and shared responsibilities – core functions of role fluidity.

Flexible roles allow teams to respond effectively to dynamic and uncertain environments, fostering collaboration and innovation. Team learning is therefore crucial for achieving sustainable team performance (Vashdi et al. 2013).

It also aligns with and underscores the inherent dynamism of shared SA in human-AI teaming. This dynamism is driven by continuous feedback loops, ensuring that shared SA remains adaptive and responsive to changing circumstances (Demir et al. 2023).

Team-related activities, as outlined in the Circumplex model of group tasks (Posner et al. 2005), typically involve generating ideas or plans, selecting the most suitable solution, negotiating a resolution to conflicts, and executing tasks. In the context of human-AI teaming, it becomes crucial to revisit and explore how team SA interacts with these complex tasks. AI’s capacity to rapidly process large volumes of data and generate insights can work for both role specification and fluidity to enhance team SA, potentially reshaping how teams handle decision-making, conflict resolution, and task execution. In addition, as discussed earlier, the integration of AI introduces new complexities, such as the need for alignment between human and AI-generated awareness. As AI’s role in teams grows, Human situation awareness Level 1: Perception Level 2: Comprehension Level 3: Projection Team activities (Generating, selecting, negotiating, and executing tasks)

Discussion. AsAIagentscontinuetoadvanceintoincreasinglyproficientcollaborators(USAF2015),human-AIteamingexemplifies a paradigm where humans and agents engage in iterative and multi-modal communications to pursue shared goals. Key phases of this collaboration include planning, facilitation, iteration, and testing. Maintaining optimal teaming dynamics in such settings demands not only coordination strategies but also conflict management to mitigate potential frictions and ensure effective task execution.

4.1.1 Expected Outcomes AI agents are expected to become active contributors/members rather than passive tools in team (Dennis et al.

2023, Vanneste and Puranam 2024). This shift envisions AI agents to process complex information and develop actionable insights to complement human creativity and decision-making. AI and human partnership can enhance problem-solving efficiency and reduce the cognitive workload on human team members. Particularly, Salas et al.

(2017) emphasize that trust among team members is essential to enhancing information sharing and reducing communication hesitancy. If trust in the AI team collaborator increases, it would lead to a more cohesive team dynamic. Trust in AI is expected to grow as its contributions become more accurate and contextually relevant, enabling smoother workflows and more confident task delegation. However, this integration brings challenges.

Over-reliance on AI may suppress human creativity, learning, and critical thinking, as individuals increasingly defer to AI-generated solutions without adequate investigation (Dell’Acqua et al. 2023, F ̈ugener et al. 2021). Misaligned expectations, where AI outputs fail to meet team goals, can lead to frustration, diminished trust, and resistance to adoption. Furthermore, excessive dependence on AI for routine and complex tasks risks eroding human expertise, leaving teams vulnerable in scenarios where AI is unavailable or unreliable. Addressing these tensions is crucial for fostering sustainable and balanced collaboration in human-AI teams.

Lines of inquiry this paper opens 24

Research framings built by reading the notes related to this paper — the questions it feeds into.

How should control and autonomy be allocated in human-AI collaboration? Why do human-AI teams underperform compared to their individual components? What organizational practices enable successful AI tool adoption? When does governance become the bottleneck for agentic AI? When do multi-agent systems outperform single agents? Can we reliably measure and demonstrate AI's actual productivity impact? Does generative AI replace coordination with individual productivity?