Agentifying Agentic AI

Agentic AI seeks to endow systems with sustained autonomy, reasoning, and interaction capabilities. To realize this vision, its assumptions about agency must be complemented by explicit models of cognition, cooperation, and governance. This paper argues that the conceptual tools developed within the Autonomous Agents and Multi-Agent Systems (AAMAS) community, such as BDI architectures, communication protocols, mechanism design, and institutional modelling, provide precisely such a foundation. By aligning adaptive, data-driven approaches with structured models of reasoning and coordination, we outline a path toward agentic systems that are not only capable and flexible, but also transparent, cooperative, and accountable. The result is a perspective on agency that bridges formal theory and practical autonomy.

Introduction. The growing enthusiasm for Agentic AI reflects a longstanding ambition in Artificial Intelligence to build systems that are able to act autonomously, reason purposefully, and interact meaningfully with others (agents or persons). Yet, despite the novelty of current approaches, mostly based on large language models (LLMs), the challenges Agentic AI faces are not new. Questions of autonomy, coordination and accountability have been at the heart of Autonomous Agents and Multiagent Systems (AAMAS) research for decades now. What has changed is the context in which Agentic systems are developed: large-scale data-driven systems aiming to act in open environments. However, the same sociotechnical challenges of autonomy, interaction, and adaptability remain. Current agentic systems promise flexible reasoning and continuous action, but often lack the explicit architectures, communication and reasoning semantics, and normative grounding that earlier agent models provided. Conversely, AAMAS research developed rich, explicit models of cognition, communication, and social structure, but often lacked the flexibility and scalability that characterize current datadriven approaches.

This paper argues that in order to make agentic AI systems truly agents, learning-based mechanisms must be complemented by structured reasoning and coordination models. Concepts such as reasoning architectures, formal interaction protocols, norms, and institutional governance provide the conceptual tools needed to turn behavioural autonomy into responsible agency. By aligning and complementing the adaptive power of foundation models with the explicit structure of agent-based reasoning, we outline a path toward systems that are not only capable and flexible, but also coherent, cooperative, and accountable.

When people discuss the combination of LLMs and agents they usually start from the premiss that LLMs are tools that bring a lot of ”intelligence” that should be harnessed by the agent. A (simplified) comparison between traditional agents and multi-agent systems and Agentic AI would say that traditional agents are controlable but limited. Due to the fact that all intelligence needs to be added manually, they might miss the knowledge to act in certain situations. Agentic AI is based on the power of LLMs that is known to be able to generate decisions in all situations, but might sometimes do very unexpected and harmful things. From the view point of computer science, the latter is very undesirable. It prevents the agents to be verifiable and validated. That is, the function of the module and its inputs and outputs cannot be guaranteed. In a proper design of a system these guarantees are needed to build a complete system of which one can later on show that it lives up to expectations and that can be verified in some way. Our argument is that LLMs are not easily seen as such predictable and specifiable components. There are two aspects of LLMs that make them inherently more difficult to handle. First LLMs imitate human intelligence in an input-output way. I.e. they can mimic people in many cases by producing output (text) that people would produce given certain input. That does not mean that the LLM is doing the same or that you can expect it to do something in the future based on your experienced interactions with it in the past. In other words the results from the past are no guarantee for results in the future. E.g. a question to book flights to go to a conference might result in a booking of a morning flight to go there and arrive just in time for the first session and leave just after the last session finished. However, the next time it might book a return flight in the morning of the last day of the conference because it happens to be on sale. Of course this could be remedied by careful prompting. But just like with traditional agents, if the user forgets something you will get bad results that are not foreseen. For some types of systems this might not be a big problem, but for systems that are designed to achieve certain predefined goals or targets that is pretty bad. In some sense producing bad results or performing unwanted actions is worse than not performing any action at all. If an agent is part of a multi-agent system other agents might start acting autonomously based on this unwanted result and the user might only notice many steps later that something is wrong. This means that potentially a lot of actions have to be undone and one needs to sort out which those were and what went wrong in the first place. If an agent cannot take an appropriate action the user will be prompted and can solve the issue right away without elaborate roll backs being necessary. The second problem of using LLMs is that they ”live” in a world of data. If we would feed an LLM a billion documents stating that 2+2=5 it will start to replicate that.

Related work. Agentic AI denotes systems that couple large-scale foundation models with capabilities to reason, act (e.g., via tools or environments), and interact with users and other systems in a sustained, goal-directed manner.While this shift extends AI from passive prediction to task-oriented autonomy, it also surfaces open problems, as indicated in recent analyses, such as the AAAI 2025 Presidential Panel on the Future of AI Research (Rossi et al. 2025). These challenges include:

(i) Reliability and grounding (Plaat et al. 2025). Agentic systems often struggle to connect their language-based reasoning with the actual state of the world. They may generate plausible but false information, or take actions that do not reflect real conditions, particularly when using tools or interacting with digital environments. This lack of grounding can lead to cascading errors and undermines user trust. (ii) Long-horizon agency (Zhang et al. 2025). Although these systems can plan or reflect in short sequences, they remain fragile when tasks require sustained reasoning, long-term goals, or adaptation over time. Their internal memory and self-monitoring are still rudimentary, which limits their capacity for persistence, learning from past outcomes, and maintaining coherent strategies. (iii) Evaluation (Rossi et al. 2025; UK AI Safety Institute 2024a,b). There are no agreed methods or benchmarks to assess the performance, safety, or autonomy of agentic systems. Current evaluations are fragmented, often focusing on narrow tasks or short interactions. As a result, it is difficult to compare systems or determine whether an agent is genuinely capable, reliable, or aligned with intended goals. (iv) Risk management and governance (National Institute of Standards and Technology 2023, 2024). Once deployed, agentic systems can act continuously and independently, which raises unresolved questions about accountability, oversight, and liability. Few governance frameworks clearly specify who is responsible for an agent’s actions, how its decisions can be audited, or when human intervention should occur. Transparency and safety assurance remain limited. (v) Security and privacy (UK AI Safety Institute 2024c; Whittaker 2025). The ability of agentic systems to access tools, data, and online services introduces new vulnerabilities. Malicious inputs, excessive permissions, or unintended data sharing can expose sensitive information or allow agents to be manipulated. Ensuring security and privacy therefore requires strict controls on access, monitoring, and data handling. (vi) Value and maturity (Zohaib et al. 2025; Reuters 2025). Despite significant enthusiasm, the practical benefits of agentic systems are still uncertain.

Method. Thus it is possible to reason about when a goal should be given up as it is deemed to be unachievable (possibly due to a changing environment). E.g. when an agent tries to book a flight for a person to a certain destination and date(s), it can stop trying after it looked for all possibilities of direct and indirect flights going in a certain spatial corridor to the destination. Thus it is prevented from trying to book a flight from Amsterdam to Paris that goes via Rome. Although Agentic AI systems can be given some restrictions in their prompt it is well known that they often do not cope very well with combinations of numerical constraints. Within AAMAS tradition, such architectures are not merely conceptual but serve as a foundation for reasoning about autonomy, cooperation, and accountability across distributed agents. E.g. in applications where agents have to cooperate to achieve a goal they might want to reason about what information another agent has available in order to communicate additional information that is needed to achieve their goal. E.g. if two agents try to book flights for colleagues to the same conference they might want to communicate the flight number and seat number such that the other agent can try to book the same flight and a seat that is adjacent. Thus having explicit architectures with mental attitudes enable explicit introspection, communication of mental attitudes, and logical verification of consistency between goals and actions, features that make systems both interpretable and governable. By contrast, contemporary Agentic AI systems, particularly those built upon large language models, typically operate without explicit internal state representations or commitment mechanisms. Their “intentionality” is implicit, inferred from surface-level behavior rather than formally specified reasoning. As a result, such systems may appear adaptive but lack the structural guarantees that link motivations to actions, limiting their alignment, predictability, and verifiability.

Communication Protocols. Communication has always been central to multi-agent systems. Classical AAMAS research developed formal languages such as KQML and FIPA-ACL (Finin et al. 1994), which define not only message syntax but also the semantics of communicative acts like requests, commitments, or promises. These frameworks ensure that agents can exchange information, negotiate, and coordinate actions with a shared understanding of intent and context. In contrast, most Agentic AI systems rely on unstructured natural language dialogue, mediated by large language models. While flexible and expressive, such exchanges lack guaranteed semantics: meaning is inferred statistically rather than interpreted formally. E.g. suppose I know that someone is arriving tomorrow at university and I have reasons to try and avoid them. I do not know when the person arrives exactly which makes it difficult to avoid them. If my personal agent asks the personal agent of the host whether it knows the arrival time of the guest, I just want that agent to state the arrival time if it can look it up in its own information. I would not like it to start asking the guest for the arrival time on my behalf (giving my request as the reason for the question). In traditional agent systems one can distinguish these two situations by sending different types of requests for information. Thus avoiding potential awkward situations. The natural language communication of agentic AI does not use this strict semantic approach and can thus lead to misunderstanding, inconsistent commitments, or spurious alignment between interacting agents. Reintroducing formal semantics or constrained communication layers could therefore improve reliability, traceability, and accountability in multi-agent interactions.

Mechanism Design and Incentive Alignment. A central insight of the AAMAS community has been that effective collective behavior requires not only intelligent agents but also mechanisms that align individual incentives with group objectives.

Discussion. Implications and Reflection Together, these AAMAS foundations illustrate that agency involves far more than autonomous action: it depends on explicit reasoning structures, communication protocols, social organization, and mechanisms for trust, accountability, and shared purpose. The comparison between AAMAS and Agentic AI shows that many of the current challenges in autonomous and interactive systems are not entirely new, but reappear in different technical forms. What distinguishes the AAMAS tradition is its commitment to making agency explicit, explainable, and socially grounded. Re-engaging with these principles provides not merely historical insight but a conceptual foundation for tackling the pressing problems of transparency, coordination, and governance in today’s agentic systems. By combining the adaptive capabilities of largescale learning with the representational and normative rigor of multi-agent research, future AI systems can move toward a form of agency that is both effective and accountable—capable of autonomous reasoning and collaboration while remaining aligned with human values and institutional expectations. Beyond these conceptual correspondences, the next step is to operationalize these principles through concrete methods, tools, and governance structures. Agentic systems function within socio-technical contexts where human oversight, interpretability, and trust are essential; explicit representations of reasoning and commitments can support shared mental models between humans and agents, enabling meaningful supervision rather than mere control. Building on AAMAS traditions of formal verification, simulation-based evaluation, and logic-based reasoning, future research should develop standardized benchmarks for autonomy, cooperation, and social compliance, aligning with ongoing initiatives such as the NIST AI Risk Management Framework, EU AI Act, and e.g. the UK AI Safety Institute. Embedding agentic systems within institutional and normative frameworks must also address questions of power and value diversity, who defines norms, whose objectives are optimized, and how plural values are represented. By combining conceptual rigor with these emerging governance standards, the field can move toward hybrid architectures that are not only capable and adaptive, but also transparent, accountable, and socially legitimate.

Conclusion. The emergence of Agentic AI marks a significant step in extending artificial intelligence from passive prediction to purposeful, interactive, and context-aware behaviour. To realize this potential, the development of agentic systems requires a clearer conceptual foundation for what it means to act, decide, and coordinate within complex socio-technical environments. Some of the major problems for agents based on foundation models are that the foundation models have learned the appropriate behaviour based on data that might contain data on actions, but rarely indicates the social context in which those actions were performed. Some of that context will be implicitly taken on in the models. Thus the agentic systems might perform good in a very standard context, but they will have problems to perform well when the context changes or an exception needs to be handled. A second important issue is that the agentic AI is a monolithic structure in which it is difficult to distinguish the different aspects that caused a decision and and action to be taken. This is problematic when agents have to cooperate and need to form expectations on each other’s behaviour. The research tradition of Autonomous Agents and Multi- Agent Systems (AAMAS) offers the theoretical and methodological scaffolding that can strengthen these foundations. Formal models of reasoning, interaction, organization, and governance provide principled ways to represent goals, commitments, and interdependencies, concepts essential for scalable and trustworthy autonomy. Integrating such models enhances coherence, verifiability, and social grounding, turning “agentic AI” from ad-hoc behavioural systems into coherent, cooperative, and accountable socio-technical actors.