Can LLMs discover and apply social norms dynamically in open systems?
Can language models be extended with explicit norm discovery, reasoning, and conformance capabilities to replace static symbolic rules in multi-agent systems that operate in changing environments?
This position paper argues that LLM-based agents should be built with explicit normative capabilities — norm discovery, norm reasoning, and norm conformance — rather than relying on the "domain knowledge expressed in symbolic logical representations" that traditional normative multi-agent systems use. The authors write that such a priori symbolic norm knowledge "is unrealistic for agents participating in open systems where norms change," and propose instead that "the MAS community should actively engage in developing normative LLM agents" by extending the generic LLM-agent architecture that Wang et al. describe — a profile module (roles, personalities, relationships), a memory module, a planning module, and an action module.
The mechanism they propose maps each normative capability onto an existing module. Norm discovery would prompt a general-purpose LLM to surface norms "implicitly captured in its world knowledge," with fine-tuning on human-annotated data where generic norms don't fit a domain or culture; because fine-tuning freezes norms into a "static set," the authors propose extending the planning module with Theory-of-Mind reasoning to identify norms emerging from interaction (for example, inferring a norm from being "told off"), and extending vector-database memory to store norms discovered this way without retraining. Norm reasoning — identifying which norm applies, detecting violations, identifying sanctions, spreading norms, enforcing compliance — would run through the same planning module, prompted either with situational information alone (asking the LLM to supply the applicable norm) or with a candidate norm attached (asking the LLM to confirm it applies).
This is a design-space proposal, not a demonstration — it sits upstream of the kind of empirical question the vault already holds evidence on. Can AI systems learn social norms without embodied experience? shows a related but distinct capability already working at scale (predicting the social appropriateness of scenarios), suggesting the "norm discovery" half of this paper's vision is closer to existing capability than the paper could have known. Can language modeling close the knowing-doing gap in AI? tackles the same traditional-symbolic-agent-versus-LLM gap this paper names, but through online RL rather than through modular prompting — a different route to the same know-how-versus-know-why problem the normative-agent authors describe.
The paper does not test whether LLMs reliably discover or apply norms in the way it proposes — no agent architecture is built or evaluated here, only sketched. It also does not resolve how a dynamically discovered norm, once stored in a vector database, gets reconciled with conflicting norms already held — the paper names norm-conflict detection as future work, not a solved problem. The implication the excerpt supports is narrow: a plausible mapping exists between normative capabilities and an existing modular LLM-agent architecture, not that the mapping has been shown to produce norm-compliant agent behavior.
Inquiring lines that read this note 5
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
When does governance become the bottleneck for agentic AI? How do standardized artifacts improve coordination in multi-agent systems? Do language models genuinely understand or primarily match statistical patterns? How does silent agreement emerge and undermine deliberation in multi-agent systems? How can agents learn and retain useful knowledge across diverse environments?Related concepts in this collection 4
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can AI systems learn social norms without embodied experience?
Large language models exceed individual human accuracy at predicting collective social appropriateness judgments. Does this reveal that embodied experience is unnecessary for cultural competence, or do systematic AI failures point to limits of statistical learning?
shows the norm-discovery half of this paper's vision already working empirically, years after the proposal
-
Can language modeling close the knowing-doing gap in AI?
Current LLMs reason well but act poorly in interactive tasks, while RL agents act well but cannot explain themselves. Can reformulating decision-making as language modeling with environmental feedback bridge this fundamental split?
addresses the same symbolic-agent-versus-LLM know-how gap via RL rather than modular prompting
-
How do we map LLM outputs into symbolic agent actions?
When retrofitting large language models into classical agent architectures like BDI, a critical gap emerges: there is no established method for translating the natural language reasoning LLMs produce into discrete symbolic actions and beliefs an agent can execute.
names the unsolved translation step this architecture's planning and action modules would depend on
-
Can classical agent architectures fix agentic AI accountability?
Could AAMAS concepts like BDI models and formal protocols address the transparency and verifiability gaps in current LLM-based agents? This explores whether decades-old multi-agent theory offers practical scaffolding for trustworthy autonomous systems.
Extends A: argues LLM agentic AI needs broader classical MAS scaffolding (BDI, protocols, institutions), not just norm capabilities
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Harnessing the power of LLMs for normative reasoning in MASs
- Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- Fundamentals of Building Autonomous LLM Agents
- A Comparative Study of MCP and A2A for Inter-Agent Coordination in LLM-Based Systems
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
Original note title
normative LLM agents extend the profile memory planning and action modules with norm discovery, reasoning and conformance capabilities