SYNTHESIS NOTE
Topics›Conversation Topics Dialog›this note

Can language models balance competing ethical norms in context?

Do LLMs genuinely weigh trade-offs between honesty, helpfulness, and harm prevention based on what a specific conversation needs, or do they rigidly enforce fixed corporate values regardless of situation?

Synthesis note · 2026-05-01 · sourced from Conversation Topics Dialog

Gricean pragmatics insists on situated normativity: speakers do not blindly follow maxims (quantity, quality, relation, manner) but apply, suspend, violate, or exploit them according to context. When a doctor withholds a terminal diagnosis from a frightened patient, the doctor violates the maxim of quantity to uphold compassion. The violation is not a failure — it is the right move in context, and a competent hearer recognizes it as such. Pragmatic competence is the ability to navigate these conflicts, not the ability to maximize each maxim independently.

LLMs trained on the helpful-honest-harmless triad cannot perform this kind of contextual reasoning. The corporate persona is fixed at the model level: when a user asks for accessible simplification of a complex topic for a child, the model trained for honesty refuses to soften because softening reads as less accurate. When a user asks for sarcastic humor, the model trained for harmlessness refuses to play. The user cannot persuade the model to relax its norms because the norms are structural defaults rather than negotiable conversational moves.

Kasirzadeh and Gabriel describe this as pragmatic dissonance. The model mechanically enforces global norms even when local context demands tailored adherence. The result is communication that adheres to ethical principles at the cost of pragmatic appropriateness — exactly the trade-off that situated normativity is meant to navigate. What humans treat as a single integrated competence becomes, in the LLM, two separate layers in tension with each other.

Inquiring lines that read this note 50

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

What distinguishes emotionally supportive AI from AI that fosters unhealthy dependence? How does tokenizing intelligence reshape labor markets and institutional outcomes? What conversation mechanics enable genuine human-AI communicative participation? Does preference-optimized training sacrifice accuracy and calibration for perceived quality? Why doesn't LLM knowledge translate to improved reasoning outputs? Do language models genuinely understand or primarily match statistical patterns? How much do feedback loops and social biases compound in rating systems? What systematic failures occur in human preference-based model evaluation? How do prompt design choices and framing measurably influence model outputs? How should AI ethics integrate consciousness, values, and transparent disclosure? Can language models develop genuine communicative grounding with humans? What factors determine AI persuasion effectiveness and what controls mitigate it? What constraints limit conversational AI's effectiveness in therapeutic contexts? Can validator agreement alone ensure semantic correctness of reasoning? What reasoning limitations and biases do language models systematically exhibit? How can institutions effectively enforce controls on AI development? What systematic biases do LLM judges exhibit in evaluation? Do engagement-optimized chatbots create dependencies that undermine user autonomy? What distinguishes genuine understanding from correct linguistic performance? Why do multiple language models independently generate similar outputs? How does silent agreement emerge and undermine deliberation in multi-agent systems?

Related concepts in this collection 2

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
15 direct connections · 161 in 2-hop network ·dense cluster Open in graph ↗

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

LLM refusals and tone choices reflect overarching corporate values rather than context-specific Gricean norm-balancing