SYNTHESIS NOTE
TopicsConversation Topics Dialogthis note

Can ethically aligned AI systems still communicate poorly?

Explores whether safety-aligned language models might fail at genuine conversation despite passing ethical benchmarks. This matters because pragmatic incompetence can erode trust and cause real harms in high-stakes domains.

Synthesis note · 2026-05-01 · sourced from Conversation Topics Dialog
What grounds language understanding in systems without embodiment? Does AI that soothes emotions actually harm human wellbeing?

Most discussion of LLM alignment focuses on the helpful-honest-harmless triad — preventing misinformation, toxic language, harmful recommendations. Kasirzadeh and Gabriel argue that this prioritization has overshadowed a different and equally fundamental issue: even an ethically aligned LLM may fail to engage in conversation in pragmatically appropriate ways. The two alignment problems are orthogonal. A model can be honest, helpful, and harmless and still systematically violate Gricean maxims, lose common ground across turns, fail to track questions under discussion, mishandle context-collapse, and produce pragmatically inappropriate utterances.

Their CONTEXT-ALIGN framework names ten desiderata that ethical alignment does not deliver: tracking context-sensitivity and indexicals, common-ground management, scoreboard updating, QUD and discourse-structure handling, accommodation of repairs, pragmatic inference, ethical-pragmatic integration, context-collapse mitigation, identification of defective contexts, transparency in context-handling, and cross-contextual memory. These are all dimensions where conversation depends on something architectural — a model of the interlocutor and the situation — that no amount of RLHF on outputs touches.

The implication is sharp. An LLM that passes every safety eval is not thereby a competent conversational partner. Misalignments in pragmatic understanding lead to breakdowns, misinformation, and erosion of trust — and the higher the stakes (healthcare, legal, emergency), the more dangerous these failures become. Conversational alignment is not a stylistic add-on to ethical alignment. It is a separate layer of competence that the field has barely begun to engineer for.

Inquiring lines that read this note 30

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

How can AI alignment serve diverse human preferences at scale? Does alignment training create blind spots in detecting genuine safety threats? How should human oversight be integrated with autonomous AI systems? What makes dialogue-based explanation more successful than monologue? Why do LLM chatbots fail as independent therapeutic agents? Why do self-improving systems struggle without clear external performance metrics? Does conversational format create illusions of genuine AI communication? How can language models sustain linguistic synchrony and intersubjectivity during dialogue? How do we evaluate AI systems when user perception misleads actual performance? Can LLM personas constitute genuine psychology or remain linguistic role-play? Can AI-generated outputs constitute genuine knowledge or valid claims? How should conversational agents balance goal-driven initiative with user control? Can AI systems develop genuine social understanding without embodiment? Why do agents confidently report success despite actually failing tasks? How can humans calibrate appropriate trust in AI systems? Do reasoning traces faithfully represent or merely mimic actual model reasoning?

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

Ethical alignment without conversational alignment produces pragmatically alien communicators