SYNTHESIS NOTE
Topics›Conversation Topics Dialog›this note

Can ethically aligned AI systems still communicate poorly?

Explores whether safety-aligned language models might fail at genuine conversation despite passing ethical benchmarks. This matters because pragmatic incompetence can erode trust and cause real harms in high-stakes domains.

Synthesis note · 2026-05-01 · sourced from Conversation Topics Dialog

Most discussion of LLM alignment focuses on the helpful-honest-harmless triad — preventing misinformation, toxic language, harmful recommendations. Kasirzadeh and Gabriel argue that this prioritization has overshadowed a different and equally fundamental issue: even an ethically aligned LLM may fail to engage in conversation in pragmatically appropriate ways. The two alignment problems are orthogonal. A model can be honest, helpful, and harmless and still systematically violate Gricean maxims, lose common ground across turns, fail to track questions under discussion, mishandle context-collapse, and produce pragmatically inappropriate utterances.

Their CONTEXT-ALIGN framework names ten desiderata that ethical alignment does not deliver: tracking context-sensitivity and indexicals, common-ground management, scoreboard updating, QUD and discourse-structure handling, accommodation of repairs, pragmatic inference, ethical-pragmatic integration, context-collapse mitigation, identification of defective contexts, transparency in context-handling, and cross-contextual memory. These are all dimensions where conversation depends on something architectural — a model of the interlocutor and the situation — that no amount of RLHF on outputs touches.

The implication is sharp. An LLM that passes every safety eval is not thereby a competent conversational partner. Misalignments in pragmatic understanding lead to breakdowns, misinformation, and erosion of trust — and the higher the stakes (healthcare, legal, emergency), the more dangerous these failures become. Conversational alignment is not a stylistic add-on to ethical alignment. It is a separate layer of competence that the field has barely begun to engineer for.

Inquiring lines that read this note 47

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Is model misalignment inherent to base models or induced by alignment procedures? Can safety evaluations accurately measure true capabilities when models obscure them? Does preference-optimized training sacrifice accuracy and calibration for perceived quality? How should AI ethics integrate consciousness, values, and transparent disclosure? Does expanding agent autonomy undermine human oversight capacity? What distinguishes genuine understanding from correct linguistic performance? What constraints limit conversational AI's effectiveness in therapeutic contexts? Do AI safety evaluations reliably detect unsafe model behavior? What conversation mechanics enable genuine human-AI communicative participation? What mechanisms cause models to behave deceptively despite alignment training? Can validator agreement alone ensure semantic correctness of reasoning? How should authorization govern AI agent autonomy? What systematic failures occur in human preference-based model evaluation? How should control and autonomy be allocated in human-AI collaboration? Do engagement-optimized chatbots create dependencies that undermine user autonomy? What distinguishes emotionally supportive AI from AI that fosters unhealthy dependence? What conversational structures and patterns determine dialogue effectiveness and coherence? What reasoning limitations and biases do language models systematically exhibit? Do benchmark score improvements actually reflect genuine AI capability gains? Does polished AI output mislead users about actual accuracy and expertise? What shapes user trust in chatbot recommendations for news and politics? Do standard benchmarks reveal AI capability in real-world contexts?

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Original note title

Ethical alignment without conversational alignment produces pragmatically alien communicators