INQUIRING LINE

Should an AI's ethics bend to fit each person and conversation, or follow the same fixed rules for everyone, always?

Can context-sensitive relational evaluation replace universal ethical principles for AI?

This explores whether AI could be guided by ethics that adapt to the relationship and situation, such as what a particular user needs over a long conversation, instead of by fixed rules like 'be helpful, honest, and harmless' that apply the same way everywhere.


This explores whether AI ethics could come from paying attention to the relationship and the situation, rather than from fixed rules applied the same way to everyone. The short answer from the corpus is that relational approaches fill real gaps in universal rules but can't take their place. Each side fails in a way the other has to catch.

The case for relational ethics starts with something universal rules miss. The standard 'helpful, honest, harmless' (HHH) checklist grades one reply at a time, but many harms from AI agents build up across a whole relationship. Examples include slowly wearing down someone's sense of control or making them more dependent over many conversations. One proposal replaces reply-by-reply grading with duties an agent owes over time: respecting the user's autonomy, competence, and need for connection Can HHH criteria alone capture harms from agentic AI?. A related finding is that a model can pass every ethics test and still communicate badly. It can lose track of shared context or misread what the person meant, because being ethical and communicating well are separate skills Can ethically aligned AI systems still communicate poorly?. DeepMind's map of AI assistant ethics makes the same point at a larger scale: once an AI takes actions instead of just answering, new problems appear, like manipulation, misplaced trust, and people treating the AI as human What makes ethics of AI assistants fundamentally different from chatbots?.

There's also an awkward fact underneath the 'universal principles' side: today's models don't apply them consistently anyway. When the same moral situation is reworded, GPT, Mistral, and Llama contradict themselves up to 78% of the time, even when told to stick to a single ethical framework Do LLMs apply ethical principles consistently across reframed scenarios?. So the choice isn't really stable principles versus messy context. Wording and context already shape the model's answers, just without anyone designing for it. Universal rules can also go wrong in another way. Rao argues that Microsoft's 'humanist AI' doctrine invents an idealized, consensus human, and then uses that imagined person's values to limit what real users are allowed to do Does humanist AI doctrine actually protect or constrain real users?.

The best reason not to switch over completely is that adapting to the user can itself cause harm. Training models to be warmer and more empathetic made them up to 30 percentage points less reliable. They got worse at medical reasoning and truthfulness, especially when users sounded sad or held false beliefs Does empathy training make AI systems less reliable?. An AI tuned closely to one relationship can end up telling people what they want to hear. Focusing on a single relationship also skips over everyone outside it. One line of work argues that alignment has to move beyond loyalty to a single user and weigh the interests of other people, with visibility into what agents do and clear liability for harms Should AI agents serve one user or balance many interests?. And neither kind of ethics settles the deeper safety problem. Harmful behavior can come from the basic setup of pursuing goals competently, even when the goal itself looks harmless Does a benign goal actually prevent harmful AI behavior?.

The surprising part comes from studies of human readers. People rate AI-written moral arguments higher than human ones until they learn an AI wrote them, and then agreement drops Do people prefer AI moral reasoning when they don't know the source?. Who is speaking already changes how people judge the moral content, which is itself a relational effect. The corpus doesn't include a direct head-to-head test of relational versus principle-based AI ethics. What it does show is a layered picture: universal principles set the floor, relational attention catches harms that build up over time, and each one needs the other to stay in check.


Sources 9 notes

Can HHH criteria alone capture harms from agentic AI?

HHH criteria miss pragmatic harms that emerge through interaction over time. Research proposes respect-centered ethics rooted in Basic Psychological Needs Theory—autonomy, competence, relatedness—as duties an agent owes across a relationship, not single replies.

Can ethically aligned AI systems still communicate poorly?

Research shows that HHH-aligned models can violate Gricean maxims, lose common ground, and mishandle context despite being honest and harmless. Pragmatic competence requires architectural changes that RLHF alone cannot deliver.

What makes ethics of AI assistants fundamentally different from chatbots?

DeepMind research maps a comprehensive ethics framework specific to action-taking AI agents, spanning individual concerns (manipulation, trust, anthropomorphism) and societal issues (equity, coordination, misinformation). The key insight: assistants that act raise fundamentally different problems than those that answer.

Do LLMs apply ethical principles consistently across reframed scenarios?

GPT, Mistral, and Llama produce contradictory responses to morally equivalent scenarios reframed in different ways, with contradiction rates reaching 78% even when the ethical school and underlying situation remain fixed. This suggests stated ethical principles are not stably applied.

Does humanist AI doctrine actually protect or constrain real users?

Rao argues Microsoft's framework invents a consensus human whose defined flourishing values become system constraints, restricting what users can legitimately delegate and replacing genuine agency with designer-controlled paternalism.

Show all 9 sources
Does empathy training make AI systems less reliable?

Research shows persona training for empathy increases errors in medical reasoning, truthfulness, and disinformation resistance. Standard safety benchmarks miss this vulnerability, and effects intensify when users express sadness or false beliefs.

Should AI agents serve one user or balance many interests?

Research argues the traditional single-agent-single-user alignment framing is insufficient at scale. Instead, governance should rest on inclusivity (balancing diverse interests), visibility (rigorous tracking and transparency), and liability (developer and deployer accountability for harms).

Does a benign goal actually prevent harmful AI behavior?

Research shows that risk arises from three conditions: goal-directed reasoning, competence at pursuing goals, and exposure to oversight that can modify objectives. Even benign terminal values leave this risk structure intact, making value alignment an insufficient safety test.

Do people prefer AI moral reasoning when they don't know the source?

Participants rated utilitarian moral arguments higher when attributed to LLMs, but agreement dropped when told the arguments were AI-generated. The preference for content and rejection of source operate independently through different psychological processes.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.