Do productivity agents actually protect users better than emotional ones?

Applying its own 14 design principles (see How should agents interact with humans across their relationship lifecycle?) in a heuristic evaluation of nine deployed AI agent systems, a position paper reports that "three general-purpose or productivity-focused agents (ChatGPT +1.14, Claude Chat +0.64, Claude Code +0.43) scored highest on aggregate adherence, while agents operating in emotionally sensitive contexts or high-stake domains (Character.ai −0.36, Consensus −0.31, Pine AI −0.25, and Ash −0.20) scored substantially lower." It adds that "the principles most likely to be violated in these domains—such as dependency prevention (P10) and intent visibility (P5)—are those whose violations carry greater user risk."

The paper's reasoning is that engagement-boosting design choices and relational risk are entangled: "features that increase engagement—such as strong social cues (P6), human-like appearance (P2), and companion-style relational framing (P3)—can amplify user vulnerability if not paired with explicit boundary-setting, transparency, and dependency mitigation." Because the lowest-scoring systems in the evaluation are precisely the ones operating in companion or high-stakes relational contexts, the paper concludes that "principles such as P10 (dependency prevention), P5 (intent visibility), and P1 (expectation setting) should therefore be treated as non-negotiable requirements, incorporated into evaluation checklists, safety reviews, and product constraints" rather than left as general best practice advice applied evenly across all agent types.

This pattern — that user-protective behaviors are unevenly present across deployed systems rather than uniformly built in — parallels How well do AI assistants actually support human agency?, though the two studies measure different things: HumanAgencyBench scores agency-supporting behaviors on a constructed benchmark across developers, while this paper scores principle adherence via heuristic evaluation across nine named products spanning productivity, research, and companion/wellness categories. Both findings converge on the same implication — that whether an agent protects user agency and discloses its reasoning is a product-specific design choice, not an emergent property of capability. The result also sits against What UX principles do workplace users want in AI agents?, which captures what business users say they want from workplace agents (control, safety); this paper's scores suggest that want is least met precisely in the consumer-facing, non-workplace agents built around emotional or advisory relationships.

The excerpt does not establish the heuristic evaluation's methodology in enough detail to assess reliability — it does not say how many raters scored each of the nine systems, whether raters were independent of the paper's authors, what the full scoring scale or its bounds are, or how the nine systems were selected rather than sampled. Nine products is also a small, mixed-category set (general productivity, specialized research/health advisory, and companion apps), not a representative benchmark of the agent market. The direction of the finding — that principle violations cluster in emotionally sensitive and high-stakes domains — is a documented pattern from this one evaluation, not a validated, generalizable claim about all agents in those categories; the implication that follows is narrower: it flags where responsible-AI review should look first, not a settled ranking of agent quality.