INQUIRING LINE

Tools that nudge people to double-check AI answers mostly help those who already like to think hard — can they be redesigned to help everyone else too?

Can forcing functions be redesigned to help low-motivation participants too?

This explores whether 'cognitive forcing functions' (designs that make people stop and think before accepting an AI's answer) could be reworked so they also help people who don't enjoy effortful thinking, not just those who already do.


This explores whether cognitive forcing functions (small pieces of friction, like making you commit to your own answer before you see the AI's) can be redesigned so they help everyone, not only people who like to think hard. The short answer: the corpus documents the problem clearly but has no tested redesign. What it does have are some useful clues about why the problem exists.

The problem first. In a 199-person experiment, forcing functions cut overreliance on AI on average, but most of the gain went to people high in 'Need for Cognition,' the trait of enjoying effortful thinking Do cognitive forcing functions help all decision makers equally?. So an intervention meant to protect people from blindly trusting AI ended up widening the gap between careful thinkers and everyone else. The same study found a second problem: participants rated the forcing designs least favorably of all the conditions, even though they worked best, and human+AI teams still did worse than the AI alone Do forcing functions reduce AI overreliance without harming team performance?. Together these findings point to the core design tension. Friction works only if people engage with it, and the people least inclined to engage are the ones who most need the help and who most dislike the friction.

A lateral clue comes from health chatbots. When three major LLMs were tested on behavior-change conversations, they did well with users who already had clear goals but failed to notice ambivalence or low readiness to change Why can't chatbots detect when users are ambivalent about change?. That is the same failure in a different setting: support built for the already-motivated quietly fails everyone else. It suggests one direction for a redesign. Forcing functions today apply the same friction to everyone. A better version might first figure out where a person is (engaged, skeptical, or just trying to get through the task) and adjust. But that study also shows current models are poor at reading those states, so 'just make it adaptive' is not a solved problem.

A more speculative angle comes from research on training AI models, where scalar scores turn out to carry only part of the useful signal. Feedback contains an evaluative part (how well did that go?) and a directive part (what should change?), and these do different jobs Can scalar rewards capture all the information in agent feedback?. Models stuck on a plateau improve when they're told why they failed rather than simply that they failed Can natural language feedback overcome numerical reward plateaus?. Treat this as an analogy, not evidence about people. Still, a typical forcing function only makes you pause, which is a kind of bare evaluative signal. For someone who won't generate their own reasoning during that pause, a nudge that names what to check might do more than a blank 'think first' gate.

The surprise worth taking away: making people think harder is not neutral. It can become an inequality machine, rewarding the people who were already doing fine. Whether a redesign can fix that is still an open question in this collection.


Sources 5 notes

Do cognitive forcing functions help all decision makers equally?

An experiment (N=199) found that forcing functions helped participants high in Need for Cognition significantly more than those low in it, creating intervention-generated inequality despite aggregate benefits.

Do forcing functions reduce AI overreliance without harming team performance?

A 199-participant study found that cognitive forcing functions significantly cut AI overreliance compared to explainable AI, yet human-AI teams still underperformed the AI alone. Participants rated these friction-inducing designs least favorably despite their effectiveness at reducing anchoring.

Why can't chatbots detect when users are ambivalent about change?

Testing three major LLMs across 25 health scenarios showed they succeed only when users have established goals but cannot detect resistance or ambivalence. Models miss relapse-prevention strategies even for users in action stages.

Can scalar rewards capture all the information in agent feedback?

Natural feedback carries two orthogonal types of information: evaluative (how well an action performed) and directive (how it should change). Scalar rewards capture evaluation but discard directional specifics that token-level distillation can recover, making the two complementary rather than redundant.

Can natural language feedback overcome numerical reward plateaus?

Critique-GRPO shows that models stuck on performance plateaus can generate correct solutions when given chain-of-thought critiques, revealing that numerical rewards lack critical information about why failures occur and how to improve.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.