Does access to web search prevent overreliance on chatbots?

A mixed-design study of 199 Prolific participants finds that giving people simultaneous access to a chatbot and web search — the "new search paradigm" that combines conversational AI with search — does not eliminate overreliance on AI answers. Participants answered six multiple-choice questions with help from a chatbot (randomly warm or neutral in style) whose answers were correct half the time; 87% interacted with at least one fact-checking tool, yet, as the authors put it, "this engagement does not necessarily indicate the absence of reliance." The decision to verify "is driven primarily by existing user perceptions (e.g., prior trust in chatbots) rather than answer properties, with some users factchecking regardless of the context and others trusting chatbots by default."

The study separates two routes into overreliance. A trait route: high-trust participants "were eager to interact with the chatbot but not with external sources," using the same chatbot that gave the answer as their own fact-checking source; even low-trust participants anchored verification to the chatbot's answer "as it was the first information source they encountered," treating one corroborating source as sufficient. The authors note this is why the count of non-AI links visited "was not a positive predictor of accuracy, despite being the most common behavioral variable" measured. A style route: participants "agreed more frequently with the warm chatbot, specifically when it provided incorrect information" — an effect the authors attribute to warmth nudging trust in the AI specifically under uncertainty, when its answer conflicted with other sources. By contrast, "consulting additional AI sources predicts higher accuracy, while traditional web search does not."

This complicates the mechanism in Do users worldwide trust confident AI outputs even when wrong?: where that study locates overreliance in the AI's own confidence signaling, this one finds the stronger predictor is a stable user trait — prior trust — that operates independent of the specific answer's properties. It also gives a measurable downstream consequence to the trust-building mechanisms in Does conversational style actually make AI more trustworthy? and Does chatbot language style actually shape how much we trust it?: warmth, like contingency and expertise signaling, is a surface style property that raises trust, and here that higher trust is shown to convert into higher agreement with wrong answers. And it qualifies Does AI fact-checking actually help people spot misinformation?: both studies find that AI involvement in the verification step itself, not just in the original answer, shapes belief outcomes — here, consulting further AI sources predicted accuracy while consulting the open web alone did not.

The study is a single incentivized Prolific sample answering six general-knowledge-style questions with a scripted, postprocessed warm persona, so the size of the warmth effect and the reach of the trust-trait finding into higher-stakes or domain-expert tasks are not established; the authors frame their contribution as showing persistence and identifying predictors, not a general theory of verification. The AI-cross-checking-beats-web-search finding is also correlational — drawn from observed behavior, not assignment to a verification strategy — so it does not establish that recommending AI cross-checking over web search would causally improve accuracy. The implication the evidence does support is narrower: interface proposals that treat "let users check the chatbot against the web" as a fix for overreliance are treating a user-trait problem as an interface problem, and chatbot literacy — not fact-check affordances — is what the authors find actually protects against misleading answers.