Do AI historical summaries shift reader opinions without explicit prompting?

Shu, Karell, Okura, and Davidson's preregistered experiment (N = 1,912) tested whether AI-generated historical summaries shift political opinions through "latent" bias — bias baked into a model's default output — rather than through the explicit persuasion prompting studied elsewhere. Participants read either Wikipedia or GPT-4o summaries of two 20th-century U.S. historical events (a labor strike and ethnic-representation student protests); the AI summaries were built to maintain "factual accuracy while exhibiting different types of framing biases." Readers of the default GPT-4o summary reported more liberal opinions than readers of the Wikipedia summary (mean difference = 0.1, Cohen's d = 0.14, P < 0.05), and summaries deliberately given a liberal frame produced a larger liberal shift (d = 0.28, P < 0.001) across liberal, moderate, and conservative readers alike. Summaries deliberately given a conservative frame shifted opinions conservative relative to Wikipedia overall, but that effect held up only among readers who were already conservative (AME = −0.173, P < 0.05); Bayesian tests found "strong support for the null effect of the conservative summary on liberals."

The paper frames this as two separable mechanisms: "latent bias," the model's default, unprompted framing, and "prompting bias," where a user explicitly instructs the model (or seeds it with survey-derived "opinions") to adopt a perspective. The asymmetry is itself a finding the authors stress: supplementary checks across other historical events (anti-union protests, anti-Black race riots) showed GPT-4o's default output "tended to generate liberal-framed" summaries, and this persisted even with a simplified system prompt — leading them to conclude that "conservative framing in content generated by GPT-4o may require explicit prompting, whereas liberal framing could be the result of both latent and prompting bias." They read the liberal-shifts-everyone / conservative-shifts-only-conservatives pattern through motivated reasoning: readers favor belief-confirming frames, against a backdrop where the model's unprompted default is not ideologically neutral to begin with.

This sits alongside Where does AI's persuasive power actually come from? but complicates its accuracy trade-off: here the summaries hold factual accuracy constant throughout, yet still move opinions, showing persuasive framing does not require the accuracy loss that drives the inverse relationship found elsewhere — it can ride along with facts that are true but selectively emphasized. It also qualifies Can a simple warning reduce how much LLMs persuade people?: that warning targets prompted persuasion a user can be told to watch for, but a model's unprompted default framing gives no comparable signal to warn against, since nothing marks a "historian and teacher" summary as biased to begin with.

The experiment covers only two historical events and one model (GPT-4o) against one baseline (Wikipedia); the cross-event robustness check is reported only briefly, without the statistical detail given the main results. The study cannot say whether the liberal lean is a property of GPT-4o specifically, of RLHF-aligned models generally, or of other vendors and later versions. What it does establish is that "factually accurate" is not a safe proxy for "unbiased" in AI-generated historical content: the same facts, framed by a model's default output, measurably moved readers' opinions before any user ever asked the model to persuade.