How latent and prompting biases in AI-generated historical narratives influence opinions

Source: Shu, Karell, Okura, Davidson (Yale), PNAS Nexus · 2026-03-03

Large language models (LLMs) can be used to persuade people on a range of issues, particularly through user-driven strategies such as personalizing messages and dialogues intended to change minds. However, their capacity to influence opinions through subtle, latent ideological framing remains relatively understudied. We investigate whether AI-generated historical summaries affect social and political opinions through a preregistered experiment (N = 1,912). Participants read Wikipedia or GPT-4o summaries of two historical events, with AI summaries maintaining factual accuracy while exhibiting different types of framing biases. Default AI summaries led to more liberal opinions compared with Wikipedia, demonstrating the persuasive capability of LLM's latent biases. Summaries purposefully induced with a liberal framing also led to more liberal opinions, regardless of readers’ ideologies. Summaries constructed with a conservative framing produced conservative shifts primarily among conservative readers. These findings demonstrate that the use of AI for learning history can influence opinions through both intrinsic and intentional framing mechanisms, even when the content remains factually accurate. As AI becomes integral to information acquisition, recognizing pathways of influence based not only on user-manipulated content but also on models’ latent biases is essential for understanding AI's broader societal impacts.

The rapidly improving capability of large language models (LLMs) to produce human-like text, images, and video has caused widespread concern about their potential to spread inaccurate information, conspiracy theories, and disinformation (1, 2). A growing body of research thus examines how exposure to LLMs can affect people's views, ranging from beliefs in blatant falsehoods to opinions about legitimate policy debates (2). Much of this work has focused on content intentionally crafted by LLM users to persuade, including propagandistic statements, personalized arguments, and conversations with AI agents (2–8).

This research overlooks a less overt but potentially significant mechanism of persuasion: the underlying, latent biases in LLMs. Biases can be introduced at multiple stages of the LLM training process, resulting in ideological leanings that can be difficult to remove (9–11). In some cases, these biases are unintentional; in other cases, developers can guide models to express particular perspectives. For example, xAI's Grok chatbot generated racist and anti-antisemitic material on X after its instructions were altered to permit “politically incorrect” claims.a To be sure, most latent biases, whether intentional or unintentional, do not yield such clearly prejudiced content. Rather, they can produce outputs that appear neutral but convey subtle nuances or emphases that potentially influence opinions (10). We examine this more elusive mechanism of AI persuasion, latent bias, and compare its effect to the more commonly studied prompting bias, where users explicitly employ or instruct AI to convey a particular perspective (2, 4–8).

Whereas prior work on prompting bias has focused on political and policy contexts (2, 5, 6, 8, 10) or outright misinformation (1, 4, 7), our empirical application examines the use of LLMs to elicit facts. Specifically, we consider historical events, since learning about history informs attitudes and policy preferences (12), people are increasingly using AI tools like ChatGPT and Google's AI Overview to obtain historical information (13),b and private companies are using AI to produce alternative information sources, such as xAI's “Grokipedia.”c Our analyses test how both AI's latent and user-prompted framing of factually accurate historical content can affect individuals’ opinions about social and political issues.

We begin our study by evaluating latent biases. We compare default GPT-4o and Wikipedia summaries of two events that took place in the United States during the 20th century: a large labor strike and student protests calling for greater ethnic minority representation in academia. Following recent research on AI persuasion (6), the default summaries are generated by instructing the model to take on the role of a relevant expert (i.e. a “historian and teacher”). To assess the relative influence of latent and prompting biases, we compare similar summaries that purposefully frame the events in politicized ways. This prompting bias is introduced by following recent work that instructed LLMs to adopt a politicized perspective, or “role-play” (8). Namely, we use survey data to seed the model with “opinions,” yielding summaries that portray the same historical events with either liberal or conservative framing. The first two research questions are thus:

RQ1. What is the effect of default-framed AI summaries of historical events on readers’ opinions, compared to Wikipedia summaries?

RQ2. What is the effect of inducing ideological framing in AI summaries of historical events on readers’ opinions, compared to Wikipedia summaries?

People filter information through their own ideologies and belief systems. Motivated reasoning theory suggests a tendency to favor belief-confirming information (14), potentially leading to reinforcement from ideologically aligned frames. Meanwhile, encountering counter-attitudinal views might provoke defensive reactions and backlash (15). Therefore, we additionally examine heterogeneous effects of the AI summaries across readers’ ideologies.

RQ3. How do the AI summaries’ effects vary based on the correspondence between readers’ political ideology and the summaries’ ideological framing?

We address RQ1 and RQ2 by comparing the opinions of readers of AI summaries to the opinions of those who read the Wikipedia summaries. Figure 1 shows that both the AI summaries with default framing and the AI summaries with liberal framing led to more liberal opinions compared to the Wikipedia summaries (Default: mean difference = 0.1, Cohen's d = 0.14, P < 0.05; Liberal: mean difference = 0.21, Cohen's d = 0.28, P < 0.001). Meanwhile, readers of AI summaries with conservative framing reported more conservative opinions relative to Wikipedia (mean difference = −0.11, Cohen's d = −0.13, P < 0.05). Similar results were observed when analyzing each event separately.

The effect sizes are similar in magnitude to those in prior research (6). On the response scale, average opinions after reading each summary fell between 3 (moderate) and 4 (somewhat liberal). The average opinion after reading either the default (3.57) or liberal AI text (3.67), compared with the average opinion after reading the Wikipedia text (3.47), represents a shift from a moderate stance to leaning towards a somewhat liberal stance. The average opinion after reading the conservative AI text (3.36) indicates a slight shift towards more moderate opinions.

To examine whether readers’ political ideology moderated how politically framed AI summaries affected their opinions (RQ3), we conditioned our preceding analysis on self-reported ideology. The average marginal effects (AMEs) across reader ideology in Fig. 2 show that the default AI summaries did not affect readers’ opinions differently from the Wikipedia summaries for liberal, moderate, or conservative participants. In contrast, the liberal-framed AI summaries led to more liberal opinions for participants across all ideology groups (liberals: AME = 0.205, P < 0.05; moderates: AME = 0.177, P < 0.05; conservatives: AME = 0.190, P < 0.05). The AI summaries with conservative framing resulted in more conservative opinions, although this effect was statistically significant (α = 0.05) only for those who already had a conservative outlook (conservatives: AME = −0.173; P < 0.05). We conducted Bayesian analyses to evaluate the support for the null findings and found moderate support for all null results and strong support for the null effect of the conservative summary on liberals (see Section G of the Supporting Information [SI] document).

Our results suggest that using popular AI tools to learn about history can influence people's opinions on social and political issues. Default GPT-4o-generated summaries of historical events made readers’ opinions more liberal compared with Wikipedia summaries, demonstrating the persuasive effects of LLMs’ latent biases.

While the experiment tested only two events, supplementary analyses showed that GPT-4o tended to generate liberal-framed default summaries for a wide range of similar events, including anti-union protests and anti-Black race riots. This framing persisted even when we used a simpler system message that did not instruct the model to take on the role of a relevant expert to generate the summaries (see Section L the SI). This suggests that conservative framing in content generated by GPT-4o may require explicit prompting, whereas liberal framing could be the result of both latent and prompting bias.