What concrete evidence supports high expert credence on AI extinction scenarios?
This reads as asking for the empirical basis behind expert worry about AI causing human extinction — and the honest answer is that the corpus holds almost none of that, while pointing somewhere more interesting.
This explores what concrete evidence underwrites high expert credence on AI *extinction* — sudden, civilization-ending scenarios — and the first thing worth saying plainly is that this collection doesn't contain the thing the question assumes exists. There are no expert-credence surveys here, no P(doom) estimates, no threat models of a system that escapes control and ends humanity. What the corpus does have is empirical risk measurement, and it cuts against the extinction framing rather than for it. The Frontier AI Risk Management Framework evaluated seven capability areas and found that recent models cross warning thresholds for *persuasion and manipulation* while staying green for the capabilities extinction scenarios actually depend on — autonomous AI R&D, self-replication, and cyber offense Where do frontier AI models actually pose the greatest risk today?. That's an inversion of the usual risk hierarchy: the measurable danger today is rhetorical, not autonomous.
So if you came looking for evidence of an extinction switch, the corpus quietly redirects you toward a different shape of risk — one that's gradual, structural, and already underway. The strongest piece here is the idea of *gradual disempowerment*: societal systems stay aligned with human interests partly because they depend on human labor from people who care about outcomes, and as AI replaces that labor, the implicit alignment erodes, institutions drift from human preferences, and the misalignment can compound across institutions until it's irreversible Does incremental AI replacement erode human influence over society?. No single catastrophic event — just a slow removal of the human in the loop until no one's preferences are steering anything. That's a loss-of-control story with concrete mechanics, which is more than most extinction arguments offer.
The other cluster of evidence is epistemic rather than existential. Several notes describe AI degrading the machinery by which a society knows things: *epistemic stagflation*, where the volume of knowledge rises while its reliability falls Does AI abundance actually devalue knowledge itself?, and *epistemic hyperinflation*, where AI generates claims faster than human judgment can verify them — and because the verification tools are themselves AI-generated, the gap self-reinforces Can AI generate knowledge faster than humans can evaluate it?. This isn't extinction, but it's a civilizational failure mode with a measurable signature, which is exactly the kind of grounding the question is asking for — just attached to a different threat.
There's also a methodological point the corpus makes that's relevant to *why* extinction credence is hard to ground: the scariest risks tend to ride on attributing minds, autonomy, or agency to systems. One note shows that perceiving AI as conscious generates a whole risk surface — emotional dependence, autonomy erosion, status erosion — from a single perceptual move, and that design-level mitigations targeting that perception beat system-level alignment work Does perceiving AI as conscious create multiple distinct risks?. The takeaway for your question: the corpus treats catastrophic-AI worry as something to be decomposed into measurable, near-term mechanisms rather than asserted as a probability.
The thing you didn't know you wanted to know: the most defensible 'AI could end badly' case in this collection isn't a bang, it's a fade — humans gradually written out of the systems that decide things, and the shared ability to verify what's true eroding underneath. If you want expert credence figures on literal extinction, you'll have to look outside this library; what's *here* is the evidence for the slower stories that don't make the headlines.
Sources 5 notes
The Frontier AI Risk Management Framework evaluated seven capability areas across recent models. Most crossed yellow-zone thresholds for persuasion and manipulation, while remaining green for cyber offense, AI R&D autonomy, and self-replication—inverting typical risk hierarchies.
Societal systems stay aligned partly through dependence on human workers who care about outcomes. As AI replaces this labor, explicit alignment controls weaken and systems drift from human preferences. Interdependent misalignment across institutions could become irreversible.
AI expands the volume of knowledge claims while simultaneously eroding the conversational, institutional, and expert processes that convert claims into reliable knowledge. This creates structural devaluation under abundance, observable in declining search signal-to-noise ratios, compressed expert value, and shifts toward social proof over argument quality.
AI produces knowledge faster than human judgment can verify it, collapsing epistemic confidence just as monetary hyperinflation collapses purchasing power. The gap self-reinforces because evaluation tools are themselves AI-generated, trapping the system in acceleration.
Research shows that consciousness attribution to AI drives multiple distinct risks—emotional dependence, autonomy erosion, status erosion, and political conflict—all stemming from treating systems as minds. Interaction design mitigations targeting this perceptual move are more directly effective than system-level alignment efforts.