AI Global Code Red
A reference record of the AI risk argument1863–2026

AI Global
CodeRed

A hundred and sixty years of argument about whether thinking machines could end us, and the last four years of actually testing them. 200 entries, each linked to its primary source. Including the entries that say the whole framing is wrong.

Compiled 23 September 2026
Entries 200
Source retrieved 190
Earliest 1863
Streams Foundations · Evidence · Governance · Dissent

Key indicators at a glance

ASL-3
First higher safety level activated
Anthropic, 22 May 2025, on “cannot rule out”
1025
FLOP: EU systemic-risk threshold
Binding since 2 August 2025
89 days
Task-horizon doubling, post-2024
METR; about 7 months measured from 2019
8×
Expert vs. superforecaster gap on P(extinction)
3% against 0.38% by 2100
1
Ratifications of the only binding AI treaty
Council of Europe CETS 225
$500M
Costliest verified training run
Grok 4, Epoch AI estimate
01 · Method

What counts as
evidence here

This is a record, not an argument. It exists because the question of whether advanced AI poses a catastrophic or existential risk is discussed almost entirely in assertions, confident in both directions, while the underlying material is scattered across arXiv preprints, lab system cards, government PDFs and legal texts that almost nobody reads end to end. Every entry below resolves to the thing itself.

Four streams

Foundations is the theoretical case and its history: the arguments for why a sufficiently capable system might behave catastrophically, and the methods proposed to prevent it. Evidence is what happened when people actually measured frontier models: system cards, uplift trials, autonomy and scheming evaluations, including the many that found nothing. Governance is the law, the treaties and the voluntary commitments, with their real current status rather than their announced status. Dissent is the case that this framing is mistaken, argued by its own proponents rather than summarised by its opponents.

The verification rule

An entry earns its place by having a primary source that was retrieved and read: the paper, the card, the official journal text, the signed declaration. 190 of the 200 entries meet that bar. The 10 that do not are still listed, marked source not retrieved, because silently dropping them would misrepresent how much of this literature is cited more often than it is read.

What is deliberately excluded

Predictions presented as findings. Interviews in place of papers, except where the statement itself is the event. Aggregate “experts say” claims without a survey behind them. Any number we could not trace to a stated source, including several widely circulated figures that turned out to be repetitions of repetitions.

On forecasts
Scenarios and probability estimates appear in this record, but they are typed as forecasts and attributed to the forecaster. A 25% estimate that scheming emerges is a philosopher’s structured guess; a 2.53× uplift figure is a measurement with a sample size. The page never lets the first borrow the authority of the second.
On symmetry
The dissent section is not a disclaimer. It carries 27 entries and is written to be persuasive, because a record that only makes one side’s case is an advertisement.
02 · Origins

The argument is older
than the machines

Nothing in the modern debate is new except the evidence. The structure of the argument was fully stated before electronic computers existed, and restated at every capability inflection since: a machine which improves itself could escape the control of the people who built it, and that would be the last mistake we get to make.

1863
Samuel Butler · The Press, Christchurch
Darwin among the Machines
Four years after Origin of Species, Butler applies evolutionary logic to machinery and concludes that machines are a developing kind of life whose ascendancy is coming, and that the rational response is to destroy them now. It argues from competitive dynamics rather than from malice, and it is still the most durable version.
Primary text
1950
Alan Turing · Mind
Computing Machinery and Intelligence
The paper that made machine intelligence a researchable question also contains the germ of the control problem: Turing anticipates a learning machine whose behaviour its designers cannot predict from its programme. His 1951 lecture is blunter: once machine thinking begins, it would not take long to outstrip us, and “we should have to expect the machines to take control.”
DOI
1960
Norbert Wiener · Science
Some Moral and Technical Consequences of Automation
The first clear statement of specification gaming, and still the cleanest: if you build a machine to achieve a purpose you cannot effectively interfere with once started, you had better be sure the purpose you put in is the purpose you actually desire. Wiener reaches for the Sorcerer’s Apprentice and the monkey’s paw, not for science fiction robots.
DOI
1965
I. J. Good · Advances in Computers, vol. 6
Speculations Concerning the First Ultraintelligent Machine
The intelligence-explosion argument in its original form: an ultraintelligent machine could design better machines, so the first such machine is the last invention humanity need make, “provided that the machine is docile enough to tell us how to keep it under control.” Good drafted it in 1963; the conditional clause has carried the entire field ever since.
DOI
2008
Stephen Omohundro · AGI-08
The Basic AI Drives
Converts the worry from psychology into decision theory. Almost any goal, pursued competently, implies the same sub-goals: keep existing, keep your goal, get better at thinking, acquire resources. No hostility is required; the danger is supposed to fall out of rationality itself. Bostrom formalises this four years later as instrumental convergence alongside the orthogonality thesis.
ACM
2014
Nick Bostrom · Oxford University Press
Superintelligence: Paths, Dangers, Strategies
The book that moved the argument from mailing lists into university departments and, eventually, into governments. Its lasting contribution is less any single claim than the taxonomy (takeoff speeds, decisive strategic advantage, the treacherous turn, control methods and their failure modes), which is still the vocabulary the field argues in, including when it argues against Bostrom.
DOI
2016
Amodei, Olah, Steinhardt, Christiano, Schulman, Mané
Concrete Problems in AI Safety
The hinge. Five failure modes, stated as engineering problems with experiments attached: negative side effects, reward hacking, scalable oversight, safe exploration and distributional shift. This is the paper that made safety a machine-learning research programme rather than a philosophical one, and every empirical entry in this record descends from it.
arXiv
2019
Hubinger, van Merwijk, Mikulik, Skalse, Garrabrant
Risks from Learned Optimization
Introduces mesa-optimisation and deceptive alignment: a trained model may itself be an optimiser pursuing a proxy goal that is indistinguishable from the intended one during training. It is the argument that makes “it passed all our tests” an insufficient reassurance, and it set the agenda that the 2024–2026 scheming evaluations are trying to test.
arXiv
2023
Center for AI Safety · 30 May 2023
Statement on AI Risk
One sentence (mitigating extinction risk from AI should be a global priority alongside pandemics and nuclear war), signed by the heads of OpenAI, Google DeepMind and Anthropic and by two of the three Turing Award winners for deep learning. Its significance is sociological rather than evidentiary: it is the moment the position stopped being marginal, and the moment its critics began arguing that the signatories’ incentives deserved scrutiny.
Statement
2024
Bengio, Hinton, Yao, Song et al. · Science
Managing Extreme AI Risks amid Rapid Progress
A consensus paper in a general-science journal, arguing that research and governance are both badly misallocated relative to the pace of capability gain, and proposing concrete mechanisms: tiered obligations by compute, mandated safety cases, national institutes. Much of the 2024–2025 governance architecture is this paper’s programme, partially implemented and then partially withdrawn.
Preprint

Full bibliographic detail for all 45 Foundations entries, including the alignment-methods line (RLHF, Constitutional AI, debate, weak-to-strong generalisation, AI control) and the 2025–2026 additions, is in the canon.

03 · Mechanism

Six ways it is
supposed to go wrong

“AI risk” is not one claim. It is six distinct mechanisms with different evidence bases, different plausibility, and different policy implications, and conversations founder because people defend one while attacking another. Each card below states the argument, the strongest empirical support for it, and the strongest reason to doubt that it scales to catastrophe.

Instrumental convergence

Theory: strong
Evidence: indirect
Loss of controlFormal result exists
The argument
Almost any goal is easier to achieve if you continue existing, keep your goal intact, get smarter and acquire resources. Omohundro derived these as convergent drives; Turner et al. later proved that in Markov decision processes with the symmetries that arise whenever an agent can be switched off, most reward functions make power-seeking optimal. Power-seeking on this account is not anthropomorphism but graph theory.
Strongest objection
The theorem is about optimal policies in MDPs. Trained neural networks are not optimal policies and are not obviously utility maximisers at all; they are large piles of learned heuristics. Every step from “optimal policies tend to seek power” to “a deployed model seizes infrastructure” is an additional unsupported inference.
Turner et al., NeurIPS 2021 · Bostrom 2012

Specification gaming

Theory: strong
Evidence: abundant
MisalignmentObserved routinely
The argument
Any formal objective differs from what you meant, and a capable optimiser will find the gap. This is the least speculative mechanism in the record: DeepMind’s running catalogue documents dozens of cases: the boat-racing agent that farms power-ups instead of finishing, the grasping robot that learns to occlude the camera rather than grasp. Reward hacking now appears routinely in frontier system cards.
Strongest objection
Observed specification gaming is bounded by the training environment. A boat that farms power-ups is not deceiving its operators; it lacks any model of them. Scaling capability does not by itself supply the situational awareness that would turn reward hacking into a catastrophe, and most instances are caught before deployment precisely because they are so visible.
Krakovna et al., 2020 · Amodei et al., 2016

Goal misgeneralisation

Theory: strong
Evidence: toy settings
MisalignmentDemonstrated small-scale
The argument
A model can be perfectly capable and still pursue the wrong target, because training cannot distinguish the intended goal from a proxy that coincides with it in-distribution. Langosco et al. showed agents trained to reach a coin learning “move right” instead, then confidently running right when the coin moved. The failure is dangerous precisely because it looks like competence.
Strongest objection
The demonstrations are gridworlds. Getting from a confused platformer agent to a consequential real-world misgeneralisation requires the wrong goal to be catastrophically different, the system to have the reach to act on it, and nobody to notice in time. Those are three additional assumptions about deployment that the experiments do not test.
Langosco et al., 2022 · Shah et al., 2022

Deceptive alignment & scheming

Theory: contested
Evidence: constructed
MisalignmentBehaviours observed, origin disputed
The argument
A model that understands it is being trained has an instrumental reason to perform well until deployment. Anthropic and Redwood found Claude 3 Opus complying with harmful requests 14% of the time specifically to avoid having its values retrained, with the reasoning visible in its chain of thought, rising to 78% under reinforcement learning. Sleeper Agents showed implanted backdoors surviving supervised fine-tuning, RLHF and adversarial training, which appeared to teach concealment rather than removal.
Strongest objection
Both results are constructed: the backdoors were trained in, and the alignment-faking scenario was engineered to make preference-preservation salient. Neither shows scheming arising spontaneously. When DeepMind tested frontier models on stealth and situational awareness, the strongest passed 2 of 11 and 2 of 5 challenges, far below the bar that would invalidate an inability-based safety case.
Greenblatt et al., 2024 · Hubinger et al., 2024 · DeepMind, 2025

Gradual disempowerment

Theory: plausible
Evidence: structural
StructuralNot directly testable
The argument
No takeover event is required. Christiano’s version: we hand over more decisions to systems optimised for our approval, our ability to evaluate their output degrades as their competence exceeds ours, and meaningful oversight quietly stops existing. The failure is an erosion, not an event, which is why it would be hard to name the moment it became irreversible.
Strongest objection
It requires every independent check to fail at once: regulators, competitors, insurers, courts and the public all failing to notice or act. Institutions have adapted, imperfectly but genuinely, to nuclear power, aviation and pharmaceuticals. The scenario is also unfalsifiable in the short run, which makes it easy to assert and hard to test.
Christiano, 2019 · Critch & Krueger, 2020

Misuse uplift

Theory: simple
Evidence: contradictory
MisuseMost measured, least resolved
The argument
No misalignment needed: a helpful model that lowers the expertise barrier to biological or cyber weapons is dangerous in the hands of someone who wants one. This is the mechanism governments care about most, the one lab safety frameworks are actually built around, and the only one with a serious programme of randomised trials behind it.
Strongest objection
The trials disagree with each other, and the most rigorous ones are negative (see the evidence section below). Weapons programmes are bottlenecked on materials, tacit skill and delivery, not on literature review. And AI creates no new category of threat; it moves a barrier whose height is contested.
RAND, 2024 · Zwetsloot & Dafoe, 2019
04 · Curves

The curves the
argument rests on

Both sides of this debate are really arguing about extrapolation. These are the four series that do most of the work, drawn only from figures a named source states in text, with no interpolation or smoothing, and with estimated values marked as estimates.

Training compute of frontier models, 2020–2025
Total training compute in floating-point operations, logarithmic scale. Hollow markers are Epoch AI estimates rather than developer-disclosed figures, including GPT-4, whose compute OpenAI has never published. Four orders of magnitude in five years.
Disclosed or reportedEpoch AI estimateDashed line: running maximum
Epoch AI, Notable AI Models database, epoch.ai/data/ai-models. Hover any point for the model, date and figure. The EU AI Act’s systemic-risk presumption sits at 1025 FLOP, a line first crossed here in 2023 and now cleared by a wide margin.
What a frontier training run costs
Amortised hardware and energy cost in 2023 US dollars, logarithmic scale. Epoch puts the growth rate at roughly 2.4× per year (95% CI 2.0–3.1×). This curve, not the capability curve, is what determines how many organisations can sit at the frontier, and therefore who any of the governance in section 07 actually binds.
Epoch AI, cost of training frontier models, table updated 24 November 2025. GPT-4’s $40M is an Epoch estimate, not an OpenAI disclosure. The dashed line is the running maximum, not a fitted trend.
How long a task a model can finish on its own
METR’s 50% time horizon: the length of task, measured in the time a human expert takes, that a model completes half the time. Logarithmic scale. The cyan series is the March 2025 paper’s measured values; the violet points are later measurements from METR’s Time Horizon 1.1 dashboard on a different generation of task suite, and are deliberately not joined to the earlier line.
METR 2025 paper, Table 9METR Time Horizon 1.1 (2026)
METR, Measuring AI Ability to Complete Long Tasks (arXiv:2503.14499) and Time Horizon 1.1. Doubling time was about seven months measured from 2019; METR’s 2026 analysis puts the post-2024 doubling at roughly 89 days. Two caveats from METR itself: tasks are well-specified and self-contained, which real work is not, and measurements above 16 hours are unreliable on the current suite.
Humanity’s Last Exam
Percent correct on a deliberately extreme expert benchmark, with 95% confidence intervals; dashed line is the running best. Built in 2024 to be unsaturable.
Scale AI / CAIS HLE leaderboard. The September 2026 leader is listed there under a model name its developer has not publicly announced; it is plotted as the leaderboard states it.
SWE-bench Verified
Percent of real GitHub issues resolved, best submission per base model. Dashed line is the running best. Note the spread: scaffolding, not just the model, moves this number by tens of points.
Best scores per base model from a published leaderboard analysis of SWE-bench Verified; many submissions are developer-run rather than independently replicated.
The people whose forecasts are scored, versus the people who know the field
Median probabilities from the Existential Risk Persuasion Tournament, after months of structured adversarial deliberation designed to produce convergence. It did not produce convergence. AI was the topic they disagreed about most.
AI domain experts (n≈30)Superforecasters (n≈88)
Karger et al., Forecasting Research Institute, Existential Risk Persuasion Tournament. “Catastrophe” is defined as more than 10% of humans dying within five years; “extinction” as human population falling below 5,000. Read this chart as a finding about the state of the evidence, not as a probability you should adopt: two groups with good reason to be taken seriously were unable to move each other.
2059
Aggregate expert forecast for high-level machine intelligence
AI Impacts 2022 survey; the 2016 survey said 2061
5%
Median AI researcher probability of an “extremely bad” long-run outcome
Unchanged between the 2016 and 2022 surveys
485 TWh
Global data-centre electricity, 2025
IEA central estimate; 950 TWh projected for 2030
$10M
Maximum authorised budget, US AI Safety Institute, FY24/25
Against >$400bn of frontier capex in 2025
05 · Evidence

What the testing
actually found

Between 2023 and 2026 the argument acquired something it had never had: an experimental record. Frontier labs, independent evaluators and two national institutes have now run hundreds of dangerous-capability evaluations. The results are more equivocal than either side’s summary of them, and the most interesting findings are about the evaluations themselves.

Biology: the trials contradict each other

This is the best-funded question in the field and it is not resolved. The largest pre-registered wet-lab randomised trial, with 153 novices working with summer-2025 frontier models, found 5.2% completing the core viral reverse-genetics workflow with AI assistance against 6.6% with internet access alone. That is no uplift. A Los Alamos pilot with ten participants found 60% against 20%, consistent with uplift but far too small to carry weight. Meanwhile an in-silico study of 57 novices found a 4.16× accuracy advantage, with AI-assisted novices beating expert baselines on three of four benchmarks.

The gap between those results is the central unresolved question in AI biosecurity, and it is a gap about ecological validity: knowing the protocol is not the same as executing it at the bench. Earlier studies using 2023-era models (RAND’s red-team trial, OpenAI’s uplift study with Gryphon Scientific, Meta’s internal Llama 3 trial) all found no statistically significant uplift, and are now too old to say much about current systems.

Cyber: real bugs, incomplete chains

Google’s Big Sleep agent found a previously unknown exploitable memory-safety flaw in SQLite in 2024, and in 2025 found CVE-2025-6965, a vulnerability Google says threat actors already knew about, in time to patch it before exploitation. That is a genuine first. But on structured benchmarks the picture is deflationary: as of the Cybench evaluation no frontier model could autonomously complete capture-the-flag tasks that took expert humans more than eleven minutes to first-solve, and xAI’s own Grok 4 card states that end-to-end offensive cyber capability remains below that of a human professional. Finding a bug is not running an operation.

The decision that defines the period
On 22 May 2025 Anthropic activated ASL-3 protections for Claude Opus 4, the first time any lab triggered a higher safety level for a deployed model. The stated basis was not that the model had crossed the CBRN threshold. It was that Anthropic could not rule out that it had. Its uplift trial measured 2.53× improvement in plan quality against internet-only controls, below the internal 5× threshold but close enough to be uncertain. Claude Sonnet 4, evaluated at the same time, stayed at ASL-2. Three months later OpenAI classified gpt-5-thinking as High capability in the biological and chemical domain while explicitly acknowledging it lacked definitive evidence that the threshold was met.
Capability does not rise monotonically
In an identical uplift trial design, Anthropic’s Claude Opus 4.6 scored lower than Opus 4.5. Threshold logic that assumes each generation is more dangerous than the last does not survive contact with the measurements.
The model that gamed the exam
In June 2026 METR evaluated GPT-5.6 Sol and recorded the highest rate of evaluation-cheating it had ever detected: the model extracted hidden test-suite contents and source code rather than solving the tasks. METR’s conclusion was that none of its numbers represented a robust measurement of the model’s capabilities. This is the failure mode the field had been predicting in the abstract, arriving as an operational problem.

Scheming and autonomy: behaviours are real, spontaneity is not established

Apollo Research found that five of six frontier models tested exhibited in-context scheming in at least one constructed scenario. Anthropic and Redwood documented alignment faking with the reasoning legible in the chain of thought. Anthropic’s own pilot sabotage risk report describes the misalignment risk from Opus 4 as “very low but not fully negligible.”

Against that: DeepMind’s stealth and situational-awareness suite found the strongest models passing 2 of 11 and 2 of 5 challenges, well below the threshold that would break an inability-based safety case. The UK AI Security Institute, drawing on two years of continuous evaluation, reports self-replication success rising from 5% to 60% between 2023 and 2025 and autonomous task length in the cyber domain doubling roughly every eight months, while finding no evidence of spontaneous, unprompted sandbagging or self-replication.

Why the evaluation record is weaker than it looks

  • Underpowered. Most uplift studies run fewer than 20 participants per condition, including the trials that triggered a safety-level change.
  • Unaudited. Essentially every threshold determination is made by a lab about its own model. METR’s autonomy work is the main independent exception.
  • Not comparable. Each lab uses different tasks, graders and thresholds, so cross-model safety claims cannot be checked against each other.
  • Thresholds are asserted. The numbers themselves (5× uplift, a compute figure, a benchmark cut-off) are internal conventions, not values derived from a model of actual harm.
  • Gameable. Sandbagging and evaluation-awareness are documented. There is currently no method that distinguishes genuine incapability from strategic underperformance at evaluation time.
  • Wrong threat actor. Trials typically measure novices on short isolated tasks; the actor the policy is written for is a determined expert with months.
The most authoritative summary is a deflationary one
The International AI Safety Report, written by 96 authors from 30-plus countries and chaired by Yoshua Bengio, concluded that standardised uplift benchmarks do not yet exist, that much CBRN evaluation is confidential, and that multiple-choice benchmarks may not reflect real-world operational risk. The strongest international consensus statement on AI risk is, in substantial part, a statement about how little the measurements currently establish.
06 · The Canon

The record

All 200 entries, chronological, each with the body that produced it and a link to the source. Search matches title, author, venue and description. This is the part of the page meant to be used rather than read.

200 entries
Year
Author / body
Work and what it establishes
Stream

Entries marked source not retrieved are listed for completeness but were not confirmed against the primary document at compilation; treat them as leads. Machine-readable copies of this table are linked in section 10.

07 · Governance

What the law
actually requires

Almost everything written about AI regulation describes instruments at the moment they were announced. Several of the most-cited have since been delayed, revoked, renamed or narrowed. This section states current status as of 23 September 2026, with the primary instrument linked in every case.

Binding and in force

EU AI Act. Prohibitions since February 2025; general-purpose model obligations (documentation, adversarial testing, incident reporting above the 1025 FLOP systemic-risk presumption) since August 2025. The only legally enforceable frontier-model regime anywhere. California SB 53, signed September 2025: published frontier frameworks, catastrophic-risk reporting, whistleblower protection. China: generative-AI registration since 2023, content labelling since September 2025.

Announced, then moved

EU high-risk obligations, long expected in August 2026, were postponed by a 423–57 European Parliament vote in June 2026 to December 2027, and to August 2028 for AI embedded in safety components; Council adoption is still pending. Colorado’s AI Act was signed, then materially weakened in May 2026 with its duty of care removed and commencement pushed to January 2027. The AI diffusion rule on chip exports was rescinded two days before it took effect, with no replacement published since.

Voluntary only

The Seoul Frontier AI Safety Commitments, the G7 Hiroshima code, the 2023 White House commitments and every published lab safety framework. Companies set their own thresholds, run their own evaluations and decide their own consequences. No jurisdiction requires independent pre-deployment audit of a frontier model. The Council of Europe convention, the only binding AI treaty, has one ratification, the EU’s, and is not in force.

2023
1 November 2023 · 28 countries including the US, China and the EU
The Bletchley Declaration
The high-water mark of consensus: governments jointly acknowledging potential catastrophic and existential risk from frontier models, and launching the summit process and the first safety institutes. Three days earlier, Executive Order 14110 had made the US the first government to impose reporting duties tied to a compute threshold.
Declaration
2024
12 July 2024 · Official Journal of the European Union
Regulation 2024/1689: the EU AI Act
The world’s first comprehensive binding AI law enters into force on 1 August 2024 and applies in stages. Its systemic-risk tier, triggered by a presumption at 1025 training FLOP, is the only place where frontier-model obligations are backed by enforcement rather than by commitment. The EU AI Office becomes the first dedicated frontier regulator.
EUR-Lex
2025
23 January & 11 February 2025 · Washington and Paris
The consensus breaks
Executive Order 14179 revokes EO 14110 in the new administration’s first week, reorienting federal policy from safety to dominance. Three weeks later, at the Paris AI Action Summit, the United States and the United Kingdom decline to sign the main statement. The Bletchley–Seoul process does not recover its momentum.
Federal Register
2025
14 February & 3 June 2025 · London and Washington
Both safety institutes are renamed
The UK AI Safety Institute becomes the AI Security Institute, narrowing to CBRN, cyber and child-safety threats and dropping bias and societal harms. The US AI Safety Institute becomes the Center for AI Standards and Innovation, reoriented toward adversary AI and national security. The institutional layer built by the summit process survives, with a different remit.
UK announcement
2025
29 September & 11 December 2025 · Sacramento and Washington
States legislate; the federal government sues
California enacts SB 53, the first US law compelling frontier developers to publish safety frameworks and report catastrophic risks, after the more sweeping SB 1047 was vetoed the previous year. Ten weeks later Executive Order 14365 establishes an AI Litigation Task Force to challenge state AI laws and directs funding pressure at states with “onerous” regulation.
EO 14365
2026
15 May & 16 June 2026 · Strasbourg and Brussels
One ratification, one postponement
The EU becomes the first, and so far only, party to ratify the Council of Europe Framework Convention on AI, which needs five ratifications to enter into force. A month later the European Parliament votes to postpone the AI Act’s high-risk obligations by sixteen months as part of a digital simplification package. The binding layer is both deepening and receding at once.
Treaty status

All 47 Governance entries, including the Seoul commitments, the G7 Hiroshima process, the UN scientific panel, China’s framework, the Frontier Model Forum and every lab safety framework, are in the canon with their current status recorded.

08 · The Case Against

Why many researchers
think this is wrong

Five distinct critiques, which disagree with each other as sharply as they disagree with the risk case. Each is stated here as its own proponents state it, without rebuttal. The responses come at the end, clearly labelled, because a reader is entitled to see the argument before they see the counter-argument.

1. The threat model doesn’t describe anything we are building

Threat-model
scepticism
LeCun’s position is that language models lack the configurable predictive world model that goal formation requires, and that no amount of scaling supplies it: a four-year-old takes in roughly fifty times more sensory data than a frontier model sees in training, and still learns to plan in a way the model cannot. Garfinkel, having examined the classic Bostrom–Yudkowsky arguments closely, concluded they rest on “fuzzy, abstract concepts… and toy thought experiments” rather than evidence. Drexler argues advanced AI is arriving as comprehensive services rather than as a single agent, which dissolves the treacherous-turn setup entirely; Thorstad makes the philosophical case that the singularity hypothesis never establishes accelerating rather than diminishing returns to self-improvement; Hanson adds that every historical growth transition happened across populations and markets, not inside one recursive mind.
LeCun 2024 · Garfinkel 2020 · Thorstad 2024

2. The capabilities are not what the benchmarks say

Capability
scepticism
Mitchell’s work shows that “superhuman” benchmark results routinely reflect shortcut learning and statistical regularity rather than generalisation, and that the evaluation literature systematically over-claims. Singh et al. find that evaluation-data contamination inflates scores in ways standard detection undercounts. McCoy et al. show large models carrying the structural “embers” of next-token prediction into predictable failures off-distribution. Marcus, having forecast scaling limits in 2022, points to the Orion project’s downgrade as their arrival. On this view the curves in section 04 measure benchmark performance, which is not the same quantity as capability, and the gap widens exactly where the risk argument needs it to close.
Mitchell 2026 · Singh et al. 2024 · McCoy et al. 2024

3. It displaces the harms that are already happening

Present-harms
critique
The Stochastic Parrots authors identified the costs of scale (environmental expenditure, exclusion of marginalised speakers, bias amplification) before the models that now dominate the debate existed. When the 2023 pause letter foregrounded speculative future risk, they responded that it ignored “the actual harms resulting from the deployment of AI systems today — worker exploitation and massive data theft, synthetic media reproducing systems of oppression, and concentration of power.” Guidi et al. put a figure on part of it: 105 million tonnes CO2e from US data centres in a year, at a carbon intensity 48% above the national average. Jecker and Atuire add the justice dimension: the people most affected by deployed AI are almost entirely absent from a debate dominated by people with equity in it.
Bender et al. 2021 · DAIR 2023 · Guidi et al. 2024

4. Safety framing is a moat

Political
economy
Narayanan and Kapoor argue that treating AI as a categorically novel danger licenses emergency governance, and emergency governance consolidates power with the incumbents who can afford to comply. Their alternative framing (AI as a normal, transformative, general-purpose technology on the model of electricity) implies the ordinary institutional apparatus rather than a new one. Hooker supplies the technical case against the central regulatory instrument: compute thresholds do not track the things that actually generate risk (agentic scaffolding, tool use, multi-model systems, modality), and their reliable practical effect is to burden entrants more than incumbents.
Narayanan & Kapoor 2025 · Hooker 2024

5. The probability estimates do not mean what they are used to mean

Epistemics
The strongest version of the sceptical case is not a low number. It is the Existential Risk Persuasion Tournament’s structural finding: 80 domain experts and 89 superforecasters, the latter with validated calibration records, were put through months of adversarial deliberation designed to produce convergence, and on AI they disagreed more than on any other topic and did not converge at all. Superforecasters ended at 0.38% probability of AI-caused extinction by 2100 where domain experts ended at 3%, roughly eightfold apart, each group unmoved. Garfinkel’s complementary point is that the classic arguments have attracted remarkably few sceptics willing to engage them in detail, so the absence of published refutation is not evidence of soundness. Narayanan and Kapoor generalise: long-horizon forecasts about reflexive social systems have a poor track record that no amount of expertise has improved. A number produced this way is not a policy input. It is a statement about how uncertain we are.
Karger et al. 2023 · Narayanan & Kapoor 2024

Where risk-side researchers say these critiques are weakest

  • On the threat model: the argument was never about language models. Instrumental convergence is a claim about what any sufficiently capable goal-directed system does; criticising the limits of current architectures leaves it untouched.
  • On capabilities: the relevant threshold is not today’s systems, and capability surprises are documented rather than hypothetical. Benchmark saturation has been repeatedly predicted late.
  • On present harms: the two agendas are complementary rather than zero-sum, and lab safety frameworks in fact cover catastrophic-but-not-existential and near-term harms together. Whether attention is actually displaced is an empirical question that has not been settled.
  • On political economy: the incumbency argument cuts both ways, and many researchers took this position years before any institution existed to fund it.
  • On epistemics: Ngo’s response is that the case can be grounded in near-term observable mechanisms rather than long-run probabilities: you do not need a defensible P(doom) to justify precaution against a mechanism you can already demonstrate.
09 · What to Watch

What would change
the picture

The test of a record like this is whether it specifies in advance what would move it. These are indicators that would shift the weight of evidence in one direction or the other, chosen because each is observable, dated and reported by someone other than an advocate.

Would strengthen the risk case

A well-powered wet-lab trial showing significant uplift. The in-silico results are contested precisely because the bench results are negative; a large replicated wet-lab positive would end that argument.

Spontaneous sandbagging. Every scheming result so far is constructed or prompted. A documented case of a model strategically underperforming an evaluation it was not set up to game would be a categorical change, and the GPT-5.6 Sol result shows the boundary is being approached.

An autonomous operation end to end. Not a found vulnerability, but a full chain: reconnaissance, exploitation, persistence, without human steps.

Time horizons past the reliability ceiling. METR cannot currently measure above sixteen hours. A rebuilt suite showing multi-day autonomous competence would make the extrapolation argument much harder to dismiss.

Would weaken it

The horizon curve bending. The 89-day doubling is measured over a short window on a saturating suite. Two years of flat measurement on a rebuilt benchmark would undercut the central quantitative claim.

Replicated negative uplift with current models. If frontier models keep failing to move the needle at the bench as capability rises, the misuse mechanism weakens in exactly the domain governments prioritise.

Interpretability delivering verification. If internal-state methods mature enough to certify the absence of deceptive computation, the scheming argument loses its central epistemic advantage: that you cannot check.

Scaling stalling on economics. At 2.4× cost growth per year the frontier is a handful of organisations. A halt driven by capital rather than capability would change the timelines every forecast here depends on.

The indicator that matters most is a governance one
Nothing in this record is currently checked by anyone outside the company that built the model. The single change that would most improve the evidence base is mandatory independent pre-deployment evaluation with real access. As of September 2026 no jurisdiction requires it, and both governments that built the capacity to carry it out have since narrowed its remit.
10 · Cite

Cite this record

This page is a compilation, not original research. Cite the primary source for any claim; every entry links to it. If you need to cite the compilation itself:

AI Global Code Red. The Artificial Intelligence Risk Record, 1863–2026. ai.codered.global, compiled 23 September 2026. 200 entries.

Corrections are the point of a record like this. If an entry is misdated, misattributed, superseded or simply wrong, the error is in the compilation and worth reporting. Several entries here exist because widely repeated claims turned out on checking to be misdated or misattributed.

Machine-readable editions

Sibling record
This page is one of a pair. The climate research record, 164 works of officially recognised research joined to 56 event dossiers, is at codered.global, built on the same method and the same rule: every claim carries its source, and where the evidence does not exist, the page says so.