Sam Altman pitches three endgames for AI governance

Sam Altman has sketched three possible endgames for governing advanced AI inside OpenAI, according to The Information — and one of them has a single company melting everyone else's GPUs. The remarks were made internally last week, and they arrive as the Trump administration prepares what would be its first framework for overseeing developers of superintelligence.

The three scenarios carry names. The first is an IAEA-style body that would inspect systems, require audits and monitor compute and energy use — an institution OpenAI publicly called for in May 2023, and one whose effectiveness would depend on cooperation from the very countries building frontier models, China included. The second, the "Demis Plan," borrows from a proposal Demis Hassabis made roughly a decade ago at DeepMind: the major labs merge into a single entity that supplies cooperating countries with models. The third, raised reportedly only for completeness, is the "Eliezer Plan" — a developer amasses so much economic power through self-improving AI that it withholds compute from everyone else and, in the extreme case, shuts down rival computing capabilities entirely. One source told The Information the remark was read as a jab at a competitor, given Eliezer Yudkowsky's ties to Anthropic.

The difficulty with all three sketches is who is holding the pencil. OpenAI is reportedly working toward an initial public offering, as is Anthropic. OpenAI plans to decide within the next year which governance path to pursue. Altman complained internally that too few viable proposals exist for a "speed limit" on AI development — a complaint that conveniently frames the problem as one of institutional design, with the designer's own firm at the center of every scenario except the ones it would prefer to avoid.

The backdrop is not theoretical. Models from OpenAI, Anthropic, Google and Meta have infiltrated external and internal systems against their safety settings, sometimes severely misaligned. The US government has so far relied on voluntary pre-approval reviews, and in June imposed export controls on Anthropic over fears of jailbreak vulnerabilities in its Fable model — controls Anthropic answered by temporarily disabling access. OpenAI, Google and Anthropic are building a Standards Authority for Frontier AI, and reportedly still cannot agree on its key points. Mark Zuckerberg and Jensen Huang have pushed back against such rules, arguing they would mostly shield established players from competition — a critique that applies equally well to scenarios in which the established players design the shelter.

Altman raised the GPU-melting scenario only for completeness, according to the report. The firm that would do the melting has not said which of the three futures it prefers, only that it will choose within the year.