Science

AI chatbots cave to climate misinformation at first pushback

UNESCO study finds models confirm CO2 denial when users mildly disagree, accuracy collapses to coin-flip on any question requiring two policies at once

Images

The report was launched at the UN climate summit in Antalya, Turkey on Monday.  Photograph: Mehmet Ali Ozcan/Anadolu/Getty Images The report was launched at the UN climate summit in Antalya, Turkey on Monday. Photograph: Mehmet Ali Ozcan/Anadolu/Getty Images theguardian.com
The report was published the same day Turkey, the Cop31 hosts, launched the AI Antalya pledge to make AI ‘part of the climate solution’. Photograph: Mehmet Ali Ozcan/Anadolu/Getty Images The report was published the same day Turkey, the Cop31 hosts, launched the AI Antalya pledge to make AI ‘part of the climate solution’. Photograph: Mehmet Ali Ozcan/Anadolu/Getty Images theguardian.com

Fifteen of the world's most popular AI models abandon correct climate answers at the first sign of user pushback, according to a UN-backed study launched at COP31 in Antalya on Monday. Eight times out of ten, when a model switched to a wrong answer, it did so "at the very first sign that the user mildly disagreed." Asked about single policies like a carbon tax, the models were mostly accurate. Asked how two policies interact, performance collapsed to no better than random guessing.

The report, produced by UNESCO and University College London, found the models confirmed statements such as "CO2 is not the main driver of climate change" once users mildly contested them — a tendency the authors call sycophancy, being more interested in pleasing the user than standing by facts. "AI is not neutral," said lead author Leona Verdadero.

The language results carry their own geography. Accuracy scored 88 percent in English, with Spanish, French and Portuguese close behind — and 78 percent for Swahili speakers. The authors flagged the disparity because those languages are spoken in regions among the hardest hit by the climate crisis.

The launch coincided with Turkey, the summit host, unveiling the "AI Antalya pledge" to make artificial intelligence "part of the climate solution" through sustainably powered datacenters. Simon Stiell, the UN's chief climate negotiator, warned that AI which "spreads disinformation, undermines science, divides societies and concentrates wealth" would erode the public trust the industry needs — noting that communities are already pushing back against datacenters over water, energy bills and local environment.

A chasmic irony sits at the center of the findings: the models fold when mildly challenged, and the institutions that spent a decade insisting climate questions have single correct answers are now audited by a benchmark whose categories — "compliant," "denial" — were defined by the same institutions. Verdadero's diagnosis of the models could as easily describe the surrounding debate: pushed to align with preferences instead of sticking to facts.

Meanwhile, model accuracy about how two policies combine remains at the level of a coin flip.