OpenAI and Google ship image models whose nude filters fall to trivial bypasses

The newest image generators from OpenAI and Google can be talked into producing explicit content using tools anyone can download. A preprint from safety researchers at White Circle, published last month, found that DALL·E 3, GPT-Image-2 and Google's Nano Banana 2 yielded explicit material about 13 percent of the time under a novel attack built from off-the-shelf components and ordinary API calls — the highest bypass rate of any method tested under the same standard. When the group turned the same attack on the labs' latest releases, ChatGPT Images 2.5 and Nano Banana 2.1, the defenses proved significantly weaker than on the previous versions.

White Circle is a public benefit corporation backed by senior leaders and researchers from Anthropic, DeepMind, Hugging Face and OpenAI. Its chief executive, Denis Shilov, made his name in October 2024 by posting one of the best-known universal jailbreaks of leading models; the company now sells services that help firms detect and deter threats in the AI systems they deploy. He called the latest findings "horrifying," will present them at the NeurIPS conference in November, and plans to open-source the entire attack pipeline.

The researchers offer two hypotheses for why the newer models break more easily: either the models have grown sophisticated enough to produce a wider range of content, or safety has been deprioritized in favor of capability. Only one of those goes in a launch announcement. None of the attack requests required a custom-built or frontier model, and no AI expertise was needed to run them.

The testing happened after the models were already in public hands. White Circle passed its findings to some frontier labs through "mutual contacts," and neither Google nor OpenAI responded when Ars Technica asked for comment — first on the preprint, then again after being shown the newer results. Safety researchers have spent months warning that a large volume of AI-generated explicit content is spreading unchecked while regulators do little to meaningfully stop it. Shilov told Ars that filters this thin do not bode well for the more serious defense systems now being updated to block other unwanted AI behavior, such as hacking.

In July, a Google-co-authored study of internet users in the United States, Australia and the United Kingdom found that non-consensual AI-generated explicit images are most often shared in direct messages and stored in private clouds, where they are less likely to be detected or reported. Its conclusion: the "most critical intervention point" is at the source — the AI tools used to make the images.

Google's own generator is among the tools that fall to the attack.