Apple Intelligence pushes unprompted notification summaries with biased hallucinations

Apple’s new “Apple Intelligence” feature set is being marketed as a privacy-forward, on-device AI assistant. But an independent analysis suggests the more consequential question is not where the model runs, but what happens when it speaks first.

According to The Decoder, the non-profit AI Forensics examined more than 10,000 Apple Intelligence notification summaries and found systematic distortions in how the system compresses identity-related information. In a controlled test using 200 fictitious news stories with explicit ethnic identifiers, the summaries mentioned ethnicity for white protagonists only 53% of the time, versus 64% for Black protagonists, 86% for Hispanics and 89% for Asians. The pattern effectively treats “white” as the invisible default and other ethnicities as notable attributes.

The same dynamic appeared in gendered language. Using 200 real BBC headlines, AI Forensics found women’s first names were retained more often than men’s. In ambiguous-pronoun scenarios, the system frequently collapsed ambiguity into a specific assignment: in 77% of cases it picked a person even when the original text didn’t. Two-thirds of those assignments followed stereotypes (e.g., “she” as nurse, “he” as surgeon). Across other social dimensions, the report says Apple Intelligence hallucinated attributes not present in the source text about 15% of the time, and nearly three-quarters of those errors matched common prejudices—examples included linking a Syrian student to terrorism or treating pregnancy as a reason someone was unfit for work.

What makes this different from the usual chatbot embarrassment is distribution and initiation. Apple Intelligence summarizes notifications, messages and emails automatically across iPhone, iPad and Mac—meaning outputs can be produced without a user prompt, and at the scale of “hundreds of millions of devices,” as the report notes. That turns a model failure mode into a push-notification feature.

Apple’s technical framing—an on-device model of roughly three billion parameters, with a “Private Cloud Compute” fallback—does not resolve the core governance problem: when the system generates a false or stigmatizing summary, what is the accountability path? Is the incident logged locally, reported to Apple, reproducible for audit, and contestable by the user? The product’s value proposition is frictionless automation; its downside is frictionless defamation.

Incentives point the wrong way. Shipping an “intelligent” assistant is a competitive necessity in Big Tech’s arms race; the cost of errors is externalised to users, employers, and social institutions that must clean up reputational or professional damage. Apple also gains a subtle legal advantage by framing the feature as “summaries” rather than original content—while the harms arise precisely because readers treat them as authoritative.

There is also a regulatory asymmetry. AI Forensics argues Apple Intelligence could qualify as a “systemic risk” model under the EU AI Act given its reach, yet Apple has not signed the voluntary Code of Practice, The Decoder reports. Europe’s regulators may discover that the most scalable form of bias is the one delivered as a convenience layer.

The irony is that private actors usually impose stricter standards when they bear the costs. Here, Apple sells the upside of automation while dispersing the downside across millions of daily micro-incidents—each too small to litigate, together large enough to reshape norms about what machines are allowed to assert about people.