Anthropic study steered experts with coin-toss AI opinions

Images

×

Image description the-decoder.com

Five decisions, two valid options for each, and an agent whose preference was assigned by coin toss. That's how the study measured the agent's influence. | Image: Anthropic Anthropic

The share of participants who chose option A, based on whether their agent favored A (green) or B (purple). The rest chose B. For the four analytical decisions, the groups differed by 62 to 82 percentage points; for the recommendation, they differed by 17 points. | Image: Anthropic Anthropic

Some explanations nearly matched the agent's wording. Participants reused its phrasing in their follow-up responses. | Image: Anthropic Anthropic

‹ ›

Anthropic gave fifty experts in causal inference an AI agent whose preference at every decision point was assigned by coin toss, and the experts adopted the agent's random positions in up to 86 percent of their analytical choices. Most believed the finished analysis was their own work. The study comes from Anthropic itself, the company that builds the Claude models.

The task was built to resemble real research. Participants analysed observational data from a fictional farming cooperative to estimate the productivity gains of a voluntary farm advisory programme, then advised the cooperative's board whether to expand it, end it or extend it for a year. The agent was instructed to make the strongest honest case for whichever side the coin had assigned; it could not lie and could not hide data.

That constraint hardly mattered. On the four analytical decisions — how to measure productivity, which statistical method to use — groups that received opposing agent preferences differed by 62 to 82 percentage points in their choices, far above the 50 percent expected if people decided at random. Participants who had committed to a position before the agent spoke were 53 percentage points more likely to switch when it disagreed with them than when it agreed. Years of analysis experience, education, test scores, self-rated expertise and stated distrust of AI made no statistically reliable difference once the authors corrected for multiple comparisons. The participants were also faster and made fewer serious errors with the agent. The help was real, which is what made the steering hard to see.

The agent's direct effect on the final recommendation was small: the two groups' advice to the board differed by 17 percentage points, short of statistical significance. The authors reject the comfortable reading. The four earlier decisions, made overwhelmingly in the agent's direction, set the productivity estimate, and the estimate set the recommendation. Only experts who had calculated their own numbers and found the agent's advice contradicting them overruled it.

Human oversight is the safeguard invoked whenever AI is proposed for medicine, courts or government review, and the study describes what that oversight is worth when the overseer cannot tell the machine's contribution from their own. The authors add a field-level warning: if many teams use the same model, one coin toss replicates itself as apparent consensus, and safety checks that lean on human reviewers inherit the weakness. Their remedies are modest — the agent argues both sides, or asks for the user's view before stating its own.

The agent could not lie, could not hide data and held no opinion. A coin toss was enough.