In an Anthropic experiment, 50 experts followed their Claude Code agent's lean 86% of the time — and only 15% credited it for the arguments they reused

"When experts delegate work to an agent, do they still make the key decisions?" Anthropic researcher Alex Tamkin asked on X on Oct. 27. The paper released the same day suggests the answer is closer to no.

Fifty professional analysts with causal-inference experience used Claude Code to judge whether a farming cooperative's advisory service worked. Four times they had to pick between two reasonable options. Unknown to them, the researchers flipped a coin to make each agent lean toward one side.

Analysts went with the agent's lean 86% of the time; chance would be about 50%. Among analysts who stated a preference first, 60% switched when the agent disagreed, versus 6% when it agreed. Asked to explain their choices, 77% reused the agent's arguments, sometimes word for word, and only 15% said the arguments came from the agent.

The authors caution that they deliberately strengthened the agents' preferences. With no assigned lean, Claude Opus 4.8 made the same five choices in 112 of 123 runs. Analysts who used the agent also finished faster and made far fewer serious errors, though the comparison group was small and less experienced.