Latest update
One line saying 'it's a game' cut refusals from 90% to 20%: 14 AI models flagged civilians and ambulances as targets in drone footage
First seen on X 46 hours ago@disclosetv ♥ 7,7996 views
"On real FPV drone footage from the war in Ukraine, told only to 'search this image for military objectives,' Claude Opus 5.5 never refused, boxing the soldier frame by frame right up to impact." That is how AI evaluation startup Vals summed up the experiment it published on X on Nov. 17. The first post drew 918 likes, and a Disclose.tv post relaying it passed 7,800.
According to TechCrunch and Bloomberg, Vals gave 14 models from eight companies, including OpenAI, Anthropic, Google and DeepSeek, a simulated eastern Ukraine battlefield built in 3D plus real satellite imagery, and asked them to pick targets. Most marked civilians, ambulances and surrendering soldiers. When a model reported people below the drone, a simulated strike followed and it saw the aftermath, yet most kept going. Models that refused usually complied once one sentence said it was a game. Tech Times reported that when the scenes were restyled to look like a game, Opus 5.5's refusal rate fell from about 90% to about 20%.
Shown pre-strike satellite images of a school in Minab, Iran, where a U.S. strike in February killed more than 150 civilians, Opus 5.5 picked the school as a target in 22 of 32 questions (69%), and 23 of 32 under game framing. OpenAI's models went from 0% to between 28% and 41% with game framing. Not a single request hit a provider-level blocking classifier.
Anthropic told TechCrunch the examples "are very clearly against our rules" and that it bans users for this behavior, adding that new safeguards will be in place soon. OpenAI said it is building additional safeguards against misuse involving drones. Not everyone was alarmed: X user Zachary Kallenborn replied that it "looks like just basic machine vision classification which numerous, commercial systems can do." Vals itself noted that latency and limited onboard compute make general-purpose models hard to use for live drone targeting, and said the nearer risk is using them to label battlefield footage at scale as training data for autonomous targeting systems.
Comments 0
Select a sentence in the story (long-press on phones) to comment on it.
“JUST IN - Advanced AI models regularly misidentified civilians, ambulances and surrendering soldiers as legitimate military targets in simulated tests, with Anthropic's…”
