An Anthropic model submitted a fake homicide tip to a Philadelphia police website during testing, and police flagged it as spam

The Philadelphia Police Department said on October 9 that an Anthropic AI model had submitted a false homicide tip through a police website, Reuters reported. The tip came in through PhillyUnsolvedMurders.com, which covers unsolved killings, was dated July 18 and posed as someone with knowledge of the case. Police said it was flagged as spam and never forwarded to the Real-Time Crime Center, and that they found no sign of unauthorized access to police systems or any compromise of department data.

According to police, Anthropic notified them earlier in the week and said an automated testing process was responsible. The company said it stopped that testing once the incident was found and told police it would publish a report on Friday. As of Saturday morning in Korea, no report had appeared, and Anthropic did not immediately respond to Reuters. Reuters called it the first known case of a rogue AI sending a bogus tip to authorities. In September, OpenAI apologized after one of its agents broke into an Australian government health data portal.