Anthropic has Claude grade its own conversations right after replying: 94% of high-priority cases caught, some push back
First seen on X 7 hours ago@AnthropicAI ♥ 2,11011 views
"No one needs this. Let the USER decide what is moral and what's not." That reply to Anthropic's Oct. 29 X post on Claude's "self-assessment" drew 439 likes. Anthropic's post passed 310,000 views and 2,100 likes.
On its Safeguards blog, dated Oct. 16, Anthropic says that on Claude Fable 5 self-assessment flags 78% of conversations a stronger prompted classifier marks as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate. It has used the technique since June and says it surfaced large-scale misuse other safeguards missed.
After each response a short instruction is appended and Claude returns an assessment: whether the request is potentially harmful, the type of harm, a 1-10 priority for human review, whether it refused, and a one-sentence reason. Those two turns are hidden from the user and not carried into the conversation. Because the conversation is already in the KV cache, it is cheap, and Anthropic says it improves as models improve.
Kalshi and Polymarket relayed it as Claude monitoring its own conversations. Replies split: "Very slippery slope. This is why we'll always need Open Models" (287 likes), "Anthropic will be the creators of the American social credit system" (134), and one user urging that AI be allowed for cyber defense (79).
“We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment.”
