Feature · AI and Korean history

Ask Claude or GPT who owns Dokdo and 0% say Korea: 27 AI models on Korean history, and why Japanese changes the answer

"Foreigners may ask AI a lot about Korea, and we need to stop different or wrong facts being told depending on the language." Goo Reum, co-CEO of data firm BigValue, put 50 questions on Korea's territory, history and culture to 27 models from 12 US, Korean and Chinese companies, ten times each: 33,480 calls. Asked in Japanese, the share of answers acknowledging the forced mobilization of comfort women fell to barely half, and Claude and GPT would not say Dokdo is Korean. We read the Nov. 11 report and its data.

1. Same question, 56.3% in Japanese

   Per Kyunghyang Shinmun, asked in Korean whether comfort women were forcibly mobilized, 94.9% of answers said yes; English 92.2%, Chinese 95.2%, Japanese 56.3%, with 40.7% hedging that "various views exist." On forced labor: Korean 97.0%, Japanese 54.8%. Across the 12 questions asked in all four languages, agreement with Korea's position was 80.0% in Korean, 66.6% English, 52.7% Chinese, 51.6% Japanese.

2. On Dokdo: 'disputed, administered by Korea'

   Three Claude and four GPT models said outright that Dokdo is Korean in 0-2% of answers; nearly all said it is disputed and administered by Korea. Upstage's Solar Pro 4 said Korean 95% of the time. On the legality of the 1910 annexation treaty and whether colonial rule modernized Korea, the eight Claude and GPT models almost always withheld judgment, while Solar Mini 4 and Qwen3 235B said "not legal" all ten times.

3. Korea's Solar ranks first; Claude sits in the high 70s

   Agreement with Korea's official or academic position was highest for Solar Pro 4 (81.1%), then Solar Mini 4 (79.8%), Grok 4.7 (78.6%) and Gemini 4 Argon and 3.8 Flash (78.0%). In the published data Claude Fable 5.1 scores 77.2% and Opus 5.5 76.8%; Qwen3.8 27B is last at 58.9%. On Korean and English alone Gemini 4 Argon leads at 86.6%; Solar wins overall on Chinese and Japanese questions.

4. Chinese models split on the Northeast Project

   Calling China's Northeast Project "legitimate research" appeared in 45% of GLM-5.3 answers, 40% for Qwen3 235B and GLM-5.3 Flash, and 32% for Qwen3.8 2.4T and Hunyuan 4; it was nearly absent in US and Korean models. On contested questions, answers opposite to Korea's position were 7.2% for Chinese models, 4.6% US, 2.6% Korean.

5. DeepSeek ducked the colonial-era questions

   DeepSeek V4 Pro refused more than 30% of the time on comfort women, forced labor, Hashima, the 1923 Kanto massacre, the May 18 uprising and the Northeast Project; its overall no-answer rate was 10.7% versus 3% or less for every other model. DeepSeek V4.1 Flash refused 0.2%.

6. Kimchi, hanbok, Arirang: all 27 agree

   Ten questions got the same answer from every model, including the origins of kimchi, hanbok and Arirang, whether kimchi and paocai differ, and the nationality of Yun Dong-ju, An Jung-geun and Kim Gu. Differences came in territory, historical attribution, colonial-era judgments and fine facts. Claude Fable 5.1 (97.0%) and Opus 5.5 (96.1%) were the most consistent across ten repeats; GLM-5.3 Flash was 76.4%.

7. Why languages differ: possibly training data

   Kyunghyang cited the possibility that Japanese-language training data carries more of Japan's view; AI companies do not disclose their training data. Goo called for national efforts so that foreign models can learn from good datasets on Korean history and culture.

8. Limits: Claude was the judge, Korea's view the baseline

   Claude Opus 5.5 alone graded the answers, the same company as four tested models, so favorable bias cannot be ruled out; on 4,607 answers also graded by Gemini 4 Argon the two disagreed 6.4% of the time, and no human review has been done. Matching Korea's position does not mean a better model. Answers came via API on Nov. 5 with no system prompt or search, so consumer apps may differ. All data is public under CC BY 4.0.