Live updates
OpenAI calls for an industry-wide cap on AI development and publishes its pretraining numbers for the first time
First seen on X 4 hours ago@sama ♥ 2,90032 views
OpenAI has published internal pretraining measurements it had never shared before and proposed that AI developers agree on common limits they will not cross. CEO Sam Altman called it "the most important thing OpenAI has published in awhile."
In a post titled "Perplexity and Intelligence" on Nov 22, chief scientist Jakub Pachocki charted base-model log-perplexity for OpenAI models from GPT-3.5 onward. Perplexity measures how well a model predicts the next token; lower is better. On an evaluation set drawn from OpenAI's internal codebase, GPT-3.5 sat at about 0.99 and GPT-4 at about 0.91. Pachocki wrote that once models dropped below 0.8, transformative capabilities emerged in cybersecurity and mathematics, and that after upgrading to a model at about 0.76, solutions to open math problems came "one after another in a single Slack thread." The chart ends with an unnamed "Internal model" at about 0.73, below the latest public model, GPT-6.2 Astra, at about 0.76. A footnote says the curve steepened in 2026, with AI already making meaningful design contributions to the latest models.
The core of the proposal is a ceiling. Pachocki expects further transformative capabilities in the 0.68-0.7 log-perplexity window. He proposed common limits, based on perplexity and other measurements, that developers would not cross until their ability to understand and monitor AI behavior reaches a concrete milestone, such as a successor to chain-of-thought monitors. He warned that an "avalanche" of AI-driven pretraining advances could arrive as soon as 2027, and said the urgency of shared safety standards outweighs the cost of showing competitors where OpenAI stands. On X, Altman said he hopes the field can agree to common limits, adding that labs committed under the White House Accords to set safety standards.
Pachocki's post on X passed 380,000 views and 2,700 likes in just over an hour, with many replies asking about the "internal model." The metric has limits: labs use different evaluation and training data, which makes numbers hard to compare, and chain-of-thought reasoning gains do not show up in perplexity. Pachocki listed both as open problems.
Comments 0
Select a sentence in the story (long-press on phones) to comment on it.
“This is the most important thing OpenAI has published in awhile.”
