OpenAI's newest AI solves 49 of 50 new physics research problems, at about $2.40 each
AI helpers matched top specialists on hard physics calculations for under $3 each, but they still can't tell which questions are worth asking.
A large team of researchers, including people from Harvard, KU Leuven and Google DeepMind, built a test called TheoPhysEval. It has 50 new theoretical physics problems with checkable answers. The Decoder reports that the authors say a top expert would need about a day or less for a typical one.
GPT-6-Astra, running in OpenAI's coding tool Codex, solved 49 of 50, or 98 percent, according to The Decoder. That averaged under $3 per problem. Anthropic's Opus 5.5 scored 92 percent, and Google's Gemini 3.8 Flash scored 68 percent.
The problems are routine research work, the kind often given to PhD students as a first project. They are not big unsolved questions. The authors say AI still lacks "taste": the sense of which problems are worth asking. When asked to invent new problems, the AI got lost in boring details.
Some of the AI's misses turned out to be small mistakes in the experts' own answer keys. After the experts corrected those errors, Astra got every one of those problems right.
To be honest, the real news is that a task that once took an expert a day now costs a few dollars. That could change how young physicists are trained, since some of these problems used to be first research projects. It doesn't mean AI is making physics discoveries yet, because choosing the right question is still a human job.
The facts
- TheoPhysEval contains 50 unpublished theoretical physics research problems with verifiable answers, built by researchers from institutions including ICTS, Harvard, KU Leuven and Google DeepMind.
- GPT-6-Astra, running in OpenAI's Codex, solved 49 of 50 problems (98 percent) at an average cost under $3 per problem and just under $116 for the whole benchmark.
- Anthropic's Opus 5.5 in Claude Code scored 92 percent, Google's Gemini 3.8 Flash scored 68 percent, and the open-weight models DeepSeek-V4.1-Flash and GLM 5.3 scored 54 and 42 percent.
- Without web search, Astra and Opus each scored 92 percent, which the authors take as a sign that leading models have effectively memorized much of the theoretical physics literature.
- One problem stumped every model. For several others, follow-ups with the experts revealed small errors in the reference solutions, and once these were corrected Astra answered them all correctly.
- The authors say some benchmark problems would until recently have worked as first research projects for early-career researchers, but that is no longer appropriate unless the goal is just to work through the material.
Sources
- The Decoder: GPT-6-Astra solves 49 of 50 physics research problems at $2.40 apiece
Published
More in Research
ResearchGoogle DeepMind's AI solves dozens of open math problems, and a top journal published it
Google DeepMind says its AI solved nine long-open math problems and proved dozens of other unproven claims. Outside experts reviewed the work before it was published.