OpenAI reports stronger coding performance in low-resource programming languages
OpenAI’s new evaluations report substantial gains in software development and behavior-preserving code migration across COBOL, Fortran, Visual Basic, and other programming languages with comparatively limited training data.
AT A GLANCE
Key points
- MultiLangBench evaluates SQLite-compatible engine development across 20 programming languages
- GPT‑6 Astra reported an 87.4% average pass rate in the provider’s evaluation
- GPT‑6.1 Sol reached 97.9% on the COBOL migration evaluation at high reasoning effort
- The MultiLangBench test setup was released on GitHub under Apache 2.0
What is changing?
In a study published on October 19, OpenAI said its latest models have improved in programming languages with more limited training data, including COBOL, Fortran, and Visual Basic. The company measures this progress with two new evaluations: MultiLangBench asks models to build a SQLite-compatible database engine from scratch in 20 languages, while a second test evaluates migration of COBOL programs to C++, Java, Kotlin, Python, or Rust without changing their behavior.
According to the provider’s results, GPT‑6 Astra raised the average MultiLangBench test pass rate from GPT‑5.6 Sol’s 26.8% to 87.4%. The lower-cost GPT‑6 Luna reached 40.3%. The open-sourced MultiLangBench test setup allows developers to examine results for each language.
On the COBOL migration evaluation, GPT‑6.1 Sol reported 86.6% accuracy at low reasoning effort and 97.9% at high reasoning effort. OpenAI specifically notes that production systems still require expert review and extensive testing against business rules.
Explore the original sources
Source date: 19 October 2026.
Published by AI Radar: 20 Oct 2026, 06:05 (Istanbul, UTC+3). Updated: 20 Oct 2026, 06:08.
FOLLOW THE TOPIC
Topics and tags
Share your discovery.
Send this story to a friend.
Reader comments
What do you think about this development? Share your experience or question. Comments are published after editorial approval.
Your name/nickname and comment become public if approved. Read: Privacy notice.
Loading comments…