AI coding agents can modernize research software but can't judge if the science is right

AI coding agents can modernize research software but can't judge if the science is right Jonathan Kemper View the LinkedIn Profile of Jonathan Kemper Aug 1, 2026 Nano Banana Pro prompted by THE DECODER A field report from OpenAI and academic partners shows that coding agents can update and speed up aging research software. Much of the work, however, shifts from writing code to verifying the results. Many widely used research tools began as supporting code for a single paper. Small academic teams often wrote them without the time or resources for proper testing, maintenance, or optimization. The result is fragile software that remains critical to entire fields but requires constant repairs. A field report from OpenAI and academic partners suggests AI coding agents could help close that gap. The report documents eight case studies, mostly in biology, in which research groups used coding agents such as Codex and Claude Code. The projects range from basic maintenance and targeted optimization to full rewrites in modern programming languages. The eight projects span from simple build modernization to a full GPU-native rewrite. | Image: OpenAI Coding agents delivered speedups of more than 60 times One of the simpler projects involved modernizing cyvcf2, a Python library for reading genetic data. GPT-5.5 replaced its outdated build and installation process with a modern one. The MHCflurry migration was far more involved. MHCflurry is an immunology model that predicts which targets immune cells will recognize. Claude Code and Codex alternated between the roles of developer and reviewer while porting about 10,000 lines of code from TensorFlow to PyTorch. The rustar-aligner project was more ambitious. It rebuilt STAR from scratch in Rust. STAR maps sequencing reads from cells to the corresponding locations in a genome. The original contains more than 20,000 lines of C and C++ and is no longer actively maintained, even though it remains part of many research pipelines. To check whether the rewrite behaved like the original, the team tested both tools on 10,000 short sequencing reads from yeast cells. For single-end reads, rustar-aligner produced the same result as STAR in 99.815 percent of cases. For paired-end reads, the agreement rate was 99.883 percent. The comparison covered more than the mapped location in the genome. It also included several other key fields that both programs produce for each read. Neither tool mapped any reads that the other failed to map. RustQC delivered the largest speedup by combining 15 separate quality-control tools in a single program. On a large dataset, runtime fell from 15 hours and 34 minutes to 14 minutes and 54 seconds, a speedup of more than 60 times. Another project, HelixForge, replaced a tool for generating synthetic genomic data with a version that runs on GPUs. In a test using data from one donor and a ten-million-base-pair section of the genome, HelixForge completed the full pipeline 59.6 times faster than BamSurgeon. The main compute step alone ran 98.6 times faster. The GPU-native rewrite beats the established CPU tool on every measured axis, not just speed. | Image: OpenAI Fast code can still produce bad science Across the case studies, agents completed well-defined tasks quickly but couldn't reliably judge whether their work was scientifically correct. Even when their code contained errors, the systems often presented it with full confidence. "With coding agents, it's quite easy to go fast; for now, to go far in science, there's still a need for expert guidance, understanding, taste, and care," writes cyvcf2 developer Brent Pedersen. Philip Ewels, who led RustQC, describes agents as "eloquent, convincing, and confidently wrong in ways that are easy to miss." He never allowed the models to judge the accur