Untrained human peer reviewers found 27% of the major errors planted in three clinical trial reports, and every one of 15 language models given the same reports found more, between 53% and 96%.
The reports come from a BMJ experiment run in 2004. A new preprint led by Ruslan Rust then worried the models might simply have seen those old papers, so it repeated the test on 58 randomized trials published between July and October 2026. The models scored between 58% and 99%, the open-weight ones between 66% and 89%, and a review took between 1.4 and 16.3 minutes.
Two things keep me from calling this a replacement. The human number comes from an older study of untrained reviewers, so it is not a head to head. And on control papers with no planted errors, the models still flagged 14% to 29% of the error types, mostly conclusions they thought the results did not support. The authors also run a platform that is adding AI-assisted review, which is worth knowing when you read the numbers.
My position is that the debate about banning AI from peer review is aimed at the wrong thing. The version with the measurable gap is the fully human review, so the sensible default is an AI first pass that costs minutes, with an editor who decides what is real.
Preprint: https://www.medrxiv.org/content/10.64898/2026.10.25.26366455v2 Chart: Brian, Pereira Hernández, Yin, Koury and Rust, medRxiv 2026