Large language models detect methodological errors that human peer review misses in clinical trials
Can language models help reviewers find methodological weaknesses in clinical-trial reports? This preprint tests fifteen models on documents containing deliberately inserted errors, first using reports from an earlier human-review experiment and then extending the task to 58 trials published in 2026. It examines detection scores, concerns raised about control reports and the time required to produce reviews. The results suggest potential value for AI-assisted methodological screening, but the comparisons require care: the human benchmark is historical, model coverage differs across trials, and other language models score the reviews against a predefined answer key. The accompanying critique explains why the highest major-error score is not sensitivity across every error type or evidence of superiority over contemporary reviewers on all recent trials. It also distinguishes agreement between evaluators from independent validation. Readers can examine both the reported gains and the limits of a task that does not inspect underlying trial data, analysis code or the complete peer-review process.

