A retrospective analysis of 400 publications reveals patterns of irreproducibility across an entire life sciences research field

Abstract

Research on Drosophila immunity has not only transformed our understanding of innate immunity but has also influenced studies on insect pests and disease vectors. Yet, as in many rapidly developing fields, some published findings have proven irreproducible. While certain results have been explicitly contradicted, many others remain untested, often due to limited follow-up or a lack of incentive to publish negative findings. To address this gap, we conducted a systematic conceptual reproducibility project evaluating claims from articles on Drosophila immunity published before 2011. From 400 papers, we extracted key claims and assessed their verifiability by cross-referencing the literature and, in selected cases, experimentally testing “unchallenged” claims without prior replication https://ReproSci.epfl.ch/. Our approach provides, uniquely, a field-wide assessment—14 years later—of the replicability of nearly all publications within an experimental life science community. We found that 61% of claims were verified, while only 7% were directly challenged (not reproducible), a replicability rate higher than previous assessments. Notably, 24% of claims had never been independently tested and remain unchallenged. We performed experimental validations of a selection of 45 unchallenged claim, that revealed that a significant fraction of them is in fact non-reproducible. We found that high-impact journals and top-ranked institutions are more likely to publish challenged claims. In line with the reproducibility crisis narrative, the rates of both challenged and unchallenged claims increased over time, especially as the field gained popularity. We characterized the uneven distribution of irreproducibility among first and last authors. Group leaders, who had prior experience as first authors in another Drosophila immunity team, had lower irreproducibility rates, underscoring the importance of early-career training. Finally, authors with a more exploratory, short-term engagement with the field exhibited slightly higher rates of challenged claims and a markedly higher proportion of unchallenged ones. We also did interviews with principal investigators to further shed light on how researchers navigate issues of reproducibility in practice. This systematic, field-wide retrospective study offers meaningful insights into the ongoing discussion on reproducibility in experimental life sciences.