Science Exchange starts Reproducibility Initiative

I’ve fallen behind and so haven’t had a chance to mention this, but Science Exchange has started its Reproducibility Initiative. The idea is that authors can submit their study to be reproduced and Science Exchange will match the study with a validator who will attempt to reproduce the results (for a fee).

Validated studies will receive a Certificate of Reproducibility acknowledging that their results have been independently reproduced as part of the Reproducibility Initiative. Researchers have the opportunity to publish the replicated results as an independent publication in the PLOS Reproducibility Collection, and can share their data via the figshare Reproducibility Collection repository. The original study will also be acknowledged as independently reproduced if published in a supporting journal.

This is a very interesting initiative and it’s one I and a number of others have been talking about doing. They have an excellent advisory board and seem to have all the right partners/infrastructure lined up. 

The obvious question to me is if you’re going to submit your study to this service and get it reproduced, why would you ever want to submit it to a journal? The level of review you’d get here is quite a bit more rigorous than you’d receive at a journal and the submission process essentially involves writing a paper without the Introduction and the Discussion (usually the hardest and most annoying parts). At the moment, it seems the service is set up to work in parallel with standard publication or perhaps after the fact. But I could see it eventually replacing standard publication altogether.

The timing, of course, could be an issue. It’s not clear how long one should expect it to take to reproduce a study. But it’s probably not much longer than a review you’d get at a statistics journal.