New web app spots statistical errors in academic papers

Methodologists from Tilburg University and the University of Amsterdam have developed a simple web app that can automatically detect statistical errors in psychology research papers.

Researchers can use it to check their own work before submitting it for publication in an academic journal, and it is also useful for academic journals to assess submissions.

The web app is called 'statcheck' and has been operational since September 26th, 2016. The 'statcheck' program on which it is based has been available for a while within the open-source statistics software package R and was written by Sacha Epskamp of the University of Amsterdam and Michèle Nuijten from Tilburg University. With the new app, people who cannot program in R can still use statcheck. The program checks whether P values reported in a paper are consistent with the related data.

Earlier research conducted by Nuijten, in which she tested 30,000 psychology research papers by means of statcheck showed that approximately half of the papers contained at least one error. It mostly concerns minor inaccuracies, but 1 in 8 papers contains an error that affects the statistical conclusion.

Nuijten hopes that the use of statcheck through this free web app will become common practice among psychology researchers.