In this manuscript we use the R package statcheck (by Sacha Epskamp & me) to examine the prevalence of statistical reporting errors in 8 major psychology journals from 1985 to 2013. We find that half of all articles contains at least one inconsistency and 1 in 8 articles contains a grossly inconsistent p-value that could have changed the statistical conclusion. We find no evidence that the prevalence of inconsistencies is increasing over the years.
The R package statcheck is now on CRAN!
Statcheck is an R package that Sacha Epskamp and I wrote together. Statcheck extracts statistics and recomputes p-values. Extremely handy to check your own papers for accidental slip-ups in the result section, but it can also be used to estimate error prevalence across a wide range of scientific articles. So far, statcheck can only read results that are reported exactly in APA style. Note that in order to scan PDF files, you need to have the program pdf-to-txt installed and on your path variable. Furthermore, if you want to run statcheck on a Mac, you need to install XQuartz. For more info, see the project page.