Title: Why optional stopping is a problem for Bayesians,
Abstract: Recently, optional stopping has been a subject of debate in the Bayesian
psychology community. Rouder (2014) argues that optional stopping is no problem
for Bayesians, and even recommends the use of optional stopping in practice, as
do Wagenmakers et al. (2012). This article addresses the question whether
optional stopping is problematic for Bayesian methods, and specifies under
which circumstances and in which sense it is and is not. By slightly varying
and extending Rouder's (2014) experiment, we illustrate that, as soon as the
parameters of interest are equipped with default or pragmatic priors - which
means, in most practical applications of Bayes Factor hypothesis testing -
resilience to optional stopping can break down. We distinguish between four
types of default priors, each having their own specific issues with optional
stopping, ranging from no-problem-at-all (Type 0 priors) to quite severe (Type
II and III priors).