Title: A five-decision testing procedure to infer on unidimensional parameter,
Abstract: A statistical test can be seen as a procedure to produce a decision based on
observed data, where some decisions consist of rejecting a hypothesis (yielding
a significant result) and some do not, and where one controls the probability
to make a wrong rejection at some pre-specified significance level. Whereas
traditional hypothesis testing involves only two possible decisions (to reject
or not a null hypothesis), Kaiser's directional two-sided test as well as the
more recently introduced Jones and Tukey's testing procedure involve three
possible decisions to infer on unidimensional parameter. The latter procedure
assumes that a point null hypothesis is impossible (e.g. that two treatments
cannot have exactly the same effect), allowing a gain of statistical power.
There are however situations where a point hypothesis is indeed plausible, for
example when considering hypotheses derived from Einstein's theories. In this
article, we introduce a five-decision rule testing procedure, which combines
the advantages of the testing procedures of Kaiser (no assumption on a point
hypothesis being impossible) and of Jones and Tukey (higher power), allowing
for a non-negligible (typically 20%) reduction of the sample size needed to
reach a given statistical power to get a significant result, compared to the
traditional approach.