arXiv:1608.02801v4 [math.ST] 20 Dec 2017

ON THE ASYMPTOTIC NORMALITY AND THE
CONSTRUCTION OF CONFIDENCE INTERVALS FOR
ESTIMATORS AFTER SAMPLING WITH PROBABILISTIC
AND DETERMINISTIC STOPPING RULES
BEN BERCKMOES AND GEERT MOLENBERGHS

Abstract. A key feature of a sequential study is that the actual sample size is a random variable that typically depends on the outcomes
collected. While hypothesis testing theory for sequential designs is well
established, parameter and precision estimation is less well understood.
Even though earlier work has established a number of ad hoc estimators
to overcome alleged bias in the ordinary sample average, recent work
has shown the sample average to be consistent. Building upon these results, by providing a rate of convergence for the total variation distance,
it is established that the asympotic distribution of the sample average
is normal, in almost all cases, except in a very specific one where the
stopping rule is deterministic and the true population mean coincides
with the cut-off between stopping and continuing. For this pathological
case, the Kolmogorov distance with the normal is found to equal 0.125.
While noticeable in the asymptotic distribution, simulations show that
there fortunately are no consequences for the coverage of normally-based
confidence intervals.

1. Introduction
In surprisingly many settings, sample sizes are random. These include
sequential trials, clusters of random size, incomplete data, etc. [MKA14] and
[MMA16] studied implications of this on estimators in a unified framework;
[MMA15] focused on the specific but important case of a sequential trial,
which is also the setting of interest in this paper.
While formal sequential methodology dates back to World War II ([W45]),
most emphasis has been placed on design and hypothesis testing. Regarding
parameter estimation after sequential trials, it has been reported that commonly used estimators, such as the sample average, exhibit bias at least in
some settings. In response, a variety of alternative estimators have been proposed ([S78, HP88, EF90]). Building upon [LH99] and [LHW06], [MKA14],
[MMA16], and [MMA15] reviewed several of these and actually showed that
the sample average is a consistent estimator in spite of earlier concern, even
though there is a small amount of finite-sample bias. Their approach is
based on considering a class of stochastic stopping rules that lead to the
Key words and phrases. asymptotic normality, confidence interval, Kolmogorov distance, random sample size, rate of convergence, sequential clinical trial, stopping rule,
total variation distance.
Ben Berckmoes is post doctoral fellow at the Fund for Scientific Research of Flanders (FWO); financial support from the IAP research network #P7/06 of the Belgian
Government (Belgian Science Policy) is gratefully acknowledged.
1

2

BEN BERCKMOES AND GEERT MOLENBERGHS

more commonly encountered deterministic stopping rules as limiting cases.
They used incomplete-data ignorable likelihood theory to this end. In addition, they showed that there exists an alternative, conditional likelihood
estimator that conditions on the sample size realized; this one is unbiased
also in small samples but is slightly less efficient than the sample average,
and is implicitly defined through an estimating equation.
While these earlier results are important, the authors did not address the
limiting distribution of the mean estimated from a sequential trial and its
implications for confidence interval estimation. This is the focus of the current paper. To this end, we consider the manageable but generic case where
in a first step n i.i.d. normally distributed N (µ, 1) observations are collected,
after which a stopping rule is applied and, depending on the outcome, a second i.i.d. set of n observations is or is not collected. The probability
of


β
stopping after the first round is assumed to be of the form Φ α + n Kn ,
with Φ(·) the probit function, Kn the sample sum of the first n observations, and α and β a priori fixed parameters. The setting is formalized in
the next section. While there are many cases where other than normal data
are collected, it is a sufficiently common and at the same time tractable case;
extension to the exponential family is of interest but outside of the scope of
this paper. Also for ease of exposition, we consider a study with two possible
sample sizes, n and 2n. Also this can be generalized in a straightforward
fashion. Finally, depending on the situation, Kn /n may or may not be the
core of the test statistic considered, even though the ratio of the sample sum
over a measure of information is very commonly encountered. Calculations
for alternative functions of Kn will follow logic similar to the one displayed
here.
Employing the total variation distance, we establish that for stochastic
stopping rules asymptotic normality applies. Likewise, we show that this is
true too for deterministic stopping rules, provided that µ 6= 0. For these
cases rates of convergence are established. When µ = 0 there is no weak
convergence; we establish the Kolmogorov distance between the true distribution and the normal.
In Section 2, the formal framework is introduced. In Section 3, the main
result is formulated. The behavior in practice is gauged by way of a simulation study, described in Section 4, with some details relegated to the
Appendix. Implications and ramifications are discussed in Section 5.

2. Formal framework
Let X1 , X2 , . . . , Xn , . . . be independent and identically distributed random variables with law N (µ, 1). Also, let N1 , N2 , . . . , Nn , . . . be random
sample sizes such that each Nn takes the values n or 2n, is independent of
Xn+1 , Xn+2 , . . ., and satisfies the conditional law

β
P [Nn = n | X1 , . . . , Xn ] = Φ α + Kn ,
n


(1)

3

where Φ is the standard normal cumulative distribution function,
n
X
Xi ,
Kn =
i=1

R+ .

α ∈ R, and β ∈
Notice that the restriction that β be positive is merely
for convenience, and that the results presented in this paper can be easily
extended for negative β. We also consider the limiting case of (1) where
β → ∞, which corresponds to
P [Nn = n | X1 , . . . , Xn ] = 1{Kn >0} ,

(2)

where 1{Kn >0} stands for the characteristic function of the set {Kn > 0}.
Finally, we define the estimator
1
µ
bNn =
KNn ,
(3)
Nn
which is the classical average of a sample with random size Nn .
In [MKA14], it is shown that µ
bNn , defined by (3), is, for both the stopping rules (1) and (2), a legitimate estimator for µ in the sense that it is
asymptotically unbiased. More precisely, it is established there that, for the
probabilistic stopping rule (1),
!
α + βµ
β
1
p
φ p
,
(4)
E[b
µ Nn ] = µ +
2n 1 + β 2 /n
1 + β 2 /n
and, for the deterministic stopping rule (2),
√
1
E[b
µNn ] = µ + √ φ( nµ),
2 n

(5)

where φ is the standard normal density. Clearly, (4) and (5) both converge to
µ as n tends to ∞. These authors also consider small sample bias corrected
estimators, but this is outside of the scope of this paper.
In this note, we consider a different aspect of the legitimacy of the estimator µ
bNn . More precisely, we examine the asymptotic normality of the
sequence

p
Nn (b
µNn − µ) .
(6)
n

3. Statement of the main result

Recall that the Kolmogorov distance between random variables ξ and η
is given by
K(ξ, η) = sup |P[ξ ≤ x] − P[η ≤ x]| ,
x∈R

and the total variation distance by
dT V (ξ, η) = sup |P[ξ ∈ A] − P[η ∈ A]| ,
A

the supremum running over all Borel sets A ⊂ R. Clearly, the inequality
K ≤ dT V

holds, and it is known to be strict in general. Also, it is well known that
a sequence of random variables (ξn )n converges weakly to a continuously
distributed random variable ξ if and only if K(ξ, ξn ) → 0. Finally, dT V

4

BEN BERCKMOES AND GEERT MOLENBERGHS

metrizes a type of convergence which is in general strictly stronger than
weak convergence. For more information on these distances, and on the
theory of probability distances in general, we refer the reader to [R91] and
[Z83].
In the following theorem, our main result, we show

√ that if the probabilistic
Nn (b
µNn − µ) n converges
stopping rule (1) is followed, then the sequence
in total variation distance to Φ, and we establish a rate of convergence in
this case. Furthermore, we prove that if the√deterministic stopping rule (2)
is followed and µ 6= 0, then the sequence
Nn (b
µNn − µ) n also converges
in total variation distance to Φ, and we again provide a rate of convergence
in this case. Finally, we establish that if the deterministic
stopping rule (2)
√
N
(b
µ
−
µ)) = 1/8. In
is followed and
µ
=
0,
then,
for
each
n,
K(Φ,
n
N
n

√
Nn (b
µNn − µ) n fails to converge weakly to Φ in this case. We
particular,
nevertheless show that in all cases it is plausible to use estimation (3) for
the construction of reliable confidence intervals for µ.
A proof is given in Appendix A.
Theorem 1. Suppose that the probabilistic stopping rule (1) is followed.
Then, for each n,
p
µNn − µ)) ≤ C(α, β, µ, n),
(7)
dT V (Φ, Nn (b

where

C(α, β, µ, n) =
Φ

r

Z

∞

φ(u)

−∞

β
2n
(α + βµ) + p
u
2n + β 2
2n + β 2

!



β
− Φ α + βµ + √ u
n



du,

which, by√the Dominated Convergence Theorem, converges to 0 as n → ∞,
µNn − µ) n converges in total variation distance to Φ. In
Nn (b
whence
particular, considering the Borel set Ax = [−x, x] for x ≥ 0, (7) gives


1
1
2Φ(x) − 1 − P µ
b Nn − √ x ≤ µ ≤ µ
bNn + √ x ≤ C(α, β, µ, n),
Nn
Nn
which makes it plausible to use µ
bNn for the construction of reliable confidence
intervals for µ.
Now suppose that the deterministic stopping rule (2) is followed. Then,
for each n,
p
µNn − µ)) ≤ C(µ, n),
(8)
dT V (Φ, Nn (b
where
Z ∞


√
φ(u) 1{u>−√nµ} − Φ u + 2nµ du,
C(µ, n) =
−∞

which, if µ 6= 0, √
by the Dominated
 Convergence Theorem, tends to 0 as
Nn (b
µNn − µ) n converges in total variation distance to
n → ∞, whence
Φ. In particular, considering the Borel set Ax = [−x, x] for x ≥ 0, (8) gives


1
1
2Φ(x) − 1 − P µ
bNn − √ x ≤ µ ≤ µ
bNn + √ x ≤ C(µ, n),
Nn
Nn
which, if µ 6= 0, makes it plausible to use µ
bNn for the construction of reliable
confidence intervals for µ.

5

If µ = 0, then, for each n,
p
µNn − µ)) = 1/8,
(9)
K(Φ, Nn (b

√
and
Nn (b
µNn − µ) n fails to converge weakly to Φ. Nevertheless, for each
x ∈ R+
,
0


1
1
P µ
bNn − √ x ≤ µ ≤ µ
bNn + √ x = 2Φ(x) − 1.
(10)
Nn
Nn
Thus, also in the case where µ = 0, it is plausible to use µ
bNn for the construction of reliable confidence intervals for µ.
4. Simulations

We have conducted a brief simulation study to illustrate Theorem 1, the
tables of which are√ given in Appendix B. We have studied the empirical
distribution En of Nn (b
µNn − µ), based on 1000 simulations, for both the
probabilistic stopping rule (1) (Tables 1 and 2) and the deterministic stopping rule (2) (Table 3), and different values for β, the true parameter µ,
and the number of observations n. In each case, we have compared the theoretical upper bound for the total variation distance between
the standard
√
µNn − µ), as
normal distribution and the theoretical distribution of Nn (b
given in Theorem 1, with the Kolmogorov distance
between the standard
√
µNn − µ). We have also
normal cdf and the empirical distribution of Nn (b
counted the number of times
out of √
1000 where the true
µ is


√ parameter
bNn + 1.96/ Nn , which would
contained in the interval µ
bNn − 1.96/ Nn , µ
√
be a 95%-confidence interval for µ if Nn (b
µNn − µ) were standard normally
distributed.
The predictions by Theorem 1 are confirmed by the simulation study.
More precisely, in the cases where the stopping rule is close to being deterministic and√µ = 0, the simulation study indeed points out that the
µNn − µ) deviates from a standard normal distribution
distribution of Nn (b
(red values in the tables). However, it is also confirmed that for the construction
√ of confidence intervals for µ, it is ‘harmless’ to nevertheless assume
µNn − µ) is standard normally distributed.
that Nn (b
5. Discussion
While sequential designs are in common use in medical and other applications, and while the hypothesis testing theory based there upon has
been well established for a long time, there is more confusion about parameter and precision estimation following such a sequential study. [MKA14],
[MMA16], and [MMA15] showed that the sample average is a valid estimator, with both stochastic and deterministic stopping rules, for a wide class
of normal and exponential-family-based models. They established that this
estimator, in spite of small-sample bias and the fact that there is no uniform
minimum-variance unbiased estimator, is consistent and hence asymptotically unbiased.
Building upon this work, in this paper, we have shown that the sample
average in the case of normally distributed outcomes is also asymptotically
normal in a broad range of situations. First, this is true with stochastic

6

BEN BERCKMOES AND GEERT MOLENBERGHS

stopping rule. Second, it applies in almost all deterministic stopping rule
situations within the class considered, except in the very specific case where
the normal population mean µ = 0. Note that the special status of the null
value stems from the fact that the cut-off between stopping and continuing
associated with our deterministic stopping rule is equal to zero. It can easily
be shown, should the cut-off point be shifted to a non-zero value, that then
the problematic value for µ also shifts.
We also showed that the Kolmogorov distance, for µ = 0, equals
p
µNn − µ)) = 1/8,
K(Φ, Nn (b

√
from which it follows that
Nn (b
µNn − µ) n does not converge weakly to
Φ in this case. It is enlightening that the qualitative non-convergence result
is supplemented with a quantitative determination of the deviation from
normality.
To further examine the extent of the result obtained, simulations show
that, indeed, asymptotic normality becomes more problematic when µ approaches zero and the parameter β approaches +∞, with the latter value
corresponding to a deterministic rule. However, asymptotic normality is invoked predominantly to calculate normally based confidence intervals. It is
therefore very reassuring that using such intervals for µ = 0 and a deterministic stopping rule does not lead to any noticeable effect on the coverage
probabilities.
In summary, we can conclude that for relevant classes of stopping rules,
the sample average and corresponding normal confidence interval can be
used without problem. It will be of interest to examine in more detail the
situation of outcomes that follow an exponential family distribution, other
than the normal one.
References
[EF90] Emerson, S. S.;Fleming, T. R. (1990). Parameter estimation following group sequential hypothesis testing. Biometrika 77, 875–892.
[HP88] Hughes, M.D.; Pocock, S.J. (1988). Stopping rules and estimation problems in
clinical trials. Statistics in Medicine 7, 1231–1242.
[LH99] Liu, A.; Hall, W. J. (1999). Unbiased estimation following a group sequential test.
Biometrika 86, 71–78.
[LHW06] Liu, A.; Hall, W. J.; Yu, K. F.; and Wu, C. (2006). Estimation following a group
sequential test for distributions in the one-parameter exponential family. Statistica
Sinica 16, 165–81.
[MMA15] Milanzi, E.; Molenberghs, G.; Alonso, A.; Kenward, M. G.; Tsiatis, A. A.;
Davidian, M.; Verbeke, G. Estimation after a group sequential trial. Stat. Biosci. 7
(2015), 187–205.
[MMA16] Milanzi, E.; Molenberghs, G.; Alonso, A.; Kenward, M. G.; Verbeke, G.; Tsiatis, A. A.; Davidian, M. Properties of estimators in exponential family settings with
observation-based stopping rules. J. of Biometrics Biostatist. 7, 272.
[MKA14] Molenberghs, G.; Kenward, M. G.; Aerts, M.; Verbeke, G.; Tsiatis, A. A.;
Davidian, M.; Rizopoulos, D. On random sample size, ignorability, ancillarity, completeness, separability, and degeneracy: sequential trials, random sample sizes, and
missing data. Stat. Methods Med. Res. 23 (2014), no. 1, 11–41.
[R91] Rachev, S. T. Probability metrics and the stability of stochastic models. Wiley Series
in Probability and Mathematical Statistics: Applied Probability and Statistics. John
Wiley & Sons, Ltd., Chichester, 1991.
[S78] Siegmund, D. (1978). Estimation following sequential tests. Biometrika 64, 191–199.

7

[W45] Wald, A. Sequential tests of statistical hypotheses. Ann. Math. Statist. 16 (1945),
117-186.
[Z83] Zolotarev, V. M. Probability metrics. (Russian) Teor. Veroyatnost. i Primenen. 28
(1983), no. 2, 264–287.

8

BEN BERCKMOES AND GEERT MOLENBERGHS

Appendix A. Proof of Theorem 1
Before writing down the proof of Theorem 1, we give three lemmas. Part
of Lemma 3 can be found in [MKA14], but as it belongs to the heart of our
calculations, we present a complete proof here.
Lemma 1. For A, B ∈ R,


Z ∞
A
φ(x)Φ(A + Bx)dx = Φ √
.
1 + B2
−∞

(11)

Proof. This is standard.



Lemma 2. For k, z ∈ R,
 

 



2z − k
k − z − nµ
k − 2nµ
z − nµ
√
√
√
φ √
.
φ
=φ
φ
n
n
2n
2n

(12)

Proof. This follows by a straightforward calculation.



Lemma 3. Let fNn ,KNn be the joint density of Nn and KNn . Then, for the
probabilistic stopping rule (1),


 
1
k − nµ
βk
√
fNn ,KNn (n, k) = √ φ
(13)
Φ α+
n
n
n
and
1
fNn ,KNn (2n, k) = √ φ
2n



k − 2nµ
√
2n







α+
1 − Φ  q

2n+β 2
2n

and, for the deterministic stopping rule (2),


1
k − nµ
√
fNn ,KNn (n, k) = √ φ
1{k>0}
n
n
and
1
fNn ,KNn (2n, k) = √ φ
2n



k − 2nµ
√
2n

βk
2n



 ,




k
1−Φ √
.
2n

(14)

(15)

(16)

Proof. First suppose that the probabilistic stopping rule (1) is followed.
Notice that
fNn ,KNn (n, k) = fNn ,Kn (n, k) = fKn (k)fNn |Kn (n | k),

(17)

with fKn the density of Kn , and fNn |Kn the conditional density of Nn given
Kn . Now, the Xi being independent and normally distributed with mean µ
and variance 1, we have


1
k − nµ
√
fKn (k) = √ φ
.
(18)
n
n
Furthermore, by (1),



βk
fNn |Kn (n | k) = Φ α +
n
Combining (17), (18), and (19), establishes (13).



.

(19)

9

We now establish (14). Observe that
fNn ,Kn (2n, k) = fNn ,K2n (2n, k)
= fK2n (k) − fNn ,K2n (n, k)

= fK2n (k) − fNn ,Kn (n, ·) ⋆ fP2n

i=n+1

Xi



(k),

(20)

⋆ being the convolution product, and the last equality following by indepence of Nn and Xn+1 , . . . , X2n . Using (13) and the fact that the Xi are
independent and normally distributed with mean µ and variance 1, (20)
equals
 



 

Z
1
k − z − nµ
z − nµ
k − 2nµ
1 ∞
βz
√ φ
√
√
√
φ
φ
Φ α+
dz,
−
n −∞
n
n
n
2n
2n

which, by (12),




Z ∞ 
 
1
k − 2nµ
k − 2nµ
2z − k
1
βz
√
√
=√ φ
dz.
φ √
− φ
Φ α+
n
n
2n
2n
2n
2n
−∞
(21)
2z−k
√
After performing the change of variables u = 2n , (21) reduces to

 


Z ∞
k − 2nµ
1
β
βk
√ φ
√
+ √ u du ,
φ(u)Φ α +
1−
2n
2n
2n
2n
−∞
which, by (11), coincides with





α + βk
k − 2nµ 
1
√ φ
√
1 − Φ  q 2n  .
2
2n
2n
1+ β
2n

This proves that (14) holds.
Now suppose that the detereministic stopping rule (2) is followed.
Of course, (17) and (18) continue to hold, and, by (2),
fNn |Kn (n | k) = 1{k>0} ,
from which (15) follows.
To establish (16), notice that (21) also continues to hold, which, by (15),
gives



 

Z
k − 2nµ
1
z − nµ
k − z − nµ
1 ∞
√
√
√
fNn ,Kn (2n, k) = √ φ
φ
−
φ
dz,
n 0
n
n
2n
2n
which, by (12),



Z ∞ 

1
k − 2nµ
k − 2nµ
2z − k
1
√
√
=√ φ
φ √
− φ
dz,
n
2n
2n
2n
2n
0
√
,
which, after performing the change of variables u = 2z−k
2n
"
#


Z ∞
1
k − 2nµ
√
= √ φ
1−
√ φ(u)du
2n
2n
−k/ 2n




1
k − 2nµ
k
√
= √ φ
1−Φ √
.
2n
2n
2n
This finishes the proof of (16).



10

BEN BERCKMOES AND GEERT MOLENBERGHS

Proof of Theorem 1. First suppose that the probabilistic stopping rule (1)
is followed.
For n and a Borel set A ⊂ R,
hp
i
Nn (b
µNn − µ) ∈ A
P


KNn − Nn µ
√
∈A
= P
Nn




Kn − nµ
K2n − 2nµ
√
√
= P
∈ A, Nn = n + P
∈ A, Nn = 2n . (22)
n
2n
Plugging in (13) and (14) in (22), gives
i
hp
µNn − µ) ∈ A = I1 + I2 ,
P
Nn (b

with

I1 =
and
I2 =

Z

1A

Z



1A



k − nµ
√
n

k − 2nµ
√
2n





1
√ φ
n

1
√ φ
2n





k − nµ
√
n

k − 2nµ
√
2n





(23)



βk
Φ α+
n





α+
1 − Φ  q



dk

βk
2n

2n+β 2
2n

√
shows that
Performing the change of variables u = k−nµ
n


Z
β
φ(u)Φ α + βµ + √ u du,
I1 =
n
A



 dk.

(24)

√
gives
and performing the change of variables u = k−2nµ
2n
"
!#
r
Z
2n
β
φ(u) 1 − Φ
I2 =
(α + βµ) + p
u
du.
2n + β 2
2n + β 2
A

(25)

Combining (23), (24), and (25), yields (7).
Now suppose that the deterministic stopping rule (2) is followed. Fix n
and a Borel set A ⊂ R. Of course, (22) continues to hold. Plugging in (15)
and (16) in (22) gives
i
hp
Nn (b
µNn − µ) ∈ A = L1 + L2 ,
(26)
P
with

L1 =

Z

∞

−∞

and
L2 =

Z

∞
−∞

1A



1A



k − nµ
√
n

k − 2nµ
√
2n





1
√ φ
2n

1
√ φ
n




k − nµ
√
n

k − 2nµ
√
2n





1−Φ

√
leads to
Performing the change of variables u = k−nµ
n
Z
φ(u)1{u>−√nµ} du,
L1 =
A

1{k>0} dk


k
√
2n



dk.

(27)

11
√
and performing the change of variables u = k−2nµ
yields
2n
Z
i
h

√
φ(u) 1 − Φ u + 2nµ du.
L2 =

(28)

A

Now (26), (27), and (28) give (8).
We now turn to the case µ = 0. Replacing A by ]−∞, x] in (26), (27),
and (28), shows that, for x ≥ 0,
Z x
Z x
p
φ(u)Φ(u)du −
φ(u)du
Φ(x) − P[ Nn (b
µNn − µ) ≤ x] =
−∞

=

0

2

[Φ(x)] /2 − Φ(x) + 1/2 ,

which assumes the maximal value 1/8 on [0, ∞[, and, for x ≤ 0,
Z x
p
Φ(x) − P[ Nn (b
φ(u)Φ(u)du
µNn − µ) ≤ x] =
−∞

= [Φ(x)]2 /2,

which assumes the maximal value 1/8 on ] − ∞, 0]. This proves (9). Finally,
replacing A by [−x, x] in (26), (27), and (28), shows that, for x ≥ 0,
p
2Φ(x) − 1 − P[−x ≤ Nn (b
µNn − µ) ≤ x]
Z x
Z x
=
φ(u)du
φ(u)Φ(u)du −
−x

=

0

2

[Φ(x)] /2 − [Φ(−x)]2 /2 − Φ(x) + 1/2

= 0,
which proves (10).



12

BEN BERCKMOES AND GEERT MOLENBERGHS

Appendix B. Tables from the simulation study
Table 1. Simulation study for estimation (3) for the probabilistic stopping rule (1); α = 0; β small; µ, true mean for the standard normal from which the sample is taken; n, number of observations; C = C(α, β, µ, n) in Theorem 1; K, the Kolmogorov
distance
between the standard normal cdf and the empirical cdf
√
of Nn (b
µNn − µ) based on 1000 simulations; L, number of times
out of 1000 where
√ the true parameter
√ µ is contained in the interval µ
bNn − 1.96/ Nn , µ
bNn + 1.96/ Nn (which would be a 95%√
µn − µ) were standard normally disconfidence interval if Nn (b
tributed).

β
µ
n
C
K
L
0 -10
10 0.000 0.024 947
0 -10 100 0.000 0.018 948
0 -10 1000 0.000 0.021 941
0 -1
10 0.000 0.026 947
0 -1 100 0.000 0.030 948
0 -1 1000 0.000 0.021 952
0
0
10 0.000 0.021 941
0
0 100 0.000 0.030 953
0
0 1000 0.000 0.011 958
0
1
10 0.000 0.027 954
0
1 100 0.000 0.017 957
0
1 1000 0.000 0.045 957
0 10
10 0.000 0.039 950
0 10 100 0.000 0.026 943
0 10 1000 0.000 0.024 956

β
µ
n
C
K
L
1 -10
10 0.000 0.015 958
1 -10 100 0.000 0.018 947
1 -10 1000 0.000 0.029 949
1 -1
10 0.081 0.024 960
1 -1 100 0.025 0.047 954
1 -1 1000 0.008 0.014 941
1
0
10 0.120 0.042 950
1
0 100 0.039 0.017 952
1
0 1000 0.013 0.012 954
1
1
10 0.071 0.026 957
1
1 100 0.023 0.016 955
1
1 1000 0.008 0.036 941
1 10
10 0.000 0.037 951
1 10 100 0.000 0.026 952
1 10 1000 0.000 0.028 949

Table 2. Same setup as in Table 1. Now β is moderately large.
β
µ
n
C
K
L
10 -10
10 0.000 0.010 949
10 -10 100 0.000 0.020 955
10 -10 1000 0.000 0.024 952
10 -1
10 0.002 0.015 953
10 -1 100 0.000 0.017 955
10 -1 1000 0.000 0.017 947
10
0
10 0.440 0.084 945
10
0 100 0.300 0.021 948
10
0 1000 0.120 0.047 950
10
1
10 0.001 0.028 942
10
1 100 0.000 0.026 973
10
1 1000 0.000 0.019 940
10 10
10 0,000 0.021 967
10 10 100 0.000 0.009 955
10 10 1000 0.000 0.033 948

β
µ
n
C
K
L
100 -10
10 0.000 0.019 948
100 -10 100 0.000 0.024 944
100 -10 1000 0.000 0.023 946
100 -1
10 0.001 0.039 960
100 -1 100 0.000 0.015 953
100 -1 1000 0.000 0.011 941
100
0
10 0.494 0.145 950
100
0 100 0.481 0.080 948
100
0 1000 0.437 0.068 946
100
1
10 0.001 0.035 946
100
1 100 0.000 0.011 938
100
1 1000 0.000 0.021 951
100 10
10 0.000 0.009 954
100 10 100 0,000 0.014 960
100 10 1000 0.000 0.010 954

13

Table 3. Simulation study for estimation (3) for the deterministic stopping rule (2); µ, true mean for the standard normal from which the sample is taken; n, number of observations; C = C(µ, n) in Theorem 1; K, the Kolmogorov distance
between the standard normal cdf and the empirical cdf of
√
Nn (b
µNn − µ) based on 1000 simulations; L, number of times
out of 1000 where
√ µ is contained in the inter√ the true parameter
bNn + 1.96/ Nn (which would be a 95%val µ
bNn − 1.96/ Nn , µ
√
µn − µ) were standard normally disconfidence interval if Nn (b
tributed).

β
µ
n
C
K
L
∞ -10
10 0.000 0.005 943
∞ -10 100 0.000 0.026 949
∞ -10 1000 0.000 0.029 949
∞ -1
10 0.002 0.034 952
∞ -1 100 0.000 0.025 956
∞ -1 1000 0.000 0.018 959
∞
0
10 0.250 0.130 940
∞
0 100 0.250 0.113 941
∞
0 1000 0.250 0.129 958
∞
1
10 0.002 0.022 955
∞
1 100 0.000 0.034 968
∞
1 1000 0,000 0.005 964
∞ 10
10 0.000 0.016 946
∞ 10 100 0,000 0.021 953
∞ 10 1000 0,000 0.023 952

