A Kolmogorov-Smirnov type test for two inter-dependent
random variables
Tommy Liu∗

arXiv:1802.09899v1 [math.PR] 27 Feb 2018

February 28, 2018

Abstract
Consider n iid random variables, where ξ1 , . . . , ξn are n realisations of a random variable
ξ and ζ1 , . . . , ζn are n realisations of a random variable ζ. The distribution of each realisation
of ξ, that is the distribution of one ξi , depends on the value of the corresponding ζi , that is
the probability P (ξi ≤ x) = F (x, ζi ). We develop a statistical test to see if the ξ1 , . . . , ξn are
distributed according to the distribution function F (x, ζi ). We call this new statistical test
the condition Kolmogorov-Smirnov test.

Contents
1 Introduction

1

2 Kolmogorov-Smirnov Test

2

3 Conditional Kolmogorov-Smirnov Test

2

4 Conclusion

4

References

4

1

Introduction

Random variables often occur in experiments. It is common to have n observations of a quantity,
which gives rise to n identically and independently distributed random variables ξ1 , . . . , ξn , where
each ξi (where i = 1, . . . , n) are different realisation of the same random variable ξ. Analysis is
then done to test whether the measured ξi follow a particular distribution. This problem was
famously answered by Kolmogorov in [1].
In this paper we consider a variant of this problem. In a physical experiment, n measurements
of two quantities ξ and ζ are taken, resulting in n iid pairs of random variables, with ξ1 , . . . , ξn
being n realisations of ξ and ζ1 , . . . , ζn being n realisations of ζ. A distribution of ζ is not known,
but the distribution of one realisation of ξ, that is ξi depends on the corresponding ζi .
We want to test whether the pairs of (ξi , ζi ) we collected follow a particular joint distribution.
The answer we offer is very similar to the original Kolmogorov-Smirnov test.

∗ University

of Reading tommy.liu.academic@gmail.com

1

2

Kolmogorov-Smirnov Test

First we recall well known results about the Kolmogorov-Smirnov statistic and the KolmogorovSmirnov test [1]. Let
ξ1 , ξ2 , . . . , ξn
be n independently and identically distributed real random variables. Each ξi is distributed with
CDF F (·) as in
Z x
P (ξi ≤ x) = F (x) =
f (s) ds
−∞

where f (·) is the PDF. Define a function by Fn (·) by
n

Fn (x) =

1X
1(−∞,x] (ξi )
n i=1

where 1A is the indicator function for a set A. Thus Fn (·) is the empirical CDF. Consider the
distance between the real and the empirical CDF through the supremum metric on the space of
real functions
n

Dn = kFn − F k∞ = sup
x∈R

1X
1(−∞,x] (ξi ) − F (x)
n i=1

where Dn is called the Kolmogorov-Smirnov statistic or KS statistic. We define what we mean by
the null hypothesis.
Definition 2.1. Let ξ1 , ξ2 , . . . , ξn be n real random variables. The null hypothesis is that each ξi
is independently distributed with CDF F (x).
We want to know how large or small Dn needs to be before deciding whether to reject the null
hypothesis. The following Theorem offers a remarkable answer to this problem.
Theorem 2.2. Suppose the null hypothesis is true, then the distribution of Dn depends only on
n.
Notice that Dn is in itself a real random variable. The PDF and CDF of Dn is a function of n
only, and will be the same whatever F (·) is. This distribution is called the KS distribution and
tables are available upto n = 100. There is a Theorem which describes the asymptotic behaviour
of the KS distribution.
√
Theorem 2.3. In the limit n −→ ∞, nDn is asymptotically Kolmogorov distributed with the
CDF
∞
X
2 2
Q(x) = 1 − 2
(−1)k−1 e−2k x
k=1

that is to say

√
lim P ( nDn ≤ x) = Q(x).

n−→∞

3

Conditional Kolmogorov-Smirnov Test

The conditional Kolmogorov-Smirnov test was first developed in the thesis [2]. Let ζ1 , ζ2 , . . . , ζn be
n iid real random variables. They are n empirical observations of a random variable ζ. Now suppose that each of the ξ1 , ξ2 , . . . , ξn is conditioned and dependent on the corresponding ζ1 , ζ2 , . . . , ζn .
The conditional CDF F (·, ·) is
Z x
f (s, ζi ) ds
P (ξi ≤ x | ζi ) = Fζi (x) =
−∞

2

But ξ1 , ξ2 , . . . , ξn are empirical measurements of the same random variable ξ. The CDF for ξ is
Z x Z u=+∞
f (s, u)m(u) du ds
P (ξ ≤ x) = F (x) =
u=−∞

−∞

where m(·) is the PDF for ζ, that is
P (ζ ∈ A) =

Z

m(s) ds
A

In our context we have the problem that the random variables are not identically distributed under
the null hypothesis. The ξ1 , ξ2 , . . . , ξn and ζ1 , ζ2 , . . . , ζn are obtained experimentally and Fζi (ξi )
can be calculated but a PDF for ζi , that is m(·), has no easy expression. We still want to perform
a statistics test that is similar to the KS test even in such situations where the distribution m(·)
of ζ is unknown. First we define what we call the total null hypothesis and the conditional null
hypothesis.
Definition 3.1. Let ξ1 , ξ2 , . . . , ξn be n empirical observations of a random variable ξ. The total
null hypothesis is that ξ is distributed with the CDF F (·). The conditional null is that each ξi is
distributed with the conditional CDF Fζi (·).
A new statistical test is developed, which is similar to the KS test.
Theorem 3.2. Suppose the conditional null hypothesis is true. Let Fζi (·) be continuous. Let Sn
be the statistic given by
n

Sn = sup
x∈[0,1]

1X
1[0,x] (Fζi (ξi )) − x
n i=1

then Sn is KS distributed.
Proof. Denote
Yi = Fζi (ξi )
which means
P (Yi ≤ x) = P (Fζi (ξi ) ≤ x)


(x)
= P ξi ≤ Fζ−1
i


−1
= Fζi Fζi (x)
=x
and 0 ≤ Yi ≤ 1, so Yi is uniformly distributed on [0, 1]. Note that Fζi (·) is a function of one
variable only. Let
n

Fn (x) =

n

1X
1X
1[0,x] (Yi ) =
1[0,x] (Fζi (ξ))
n i=1
n i=1

where Fn (·) is the empirical CDF of a uniformly distributed random variable, computed using n
observations. The statistic Sn is the suprenum metric
n

Sn = kFn − xk∞ = sup

x∈[0,1]

So clearly Sn is KS distributed.
3

1X
1[0,x] (Fζi (ξi )) − x
n i=1

We call Sn the conditional KS statistic. Compare this to the original KS statistic, which under
the assumption of the total null hypothesis can be rewritten as
n

n

1X
1X
Dn = sup
1(−∞,x] (ξi ) − F (x) = sup
1[0,x] (F (ξi )) − x
x∈R n i=1
x∈[0,1] n i=1
When both the total and conditional hypothesis are true Dn and Sn are KS distributed, that is
P (Dn ∈ A) = P (Sn ∈ A)

and P (Dn ≤ x) = P (Sn ≤ x)

The subtlety here is that Dn and Sn are different objects, yet they have the same distribution.
Dn is KS distributed under the total null hypothesis, whereas Sn is KS distributed under the
conditional null hypothesis. This can be explained in another way. We have n experimental observations of a random variable ξ denoted by ξ1 , ξ2 , . . . , ξn and each are conditioned on observations
of another random variable ζ denoted by ζ1 , ζ2 , . . . , ζn . The Dn is KS distributed if the random
variable ξ is distributed by CDF F (·), but the Sn is KS distributed if each ξi is conditionally
distributed by the CDF Fζi (·).

4

Conclusion

An example of the conditional Kolmogorov-Smirnov test reliably working with experimental data
is studied in the paper [3], which is also the first time of it being implemented.

References
[1] A. N. Kolmogorov, “Sulla Determinazione Empirica di una Legge di Distribuzione,” Giornale
dell’Istituto Italiano degli Attuari, vol. 4, pp. 83–91, 1933.
[2] T. Liu, Stochastic Resonance for a Model with Two Pathways. PhD thesis, University of
Reading, 2016. Available on arXiv.
[3] T. Kuna, V. Lucarini, and T. Liu, “A comparative study of stochastic resonance for a model
with two pathways by escape times, linear response, invariant measures and the conditional
kolmogorov-smirnov test,” 2018. Available on arXiv.

4

