A Kolmogorov-Smirnov type test for two inter-dependent random variables
Tommy Liu




A Kolmogorov-Smirnov type test for two inter-dependent random variables
    


Tommy Liu[University of Reading tommy.liu.academic@gmail.com]

    December 30, 2023
=======================================================================




Consider n iid random variables, where ξ_1, …, ξ_n are n realisations of a random variable ξ and ζ_1, …, ζ_n are n realisations of a random variable ζ. The distribution of each realisation of ξ, that is the distribution of one ξ_i, depends on the value of the corresponding ζ_i, that is the probability P(ξ_i≤ x)=F(x,ζ_i). We develop a statistical test to see if the ξ_1, …, ξ_n are distributed according to the distribution function F(x,ζ_i). We call this new statistical test the condition Kolmogorov-Smirnov test. 







§ INTRODUCTION

Random variables often occur in experiments. It is common to have n observations of a quantity, which gives rise to n identically and independently distributed random variables ξ_1, …, ξ_n, where each ξ_i (where i=1,…,n) are different realisation of the same random variable ξ. Analysis is then done to test whether the measured ξ_i follow a particular distribution. This problem was famously answered by Kolmogorov in <cit.>.

In this paper we consider a variant of this problem. In a physical experiment, n measurements of two quantities ξ and ζ are taken, resulting in n iid pairs of random variables, with ξ_1, …, ξ_n being n realisations of ξ and ζ_1, …, ζ_n being n realisations of ζ. A distribution of ζ is not known, but the distribution of one realisation of ξ, that is ξ_i depends on the corresponding ζ_i.  

We want to test whether the pairs of (ξ_i,ζ_i) we collected follow a particular joint distribution. The answer we offer is very similar to the original Kolmogorov-Smirnov test.




§ KOLMOGOROV-SMIRNOV TEST

First we recall well known results about the Kolmogorov-Smirnov statistic and the Kolmogorov-Smirnov test <cit.>. 
Let

    ξ_1, ξ_2, …, ξ_n

be n independently and identically distributed real random variables. 
Each ξ_i is distributed with  CDF F(·) as in 

    P(ξ_i≤ x)=F(x)=∫^x_-∞ f(s) ds

where f(·) is the PDF. 
Define a function by F_n(·) by 

    F_n(x)=
    1/n∑_i=1^n
    1_(-∞,x]
    (ξ_i)

where 1_A is the indicator function for a set A. 
Thus F_n(·) is the empirical CDF. 
Consider the distance between the real and the empirical CDF through the supremum metric on the space of real functions

    D_n=
    ‖
    F_n-F
    ‖_∞
    =sup_x∈ℝ|
    1/n∑_i=1^n
    1_(-∞,x]
    (ξ_i)
    -F(x)
    |

where D_n is called the Kolmogorov-Smirnov statistic or KS statistic. 
We define what we mean by the null hypothesis. 


Let ξ_1,ξ_2,…,ξ_n be n real random variables.
The null hypothesis is 
that each ξ_i is independently distributed with CDF F(x).


We want to know how large or small D_n needs to be before deciding whether to reject the null hypothesis. 
The following Theorem offers a remarkable answer to this problem.


Suppose the null hypothesis is true,
then the distribution of D_n depends only on n. 


Notice that D_n is in itself a real random variable. 
The PDF and CDF of D_n is a function of n only, and will be the same whatever F(·) is.
This distribution is called the KS distribution and tables are available upto n=100. 
There is a Theorem which describes the asymptotic behaviour of the KS distribution. 


In the limit n ⟶∞, √(n)D_n is asymptotically Kolmogorov distributed with the CDF 

    Q(x)=1-2∑_k=1^∞ (-1)^k-1e^-2k^2x^2

that is to say 

    lim_n⟶∞ P(√(n) D_n ≤ x)=Q(x).







§ CONDITIONAL KOLMOGOROV-SMIRNOV TEST

The conditional Kolmogorov-Smirnov test was first developed in the thesis <cit.>. 
Let ζ_1,ζ_2,…,ζ_n be n iid real random variables. 
They are n empirical observations of a random variable ζ. 
Now suppose that each of the ξ_1,ξ_2,…,ξ_n is conditioned and dependent on the corresponding ζ_1,ζ_2,…,ζ_n.
The conditional CDF F(·,·) is

    P(ξ_i≤ x | ζ_i)=F_ζ_i(x)=∫^x_-∞f(s,ζ_i) ds

But ξ_1,ξ_2,…,ξ_n are empirical measurements of the same random variable ξ. 
The CDF for ξ is 

    P(ξ≤ x)=F(x)=∫^x_-∞∫^u=+∞_u=-∞f(s,u)m(u) du ds

where m(·) is the PDF for ζ, that is 

    P(ζ∈ A)=∫_Am(s) ds

In our context we have the problem that the random variables are not identically distributed under the null hypothesis. 
The  ξ_1,ξ_2,…,ξ_n and ζ_1,ζ_2,…,ζ_n are obtained experimentally and F_ζ_i(ξ_i) can be  calculated but a PDF for ζ_i, that is m(·), has no easy expression. 
We still want to perform a statistics test that is similar to the KS test even in such situations where the distribution m(·) of ζ is unknown. 
First we define what we call the total null hypothesis and the conditional null hypothesis. 


Let ξ_1, ξ_2, …, ξ_n be n empirical observations of a random variable ξ. 
The total null hypothesis is that
ξ is distributed with the CDF F(·).
The conditional null is that
 each ξ_i is distributed with the conditional CDF F_ζ_i(·).


A new statistical test is developed, which is similar to the KS test. 


Suppose the conditional null hypothesis is true. 
Let F_ζ_i(·) be continuous. 
Let S_n be the statistic given by 

    S_n=sup_x∈[0,1]|
    1/n∑^n_i=11_[0,x](
    F_ζ_i(ξ_i)
    )
    -x
    |

then S_n is KS distributed. 



Denote

    Y_i=F_ζ_i(ξ_i)

which means 

    P(Y_i≤ x)   =P(F_ζ_i(ξ_i)≤ x)
       =P(ξ_i≤ F^-1_ζ_i(x))
       =F_ζ_i(F^-1_ζ_i(x))
       =x

and 0≤ Y_i≤ 1, so Y_i is uniformly distributed on [0,1]. 
Note that F_ζ_i(·) is a function of one variable only. 
Let 

    F_n(x)=
    1/n∑_i=1^n
    1_[0,x]
    (Y_i)
    =
    1/n∑_i=1^n
    1_[0,x](F_ζ_i(ξ))

where F_n(·) is the empirical CDF of a uniformly distributed random variable, computed using n observations. 
The statistic S_n is the suprenum metric

    S_n=‖ F_n-x   ‖_∞
    =sup_x∈[0,1]|
    1/n∑^n_i=11_[0,x](
    F_ζ_i(ξ_i)
    )
    -x
    |

So clearly S_n is KS distributed. 


We call S_n the conditional KS statistic. 
Compare this to the original KS statistic, which under the assumption of the total null hypothesis can be rewritten as 

    D_n=sup_x∈ℝ|
    1/n∑_i=1^n
    1_(-∞,x]
    (ξ_i)
    -F(x)
    |
    =sup_x∈[0,1]|
    1/n∑^n_i=11_[0,x](
    F(ξ_i)
    )
    -x
    |

When both the total and conditional hypothesis are true D_n and S_n are KS distributed, that is 

    P(D_n∈ A )=P(S_n∈ A )
      and   
    P(D_n ≤ x)=P(S_n≤ x)

The subtlety here is that
D_n and S_n are different objects, yet they have the same distribution. 
D_n is KS distributed under the total null hypothesis, 
whereas S_n is KS distributed under the conditional null hypothesis. 
This can be explained in another way.
We have n experimental observations of a random variable ξ denoted by ξ_1,ξ_2, …, ξ_n and each are conditioned on observations of another random variable ζ denoted by ζ_1, ζ_2, …, ζ_n. 
The D_n is KS distributed if the random variable ξ is distributed by CDF F(·), 
but the S_n is KS distributed if each ξ_i is conditionally distributed by the CDF F_ζ_i(·). 



§ CONCLUSION

An example of the conditional Kolmogorov-Smirnov test reliably working with experimental data is studied in the paper <cit.>, which is also the first time of it being implemented. 




tocsectionReferences
ieeetr












