Asymptotic Distribution of Centralized $r$ When Sampling from Cauchy
Veson Lee Jan Vrbik



1]Veson Lee
1]Jan Vrbik
[1]Department of Mathematics and Statistics
 Brock University, Canada

Asymptotic Distribution of Centralized r When Sampling from Cauchy
    [
    September 15, 2018
==================================================================




Assume that X and Y are independent random variables, each having a Cauchy distribution with a known median. Taking a random independent sample of size n of each X and Y, one can then compute their centralized empirical correlation coefficient r. Analytically investigating the sampling distribution of this r appears possible only in the large n limit; this is what we have done in this article, deriving several new and interesting results.




§ INTRODUCTION

It can be easily shown, based on Central Limit Theorem, that the sampling distribution of the usual sample correlation coefficient further multiplied by √(n) i.e.

    √(n)·∑_i=1^n(X_iY_i-∑_i=1^nX_i·∑_i=1^nY_i/n)/√(∑_i=1^nX_i^2-(∑_i=1^nX_i)^2/n)√(∑_i=1^nY_i^2-(∑_i=1^nY_i)^2/n)

tends to, when n→∞ and X_i and Y_i are independent (both within and between), the standard Normal distribution whenever both X and Y have finite means and variances. The situation changes dramatically when sampling from a Cauchy distribution. Investigating what happens in that case is rather difficult; to simplify the task, we assume that the medians of X and Y are known, and can therefore be subtracted from the X_i and Y_i values. This amounts to assuming that the two Cauchy distributions have zero medians and it is then sufficient to define what we call the centralized sample correlation coefficient as 

    r_c=∑_i=1^nX_iY_i/√(∑_i=1^nX_i^2)√(∑_i=1^nY_i^2)

It is obvious that the second parameter (the quartile deviation) of each of the two Cauchy distributions cancels out of the last expression; we can thus assume (without a loss of generality), that both the X_i and Y_i are drawn independently from a Cauchy distribution with median equal to 0 and the quartile deviation equal to 1. The objective of this article is to find the asymptotic (i.e. large-n) behaviour of r_c.  This centralized r_c is also known to others in the computer science, big data and data science fields as the cosine similarity measure. It has applications in text mining, data mining and information retrieval <cit.>; an investigation of the statistical distribution related to this centralized r_c has been done by <cit.>.



§ RELATED SAMPLING DISTRIBUTIONS

To obtain its asymptotic distribution, we must first explore the distribution of the individual terms of r_c. We start by quoting the probability density function (PDF) and the characteristic function (CF) of each X_i and Y_i: 

    f(x)=1/π(x^2+1)

and 

    φ(t)=exp(-|t|)

respectively (these are well-known results <cit.>).

This implies that the PDF each X_i^2 and Y_i^2 (denoted Z) is given by 

    f(z)= 1/π√(z)(z+1)    z>0
    
    0    elsewhere

and the distribution function of Z is given by 

    F_X^2(z)=(X^2≤ z)=1/π∫_-√(z)^√(z)x̣/1+x^2=2/πarctan√(z)


The corresponding CF is then

    φ_X^2(t)=exp(-it)[1-erf(√(-it))]

and can be found by taking the appropriate Fourier transform of the PDF <cit.>; here we rely on computer software (such as Maple or Mathematica) to provide these.

To obtain the asymptotic CF of each lim_n→∞∑_i=1^nX_i^2/n^2 and lim_n→∞∑_i=1^nY_i^2/n^2 (denoted W), we have to raise (<ref>) to the power of n, replace t by t/n^2 and then take the n→∞ limit of the resulting expression (note that only by dividing by n^2 can one reach a finite limit - that is how these `normalizing factors' are found in general).

This yields 

    φ_W(t)=exp(-2√(-it/π))

since erf(x)≃2x/√(π) for small x. The appropriate inverse Fourier transform converts this CF to the corresponding PDF, namely 

    f(w)=exp(-1/π w)/π w^3/2    w>0
    
    0    elsewhere

One can show via Monte Carlo simulation that this constitutes a fairly accurate approximation for the PDFs of ∑_i=1^nX_i^2/n^2 and ∑_i=1^nY_i^2/n^2 even for relatively small n values (n≥30). This is due to the fact that the corresponding CF can be expanded in increasing powers of 1/n. The error of the approximation is thus of the O(1/n) type which is faster than O(1/√(n)) of the Central Limit Theorem.

The PDF of W can be readily converted into the (asymptotic) PDF of each lim_n→∞n/√(∑_i=1^nX_i^2) and lim_n→∞n/√(∑_i=1^nY_i^2) (denoted U and equal to 1/√(W)) using a simple univariate transformation, yielding 


    f(u)=exp(-u^2/π)/π    u>0
    
    0    elsewhere

resulting in a half-normal distribution.

Finally, each of the X_iY_i (denoted S) has a CF found by 

    ∫_-∞^∞∫_-∞^∞exp(ixyt)·1/π(x^2+1)·1/π(y^2+1)x̣ỵ

resulting in 

    φ_S(t)=2(sin|t|Ci|t|-cos|t|Si|t|)/π+cos|t|    t≠0
    
    1     t=0

which corresponds to the relatively simple PDF 

    f(s)=
    ln(s^2)/π^2(s^2-1)    v=ℝ∖{ -1,0,1}
     1/π^2    v={ -1,1}
    
    0    elsewhere


Assuming that each X_iY_i (and also their sum) are asymptotically independent of n/√(∑_i=1^nX_i^2) and n/√(∑_i=1^nY_i^2) (something we have been able to verify only empirically), we now find the CF of 

    X_iY_i·n/√(∑_i=1^nX_i^2)·n/√(∑_i=1^nY_i^2)≃ S· U_1· U_2

where U_1 and U_2 are independent random variables, each having an asymptotic PDF of (<ref>). This resulting approximate CF is computed by 

    4/π^4∫_0^∞∫_0^∞∫_-∞^∞cos(u_1u_2st)
    ·exp(-u_1^2+u_2^2/π)·lns^2/s^2-1ṣụ_1ụ_2

where the usual exp(iu_1u_2st) has beeen replaced by cos(u_1u_2st), since the resulting distribution is symmetric, implying that it is CF has no imaginary part.

To simplify the ụ_1ụ_2 integration, we perform it in the usual polar coordinates (denoted R and Θ) getting 

    4/π^4∫_0^∞
    int_0^π/2∫_-∞^∞
    Rcos(stR^2sin2Θ/2) ·exp(-R^2/π)·lns^2/s^2-1ṣΘ̣Ṛ
        =
    2/π^3∫_0^π/2∫_-∞^∞1/1+(π stsin2Θ/2)^2·lns^2/s^2-1ṣΘ̣
        =
    2/π∫_0^π/21+|t|sin2Θln(π|t|sin2Θ/2)/1+(π tsin2Θ/2)^2Θ̣




§ ASYMPTOTIC PDF OF R_C


To get an approximate CF of 

    ∑_i=1^nX_iY_i/nln n·n/√(∑_i=1^nX_i^2)·n/√(∑_i=1^nY_i^2).

Note the unusual normalizing factor – the only way to achieve a finite limit. We need to first raise (<ref>) to the power of n while simultaneously replacing t by t/nlnn, taking the n→∞ limit and finally evaluating the remaining integral. These operations can be carried out in any order in this particular case - something not true in general.

This yields before the Θ̣ integration

    exp(-|t|sin2Θ) + 𝒪(1/lnn)

Since the error of this approximation is proportional to 1/lnn, the actual convergence of the subsequent result is expected to be rather slow (reaching a good accuracy only when n is at least a few hundred). One can recover a part of this error (trying to recover the full 1/lnn proportional term would make the resulting expression too cumbersome) by replacing (<ref>) with the more accurate

    exp(-a|t|sin2Θ)

where

    a=1-lnπ/lnn .


To complete the computation, we evaluate 

    φ_r_c(t)=2/π∫_0^π/2exp(-a|t|sin2Θ)Θ̣=I_0(a|t|)-L_0(a|t|)

where I_0 and L_0 are the Bessel and modified Struve functions respectively. Converting to PDF results in: 

    f(r_c)=2ln(a+√(a^2+r_c^2))-ln r_c^2/π^2√(a^2+r_c^2)

which is our final answer for the approximate distribution of the sum of (<ref>) terms, which is the same as

    n· r_c/lnn .

Note that similarly to the Cauchy distribution itself (<ref>) has an indefinite mean and an infinite variance.



§ MONTE CARLO VERIFICATION

We will now verify the accuracy of our approximation by randomly generating 100,000 values of (<ref>), using n=400. The following Mathematica program does this and plots the corresponding histogram together with our approximate PDF (<ref>).


The two results are in good agreement, as can be seen in the Figure <ref>.




§ CONCLUSION

We have derived an asymptotic distribution of the centralized sample correlation coefficient when sampling from Cauchy distribution, discovering some of its unusual properties. As a byproduct, several other interesting distributions have been introduced in the process.

plain
1
key-1 Tan, P-N, Steinbach, M, Kumar, Vipin: Introduction
to Data Mining. Pearson Addison Wesley, Boston (2005)

key-2 Singhal, A: Modern Information Retrieval: A Brief
Overview. Bul. IEEE on Data Engineering Vol 24. No 4. 35-43 (2001)

key-3 Giller, L. G: The Statistical Properties of Random
Bitstreams and the Sampling Distribution of Cosine Similarity. (2012)
doi:10.2139/ssrn.2167044

key-4 Johnson, N. L, Kotz, S, Balakrishnan, N: Continuous
Univariate Distributions, Volume 1, Second Edition. Wiley, New York
(1994)

key-5 Billingsley, P: Probability and Measure, 3rd Edition.
John Wiley & Sons, New York (1995) 



