Heinrich Hartmann

# A Categorical Perspective On Covariance

Written on 2014-12-08

For functions $f,g$ on a measure space $\Omega, \mu$ (e.g. a real interval $[0,1]$) there is a well known scalar product

This scalar product is of fundamental importance to the study of function and operators on measure spaces (Function Analysis). For example, there is a rich theory of how to decomposes functions on an interval into orthogonal Fourier-components.

If $\mu$ is a probability measure, we can regard $f,g$ as random variables. In this setting the viewpoint changes quite a bit. We give $f,g$ new names: $X,Y$. We are no longer interested in $\Omega$, and also rarely consider the scalar product. Instead the focus lies on expectation

and the covariance

This note considers the relation between these the probabilistic notions and the functional point of view.

## Constant Split

Let $\Omega, P$ be a probability space. Consider the vector space of square integrable functions:

See e.g. wikipedia for the details. We have the integral linear operator

The inclusion of the constant functions $\iota$ leaves us with a sequence:

whose composition $I \circ \iota = id_\IR$, since $P$ is a probability measure. We get an induced splitting of $\LL$ into constant functions plus functions with integral $I = 0$:

A function $x$ can be decomposed accordingly into its constant part $I(x)$ and its integral-0 part: $N(x) = x - I(x)$.

## Bilinear Forms

The space $\LL$ comes with a scalar product

which is non-degenerate and complete and thus makes this space into a Hilbert Space.

The covariance product is defined as

This product is bilinear but degenerate. The radical of $Cov$ are precisely the (almost) constant functions $\iota(\IR)$.

In the light of the above decomposition, we see that $Cov$ is the restriction of $S$ to $\LL_0$ extended back to $\LL$ using the projection $N(x) = x - I(x)$.

## Conclusion

• The space of integral-0 function $\LL_0$ together with the covariance product $Cov$ is a Hilbert Space.

• The natural inclusion $\LL_0 \rightarrow \LL$ is isometric with adjoint linear operator $N(x) = x - I(x)$.

• There is an orthogonal direct sum decomposition:

## Update 2015-05-17: Correlation as Cosine

The Pearson correlation is defined as:

It measures the linear dependece between two random variables. E.g. in the case of a discrete probability measure obtained from a sample, the correlation is the ratio between explained variance in a linear regression and total variance of the sample cf. wikipedia. Correlated (\rho ~ 0.88) and non-corrleated (\rho ~ -0.14) samples.

In analogy to the Euclidean plane, we define the cosine similarity between two functions by

Hence for centered functions $x,y \in \LL_0$ we have

which gives a surprising relation between two different geometric interpretations of the same data:

• Regrssion line of coefficient pairs $(x_i,y_i)$ in $\IR^2$
• Angle between vectors $(x_1,\dots,x_n)$ and $(y_1,\dots,y_n)$ in $\IR^n$.

Note, that for cetnered functions ($I(x) = I(y) = 0$) the regression line will always pass through the origin and the slope can be calculated to

However, at the moment I do not see how this helps to understand the above observation. My feeling is, that there should be a more conceptual reason for it. By interpreting $x,y$ as maps $\IR^2 \lra \IR^n$ we can bring duality theory for vector spaces into play and maybe gain more insight form this perspective.

View the version history of this post on GitHub.
Comments have been disabled until the dust around the GDPR settled.
{"ft_lang_label":"__label__en","ft_lang_prob":0.90679854,"math_prob":0.9994541,"size":2707,"snap":"2019-35-2019-39","text_gpt3_token_len":519,"char_repetition_ratio":0.12689604,"word_repetition_ratio":0.0,"special_character_ratio":0.18618397,"punctuation_ratio":0.09631147,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9997377,"pos_list":[0,1,2],"im_url_duplicate_count":[null,1,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2019-09-16T05:39:20Z\",\"WARC-Record-ID\":\"<urn:uuid:1c88cee5-40b0-45d0-9eb9-315840045e49>\",\"Content-Length\":\"11954\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:5ea4875e-c839-4af7-8aec-c2c547c85167>\",\"WARC-Concurrent-To\":\"<urn:uuid:cef2a527-9f49-4084-bdf5-57837cb29631>\",\"WARC-IP-Address\":\"192.30.252.154\",\"WARC-Target-URI\":\"http://heinrichhartmann.com/math/blog/2014/12/08/A-categorical-perspective-on-Covariance.html\",\"WARC-Payload-Digest\":\"sha1:IJTY5EOXEMCZDMMS3GOTGHTZKYHN4KIY\",\"WARC-Block-Digest\":\"sha1:2KI3LLSKYIVVAJ632NQG63K7K4DJ4MM7\",\"WARC-Identified-Payload-Type\":\"application/xhtml+xml\",\"warc_filename\":\"/cc_download/warc_2019/CC-MAIN-2019-39/CC-MAIN-2019-39_segments_1568514572484.20_warc_CC-MAIN-20190916035549-20190916061549-00299.warc.gz\"}"}