# How to whiten the data using principal component analysis?

I want to transform my data $\mathbf X$ such that the variances will be one and the covariances will be zero (i.e I want to whiten the data). Furthermore the means should be zero.

I know I will get there by doing Z-standardization and PCA-transformation, but in which order should I do them?

I should add that the composed whitening transformation should have the form $\mathbf{x} \mapsto W\mathbf{x} + \mathbf{b}$.

Is there a method similar to PCA which does exactly both these transformations and gives me a formula of the form above?

• (My first comment was based on misreading your question.) PCA gives you zero covariances; you can standardize the PCs afterwards if you wish. It sounds an odd thing to do, but you can do it. – Nick Cox Apr 30 '14 at 11:59
• @NickCox Maybe it seems odd because the transformed data is then spherical, which seems uninformative. However, it is the transformation I need to know, and not the end result. Still I don't know what the transformation would look like. I'm still reading up on PCA, though. – Angelorf Apr 30 '14 at 12:24

First, you get the mean zero by subtracting the mean $\boldsymbol \mu = \frac{1}{N}\sum \mathbf{x}$.

Second, you get the covariances zero by doing PCA. If $\boldsymbol \Sigma$ is the covariance matrix of your data, then PCA amounts to performing an eigendecomposition $\boldsymbol \Sigma = \mathbf{U} \boldsymbol \Lambda \mathbf{U}^\top$, where $\mathbf{U}$ is an orthogonal rotation matrix composed of eigenvectors of $\boldsymbol \Sigma$, and $\boldsymbol \Lambda$ is a diagonal matrix with eigenvalues on the diagonal. Matrix $\mathbf{U}^\top$ gives a rotation needed to de-correlate the data (i.e. maps the original features to principal components).

Third, after the rotation each component will have variance given by a corresponding eigenvalue. So to make variances equal to $1$, you need to divide by the square root of $\boldsymbol \Lambda$.

All together, the whitening transformation is $\mathbf{x} \mapsto \boldsymbol \Lambda^{-1/2} \mathbf{U}^\top (\mathbf{x} - \boldsymbol \mu)$. You can open the brackets to get the form you are looking for.

Update. See also this later thread for more details: What is the difference between ZCA whitening and PCA whitening?

• I think you need to divide by the square roots of the eigenvalues, as it is a matter of scaling by SD, not variance. – Nick Cox Apr 30 '14 at 13:52
• @NickCox: yes, of course you are right. I corrected my answer. Thank you! – amoeba says Reinstate Monica Apr 30 '14 at 14:07
• I have empirically verified the formula. Thanks for helping me! – Angelorf Apr 30 '14 at 14:38
{"ft_lang_label":"__label__en","ft_lang_prob":0.8072408,"math_prob":0.98614925,"size":1172,"snap":"2019-51-2020-05","text_gpt3_token_len":300,"char_repetition_ratio":0.16181506,"word_repetition_ratio":0.0,"special_character_ratio":0.24061434,"punctuation_ratio":0.10194175,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9992342,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-01-20T18:21:13Z\",\"WARC-Record-ID\":\"<urn:uuid:ee0d7453-cf9c-494e-8610-e75354ab76b7>\",\"Content-Length\":\"144107\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:154720dc-9d66-42eb-b7ef-fc271d9d8ff2>\",\"WARC-Concurrent-To\":\"<urn:uuid:f9dc0e47-9a85-401f-a494-0be5d916b0b2>\",\"WARC-IP-Address\":\"151.101.129.69\",\"WARC-Target-URI\":\"https://stats.stackexchange.com/questions/95806/how-to-whiten-the-data-using-principal-component-analysis\",\"WARC-Payload-Digest\":\"sha1:6PGHIROVPVGPQ4OJLNZQJV6JNOOUJ34L\",\"WARC-Block-Digest\":\"sha1:3H2VPOGDFAK744JM7CHGRXY56PUGJXFE\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-05/CC-MAIN-2020-05_segments_1579250599718.13_warc_CC-MAIN-20200120165335-20200120194335-00489.warc.gz\"}"}