# How is the first column of the matrix orthogonal to all the others

$$\mathbf{X}_{n\times(r+1)} = \begin{bmatrix} 1 & (x_{11}-\bar x_1) &\cdots & (x_{1r}-\bar x_r) \\ 1 &(x_{21}-\bar x_1) &\cdots & (x_{2r}-\bar x_r) \\ \vdots & \vdots &\ddots &\vdots\\ 1 &(x_{n1}-\bar x_1) &\cdots & (x_{nr}-\bar x_r) \\ \end{bmatrix}$$

The first column is supposed to be orthogonal to all the others, but I can't see why this is true.

• Is there a "hat" missing on the subtracted X's in the first row? Probably not... probably that's the reason for your question... Mar 22, 2015 at 14:09
• @AntoniParellada I apologize. I've not understood why are you talking about "hat"?
– ABC
Mar 22, 2015 at 14:22

The test of orthogonality is the dot product. The column of ones doesn't change any of the values in the other columns. So you are left with the summation of the distance of every point in any of the remaining columns to the mean of the column, which is zero by definition.

$$\begin{bmatrix}1&1\cdots&1\end{bmatrix} \begin{bmatrix}(X_{11}−\bar{X}_1)\\(X_{12}−\bar{X}_1)\\\vdots\\(X_{1n}−\bar{X}_1)\end{bmatrix}= \sum_{i=1}^{n}(X_{1i}−\bar{X}_1) = n\,(\bar{X}_{1} -\bar{X}_{1})=0$$

Since, $$\sum_{i=1}^{n}(X_{1i})=n\bar{X}_{1}.$$

Same for other columns.

Two vectors $\mathbf{u}, \mathbf{v}$ are said to be orthogonal iff $$\langle \mathbf{u}, \mathbf{v} \rangle = 0$$ If you do this inner product $r$ times with $\mathbf{v}$ being column 1, and $\mathbf{u}$ being columns $2, ..., r+1$ respectively, you just have to add the column elements. The orthogonality follows from the properties of the arithmetic mean. If you recall, its second property is that the sum of the deviations of the sample from the mean is zero.

Proof Let $\mathbf{x} = (x_1, \dots, x_n)'$ be the observed sample of the r.v. $X$ and let $\bar{x} = n^{-1} \mathbf{x}'\boldsymbol 1$ be the sample mean. Then $$\sum_{i=1}^n (x_i - \bar{x}) = n \bar{x} - n \bar{x} = 0$$

In your case you just have samples $\mathbf{x}_1 = (x_{11}, \dots, x_{n1})', \dots, \mathbf{x}_r = (x_{1r}, \dots, x_{nr})'$ and the corresponding means are $\bar{x}_1, \dots, \bar{x}_r$. You can apply the same property taking care of adding the sample subscript:

$$\sum_{i=1}^n (x_{ij} - \bar{x}_j) = n \bar{x}_j - n \bar{x}_j = 0 \qquad j=1,\dots, r.$$

I'll change the notation of your question to make it clearer. Hope this will be useful!
{"ft_lang_label":"__label__en","ft_lang_prob":0.7593672,"math_prob":0.9998004,"size":1133,"snap":"2022-05-2022-21","text_gpt3_token_len":383,"char_repetition_ratio":0.16031887,"word_repetition_ratio":0.0,"special_character_ratio":0.36275375,"punctuation_ratio":0.13147411,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.99999726,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2022-05-20T02:05:10Z\",\"WARC-Record-ID\":\"<urn:uuid:e52299cd-41f1-4944-99a8-03190df35ee8>\",\"Content-Length\":\"237038\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:e80ab9ff-0ed8-44d1-9185-a8e814cb5118>\",\"WARC-Concurrent-To\":\"<urn:uuid:03bc5678-fa4b-4c2b-8c2f-c7dcc1258517>\",\"WARC-IP-Address\":\"151.101.65.69\",\"WARC-Target-URI\":\"https://stats.stackexchange.com/questions/142874/how-is-the-first-column-of-the-matrix-orthogonal-to-all-the-others\",\"WARC-Payload-Digest\":\"sha1:6QRJJNEPY66CVYQQC5NHBXKRBCV6ZXB6\",\"WARC-Block-Digest\":\"sha1:PV6CDEDRP5RSBFYBWXQPNDXAV72G5KCX\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2022/CC-MAIN-2022-21/CC-MAIN-2022-21_segments_1652662530553.34_warc_CC-MAIN-20220519235259-20220520025259-00146.warc.gz\"}"}