Homomorphic Encryption operation for regression I must be missing something, and I am sorry if this is a novice question...

I have seen in places (eg. https://crypto.stanford.edu/~dwu4/talks/SecurityLunch0214.pdf page 13) that linear regression is possible on the cipher text of a fully Homomorphic Encryption system. 

My question is:
if you can generate the inverse of the matrix (X^T X), then can't you generate the cipher version of the identity matrix, and determine what E(1) is? And if you have E(1) can't you generate E(2) ... E(n) using either encrypted addition or plain text multiplication?
if you can generate the inverse of the matrix (X^T X), then can't you generate the cypher version of the identity matrix, and determine what E(1) is?


I believe that the point you're missing is that homomorphic encryption is probabilistic; there are lots of legal representations of $E(1)$.  The encryption process takes the plaintext (1 in this case), and some random values (we call them random coins); different values for the coins give you different ciphertexts.

Yes, you can generate values which you know is an encryption of 1 (in fact, they can be a lot easier than doing a full matrix multiply, if you have the public key, you can just take 1 and encrypt it yourself).

And, because there are many different ways of representing $E(1)$, just because you know one of them doesn't mean that you can recognize the others (in fact, we hope there's no such way, otherwise (as you implied) the whole system is broken).