How to implement ZCA Whitening? Python
Question: Im trying to implement **ZCA whitening** and found some articles to do it, but
they are a bit confusing.. can someone shine a light for me?

Any tip or help is appreciated!

Here is the articles i read :

<http://courses.media.mit.edu/2010fall/mas622j/whiten.pdf>
<http://bbabenko.tumblr.com/post/86756017649/learning-low-level-vision-
feautres-in-10-lines-of>

I tried several things but most of them i didnt understand and i got locked at
some step. Right now i have this as base to start again :

    
    
    dtype = np.float32
    data = np.loadtxt("../inputData/train.csv", dtype=dtype, delimiter=',', skiprows=1)
    img = ((data[1,1:]).reshape((28,28)).astype('uint8')*255)
    


Answer: Is your data stored in an mxn matrix? Where m is the dimension of the data and
n are the total number of cases? If that's not the case, you should resize
your data. For instance if your images are of size 28x28 and you have only one
image, you should have a 1x784 vector. You could use this function:

    
    
    import numpy as np
    
    def flatten_matrix(matrix):
        vector = matrix.flatten(1)
        vector = vector.reshape(1, len(vector))
        return vector
    

Then you apply ZCA Whitening to your training set using:

    
    
    def zca_whitening(inputs):
        sigma = np.dot(inputs, inputs.T)/inputs.shape[1] #Correlation matrix
        U,S,V = np.linalg.svd(sigma) #Singular Value Decomposition
        epsilon = 0.1                #Whitening constant, it prevents division by zero
        ZCAMatrix = np.dot(np.dot(U, np.diag(1.0/np.sqrt(np.diag(S) + epsilon))), U.T)                     #ZCA Whitening matrix
        return np.dot(ZCAMatrix, inputs)   #Data whitening
    

It is important to save the `ZCAMatrix` matrix, you should multiply your test
cases if you want to predict after training the Neural Net.

Finally, I invite you to take the Stanford UFLDL Tutorials at
<http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial> or
<http://ufldl.stanford.edu/tutorial/> . They have pretty good explanations and
also some programming exercises on MATLAB, however, almost all the functions
found on MATLAB are on Numpy by the same name. I hope this may give an
insight.

