title: PCA using princomp in MATLAB (for face recognition)
body: <p>I'm trying to do dimensionality reduction using <a href="http://en.wikipedia.org/wiki/MATLAB" rel="nofollow">MATLAB's</a> <code>princomp</code>, but I'm not sure I'm doing it right.</p>
  
  <p>Here is the my code just for testing, but I'm not sure that I'm doing projection right:</p>
  
  <pre><code>A = rand(4,3)
  AMean = mean(A)
  [n m] = size(A)
  Ac = (A - repmat(AMean,[n 1]))
  pc = princomp(A)
  k = 2; %Number of first principal components
  A_pca = Ac * pc(1:k,:)'  %Not sure I'm doing projection right
  reconstructedA = A_pca * pc(1:k,:)
  error = reconstructedA- Ac
  </code></pre>
  
  <p>And my code for face recognition using ORL dataset:</p>
  
  <pre><code>%load orl_data 400x768 double matrix (400 images 768 features)
  %make labels
  orl_label = [];
  for i = 1:40
      orl_label = [orl_label;ones(10,1)*i];
  end
  
  n = size(orl_data,1);
  k = randperm(n);
  s = round(0.25*n); %Take 25% for train
  
  %Raw pixels
  %Split on test and train sets
  data_tr = orl_data(k(1:s),:);
  label_tr = orl_label(k(1:s),:);
  data_te = orl_data(k(s+1:end),:);
  label_te = orl_label(k(s+1:end),:);
  
  tic
  [nn_ind, estimated_label] = EuclDistClassifier(data_tr,label_tr,data_te);
  toc
  
  rate = sum(estimated_label == label_te)/size(label_te,1)
  
  %Using PCA
  tic
  pc = princomp(data_tr);
  toc
  
  mean_face = mean(data_tr);
  pc_n = 100;
  f_pc = pc(1:pc_n,:)';
  data_pca_tr = (data_tr - repmat(mean_face, [s,1])) * f_pc;
  data_pca_te = (data_te - repmat(mean_face, [n-s,1])) * f_pc;
  
  tic
  [nn_ind, estimated_label] = EuclDistClassifier(data_pca_tr,label_tr,data_pca_te);
  toc
  
  rate = sum(estimated_label == label_te)/size(label_te,1)
  </code></pre>
  
  <p>If I choose enough principal components it gives me equal recognition rates. If I use a small number of <a href="https://en.wikipedia.org/wiki/Principal_component_analysis" rel="nofollow">principal components</a> (PCA) then the rate using PCA is poorer.</p>
  
  <p>Here are some questions:</p>
  
  <ol>
  <li>Is <code>princomp</code> function the best way to calculate first <strong>k</strong> principal components using MATLAB?</li>
  <li>Using PCA projected features vs raw features don't give extra accuracy, but only smaller features vector size? (faster to compare feature vectors).</li>
  <li>How to automatically choose min <strong>k</strong> (number of principal components) that give the same accuracy vs raw feature vector?</li>
  <li>What if I have very big set of samples can I use only subset of them with comparable accuracy? Or can I compute PCA on some set and later "add" some other set (I don't want to recompute pca for set1+set2, but somehow iteratively add information from set2 to existing PCA from set1)?</li>
  </ol>
  
  <p>I also tried the GPU version simply using <code>gpuArray</code>:</p>
  
  <pre><code>%Test using GPU
  tic
  A_cpu = rand(30000,32*24);
  A = gpuArray(A_cpu);
  AMean = mean(A);
  [n m] = size(A)
  pc = princomp(A);
  k = 100;
  A_pca = (A - repmat(AMean,[n 1])) * pc(1:k,:)';
  A_pca_cpu = gather(A_pca);
  toc
  clear;
  
  tic
  A = rand(30000,32*24);
  AMean = mean(A);
  [n m] = size(A)
  pc = princomp(A);
  k = 100;
  A_pca = (A - repmat(AMean,[n 1])) * pc(1:k,:)';
  toc
  clear;
  </code></pre>
  
  <p>It is working faster, but it's not suitable for big matrices. Maybe I'm wrong?</p>
  
  <p>If I use a big matrix, it gives me:</p>
  
  <blockquote>
    <p>Error using gpuArray  Out of memory on device.</p>
  </blockquote>