title: python - linear regression - image
body: <p>I am trying to wrap my head around on machine learning within python. i have been working with the following example (<a href="http://scikit-learn.org/stable/auto_examples/plot_multioutput_face_completion.html#example-plot-multioutput-face-completion-py" rel="nofollow">http://scikit-learn.org/stable/auto_examples/plot_multioutput_face_completion.html#example-plot-multioutput-face-completion-py</a>) with the code example per below.</p>
  
  <p>i would love to test / validat my understanding on the inner working of the linear regression. The aim is to predict the lower missing half of a picture by looking at the known upper half of a picture. There were originally 300 64*64 images (4096 pixels). The independent variable X is a 300*2048 matrix (300 pictures, 2048 pixels (upper half of those pictures). The dependent variable is also a 300*2048 matrix (lower half of the pictures). It seems that the coefficient matrix is a 2048*2048 matrix. Am i right in my understanding that:</p>
  
  <ul>
  <li><p>that the prediction for a single pixel of y (e.g. picture 1, most uppper left pixel) is performed by the multiplicatoin of all 2048 pixels in the upper half of picture 1 times the set of regression coefficients - and that thus each missing pixel in the lower half is estimated by taking into account all 2048 pixels of that specific image? </p></li>
  <li><p>that the regression coefficients are pixel dependent (each y pixel has different set of 2048 regression coefficents), and that these coefficients are estimated by finding the OLS fit for that specific pixel location across the identical pixel location across the 300 images available? </p></li>
  </ul>
  
  <p>I might very well be confused by the matrices - so please correct me if i am wrong. many thanks. W</p>
  
  <pre><code>print(__doc__)
  
  import numpy as np
  import matplotlib.pyplot as plt
  
  from sklearn.datasets import fetch_olivetti_faces
  from sklearn.utils.validation import check_random_state
  
  from sklearn.ensemble import ExtraTreesRegressor
  from sklearn.neighbors import KNeighborsRegressor
  from sklearn.linear_model import LinearRegression
  from sklearn.linear_model import RidgeCV
  
  # Load the faces datasets
  data = fetch_olivetti_faces()
  targets = data.target
  
  data = data.images.reshape((len(data.images), -1))
  train = data[targets &lt; 30]
  test = data[targets &gt;= 30]  # Test on independent people
  
  # Test on a subset of people
  n_faces = 5
  rng = check_random_state(4)
  face_ids = rng.randint(test.shape[0], size=(n_faces, ))
  test = test[face_ids, :]
  
  n_pixels = data.shape[1]
  X_train = train[:, :np.ceil(0.5 * n_pixels)]  # Upper half of the faces
  y_train = train[:, np.floor(0.5 * n_pixels):]  # Lower half of the faces
  X_test = test[:, :np.ceil(0.5 * n_pixels)]
  y_test = test[:, np.floor(0.5 * n_pixels):]
  
  # Fit estimators
  ESTIMATORS = {
      "Extra trees": ExtraTreesRegressor(n_estimators=10, max_features=32,
                                         random_state=0),
      "K-nn": KNeighborsRegressor(),
      "Linear regression": LinearRegression(),
      "Ridge": RidgeCV(),
  }
  
  y_test_predict = dict()
  for name, estimator in ESTIMATORS.items():
      estimator.fit(X_train, y_train)
      y_test_predict[name] = estimator.predict(X_test)
  </code></pre>