from sklearn.datasets import load_iris
iris = load_iris()
X = iris.data
y = iris.target

"""
Explanation: Dimensionality Reduction and Visualization
Dimensionality reduction is the task of deriving a set of new
artificial features that is smaller than the original feature
set while retaining most of the variance of the original data.
Here we'll use a common but powerful dimensionality reduction
technique called Principal Component Analysis (PCA).
We'll perform PCA on the iris dataset that we saw before:
End of explanation
"""


from sklearn.decomposition import PCA
pca = PCA(n_components=2, whiten=True).fit(X)

"""
Explanation: PCA is performed using linear combinations of the original features
using a truncated Singular Value Decomposition of the matrix X so
as to project the data onto a base of the top singular vectors.
If the number of retained components is 2 or 3, PCA can be used
to visualize the dataset:
End of explanation
"""


pca.components_                           

pca.explained_variance_ratio_

pca.explained_variance_ratio_.sum()

"""
Explanation: Once fitted, the pca model exposes the singular vectors in the components_ attribute:
End of explanation
"""


X_pca = pca.transform(X)

"""
Explanation: Let us project the iris dataset along those first two dimensions:
End of explanation
"""


import numpy as np
np.round(X_pca.mean(axis=0), decimals=5)

np.round(X_pca.std(axis=0), decimals=5)

"""
Explanation: The dataset has been “normalized”, which means that the data
is now centered on both components with unit variance:
End of explanation
"""


np.round(np.corrcoef(X_pca.T), decimals=5)

"""
Explanation: Furthermore the samples components do no longer carry any linear correlation:
End of explanation
"""


%pylab inline

"""
Explanation: We can visualize the projection using pylab, but first
let's make sure our ipython notebook is in pylab inline mode
End of explanation
"""


import pylab as pl
from itertools import cycle

def plot_PCA_2D(data, target, target_names):
    colors = cycle('rgbcmykw')
    target_ids = range(len(target_names))
    pl.figure()
    for i, c, label in zip(target_ids, colors, target_names):
        pl.scatter(data[target == i, 0], data[target == i, 1],
                   c=c, label=label)
    pl.legend()

"""
Explanation: Now we can visualize the results using the following utility function:
End of explanation
"""


plot_PCA_2D(X_pca, iris.target, iris.target_names)

"""
Explanation: Now calling this function for our data, we see the plot:
End of explanation
"""


from sklearn.decomposition import RandomizedPCA
#apply randomized PCA to the iris data as above, and plot the result.

"""
Explanation: Note that this projection was determined without any information about the
labels (represented by the colors): this is the sense in which the learning
is unsupervised.  Nevertheless, we see that the projection gives us insight
into the distribution of the different flowers in parameter space: notably,
iris setosa is much more distinct than the other two species.
Note also that the default implementation of PCA computes the SVD of the full
data matrix, which is not scalable when both n_samples and
n_features are big (more that a few thousands).
If you are interested in a number of components that is much
smaller than both n_samples and n_features, consider using
:class:sklearn.decomposition.RandomizedPCA instead.
Exercise:
Repeat the above dimensionality reduction with
sklearn.decomposition.RandomizedPCA.
You can re-use the plot_PCA_2D function from above.
Are the results similar to those from standard PCA?
End of explanation
"""


