import warnings
warnings.filterwarnings("ignore")

from sklearn import datasets
import matplotlib.pyplot as plt
import numpy as np

plt.style.use('seaborn-poster')
%matplotlib inline

iris = datasets.load_iris()

print(iris.feature_names)
# only print the first 10 samples
print(iris.data[:10])
print('We have %d data samples with %d features'%(iris.data.shape[0], iris.data.shape[1]))

"""
Explanation: Quick concept of Machine Learning
Most machine learning algorithms share the same pipeline structure as the one shown below:
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_0.png" width="600"/> 
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_1.png" width="600"/> 
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_2.png" width="600"/> 
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_3.png" width="600"/> 
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_4.png" width="600"/> 
<img style="border:1px solid black;" src="https://raw.githubusercontent.com/qingkaikong/20171206_ML_basics_THW/master/images/image_5.png" width="600"/> 
Scikit-learn Basics
This session will cover the basics of Scikit-Learn, a popular package containing a collection of tools for machine learning written in Python. See more at http://scikit-learn.org.
Loading an example dataset
Let's start by loading some pre-existing datasets in the scikit-learn, which comes with a few standard datasets. For example, the iris and digits datasets for classification and the boston house prices dataset for regression. Using these existing datasets, we can easily test the algorithms that we are interested in. 
A dataset is a dictionary-like object that holds all the data and some metadata about the data. This data is stored in the .data member, which is a n_samples, n_features array. In the case of supervised problem, one or more response variables are stored in the .target member. More details on the different datasets can be found in the dedicated section.
Load iris
The iris dataset consists of 50 samples from each of three species of Iris (Iris setosa, Iris virginica and Iris versicolor). Four features were measured from each sample: the length and the width of the sepals and petals, in centimetres.
|   |  |  |
|:---:|:---:|:---:|
| Iris Setosa| Iris Virginica| Iris Versicolor|
End of explanation
"""


print(iris.target_names)
print(set(iris.target))

"""
Explanation: The data is always a 2D array, shape (n_samples, n_features), although the original data may have had a different shape. The following prints out the target names and the representatoin of the target using 0, 1, 2. Each of them represent a class.
End of explanation
"""


digits = datasets.load_digits()
print('We have %d samples'%len(digits.target))

print(digits.data)
print('The targets are:')
print(digits.target_names)

"""
Explanation: Load Digits
This dataset is made up of 1797 8x8 images. Each image, like the ones shown below, is of a hand-written digit. In order to utilize an 8x8 figure like this, we’d have to first transform it into a feature vector with length 64.
End of explanation
"""


print(digits.data.shape)

## plot the first 64 samples, and get a sense of the data
fig = plt.figure(figsize = (8,8))
fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)
for i in range(64):
    ax = fig.add_subplot(8, 8, i+1, xticks=[], yticks=[])
    ax.imshow(digits.images[i],cmap=plt.cm.binary,interpolation='nearest')
    ax.text(0, 7, str(digits.target[i]))

"""
Explanation: In the digits, each original sample is an image of shape (8, 8) and it is flattened into a 64 dimension vector here.
End of explanation
"""


boston = datasets.load_boston()
print(boston.DESCR)

boston.feature_names

# let's just plot the average number of rooms per dwelling with the price
plt.figure(figsize = (10,8))
plt.plot(boston.data[:,5], boston.target, 'o')
plt.xlabel('Number of rooms')
plt.ylabel('Price (thousands)')

"""
Explanation: Load Boston Housing Data
The Boston housing dataset reports the median value of owner-occupied homes in various places in the Boston area, together with several variables which might help to explain the variation in median value, such as Crime (CRIM), areas of non-retail business in the town (INDUS), the age of people who own the house (AGE), and there are many other attributes that you can find the details here.
End of explanation
"""


from sklearn.linear_model import LinearRegression

"""
Explanation: The Scikit-learn Estimator Object
Every algorithm is exposed in scikit-learn via an ''Estimator'' object. For instance a linear regression is implemented as so:
End of explanation
"""


# you can check the parameters as
LinearRegression?

# let's change one parameter
model = LinearRegression(normalize=True)
print(model.normalize)

print(model)

"""
Explanation: Estimator parameters: All the parameters of an estimator can be set when it is instantiated, and have suitable default values:
End of explanation
"""


x = np.arange(10)
y = 2 * x + 1

plt.figure(figsize = (10,8))
plt.plot(x,y,'o')

# generate noise between -1 to 1
# this seed is just to make sure your results are the same as mine
np.random.seed(42)
noise = 2 * np.random.rand(10) - 1

# add noise to the data
y_noise = y + noise

plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')

# The input data for sklearn is 2D: (samples == 10 x features == 1)
X = x[:, np.newaxis]
print(X)
print(y_noise)

# model fitting is via the fit function
model.fit(X, y_noise)

# underscore at the end indicates a fit parameter
print(model.coef_)
print(model.intercept_)

# then we can use the fitted model to predict new data
predicted = model.predict(X)

plt.figure(figsize = (10,8))
plt.plot(x,y_noise,'o')
plt.plot(x,predicted, label = 'Prediction')
plt.legend()

"""
Explanation: Estimated Model parameters: When data is fit with an estimator, parameters are estimated from the data at hand. All the estimated parameters are attributes of the estimator object ending by an underscore:
Simple regression problem
Let's fit a simple linear regression model to see what is the sklearn API looks like. We use a very simple datasets with 10 samples with added in noise.
End of explanation
"""


%load ../solutions/solution_01.py

"""
Explanation: Exercise
In the next section, we will use support vector machine (SVM) to do a classification problem, have a look of the sklearn API, and find out which class we will use for a classification problem using SVM. (hint: we will use the one with C support).
End of explanation
"""


