from sklearn.ensemble import RandomForestClassifier
import numpy as np
import pandas as pd

# create the training & test sets, skipping the header row with [1:]
dataset = pd.read_csv("../data/train.csv")
target = dataset[[0]].values.ravel()
train = dataset.iloc[:,1:].values
test = pd.read_csv("../data/test.csv").values

# create and train the random forest
# multi-core CPUs can use: rf = RandomForestClassifier(n_estimators=100, n_jobs=2)
#rf = RandomForestClassifier(n_estimators=100)
#rf.fit(train, target)
#pred = rf.predict(test)

#np.savetxt('submission_rand_forest.csv', np.c_[range(1,len(test)+1),pred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')

"""
Explanation: Digit Recognizer using Convolutional Neural Nets
Starting with a brief benchmark below, which was found in the forums we can use this to solve the digit recognizer problem with 0.96 accuracy score in less than a minute of learning. However, a neural net would have better accuracy.
End of explanation
"""


dataset.head()

"""
Explanation: Lets take a look at the data.
train.csv has 42k rows. THe first column is the digit label and the rest 784 columns are 8 bit 256 pixel color values ranging from 0 to 255.
The test.csv contains only the pixel color values which we need to classify as digits. lets see what we're working with here
End of explanation
"""


# convert to array, specify data type, and reshape
target = target.astype(np.uint8)
train = np.array(train).reshape(-1,1, 28, 28) # -1 means the value is inferred from the original array 42k in this case
test = np.array(test).reshape((-1, 1, 28, 28)).astype(np.uint8)

"""
Explanation: Convert each data variable from a python list into a numpy array. For the target array I specify the integer data type. The train set has 42k rows and 784 columns, so its shape is (42k,784). Each row is a 28 by 28 pixel picture. I will reshape the train set to have (42k,1) shape, i.e. each row will contain a 28 by 28 matrix of pixel color values. Same for the test set.
End of explanation
"""


import matplotlib.pyplot as plt
import matplotlib.cm as cm

figs, axes = plt.subplots(4, 4, figsize=(6, 6))
for i in range(4):
    for j in range(4):
        axes[i, j].imshow(-train[i + 4 * j].reshape(28, 28), cmap='gray', interpolation='none')
        axes[i, j].set_xticks([])
        axes[i, j].set_yticks([])
        axes[i, j].set_title("Label: {}".format(target[i + 4 * j]))
        axes[i, j].axis('off')
        
# plt.imshow(train[1700][0], cmap=cm.binary) # draw the picture

"""
Explanation: Now, we can actually plot those pixel color values and see what a sample picture of a digit looks like. Below is the picture of a digit in the 1700th row:
End of explanation
"""


import lasagne
from lasagne import layers
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet
from nolearn.lasagne import visualize

net1 = NeuralNet(
        layers = [
            # layer dealing with the input data
            (InputLayer, {'shape': (None, X.shape[1], X.shape[2], X.shape[3])}),

            # first stage of our convolutional layers
            (Conv2DLayer, {'num_filters': 96, 'filter_size': 5}),
            (Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
            (Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
            (Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
            (Conv2DLayer, {'num_filters': 96, 'filter_size': 3}),
            (MaxPool2DLayer, {'pool_size': 2}),

            # second stage of our convolutional layers
            (Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
            (Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
            (Conv2DLayer, {'num_filters': 128, 'filter_size': 3}),
            (MaxPool2DLayer, {'pool_size': 2}),

            # two dense layers with dropout
            (DenseLayer, {'num_units': 64}),
            (DropoutLayer, {}),
            (DenseLayer, {'num_units': 64}),

            # the output layer
            (DenseLayer, {'num_units': 10, 'nonlinearity': softmax}),
        ],
        # layer parameters:
        input_shape=(None,1,28,28),
        hidden_num_units=1000, # number of units in 'hidden' layer
        output_nonlinearity=lasagne.nonlinearities.softmax,
        output_num_units=10,  # 10 target values for the digits 0, 1, 2, ..., 9

        # optimization method:
        update=nesterov_momentum,
        update_learning_rate=0.0001,
        update_momentum=0.9,
        max_epochs=15,
        verbose=1,
        )

# Train the network
net1.fit(train, target)

"""
Explanation: Train, Predict and Save
Below is a simple NN set up. Supposedly, it should give >0.9 accuracy score. I had trouble with figuring out the training part in that the accuracy I was getting would not change during the training process. All I had to do was to decrease the learning rate from 0.01 to 0.0001, nota bene.
End of explanation
"""


