"""
![](https://deepsense.ai/wp-content/uploads/2019/02/Keras-or-PyTorch.png)

"""
"""
<h1 align="center"> INTRODUCTION </h1>
<hr>
Hello Kagglers! In this kernel I will talk about one of the most confusing questions that is, <b>Which Deep Learning Framework to use?</b> This questions usually comes to a beginner when he/she starts his/her journey in deep learning. Although it not only happens with beginners, sometimes others also get confused which to use. 

There are many deep learning framework options available like:-
* CNTK
* TensorFlow
* Keras
* DL4J
* PyTorch
* Lasagne
* Mxnet

And many more......

But the most famous frameworks are TensorFlow, Keras and PyTorch. Out of these 3, here I will be comparing PyTorch and Keras in greater depth and proper examples. So let's get started.

<font color='red'><h2>Please do an upvote if you find this kernel useful!</h2></font>
"""
"""
# Table of Contents
<ul style="list-style-type:square">
    <li><a href="#1">Basic Information</a></li>
    <li><a href="#2">Factors for Comparison</a></li>
    <li><a href="#4">Importing Libraries</a></li>
    <ul>
        <li><a href="#4.1">Importing Common Libraries</a></li>
        <li><a href="#4.2">Importing Keras Libraries</a></li>
        <li><a href="#4.3">Importing PyTorch Libraries</a></li>
    </ul>
    <li><a href="#5">Comparison Based on ANN</a></li>
     <ul>
        <li><a href="#5.1">Reading the Data</a></li>
        <li><a href="#5.2">Data Preparation</a></li>
        <li><a href="#5.3">ANN Model</a></li>
        <li><a href="#5.4">Optimizers</a></li>
        <li><a href="#5.5">Training</a></li>
    </ul>
    <li><a href="#6">Comparison Based on CNN</a></li>
     <ul>
        <li><a href="#6.1">Reading the Data</a></li>
        <li><a href="#6.2">Data Preparation</a></li>
        <li><a href="#6.3">CNN Model</a></li>
        <li><a href="#6.4">Optimizers</a></li>
        <li><a href="#6.5">Training</a></li>
        <li><a href="#6.6">Predicting on Test Set</a></li>
    </ul>
    <li><a href="#7">When to use which?</a></li>
    <li><a href="#8">Bonus Part(Digit Recognizer)</a></li>
    <li><a href="#9">Ending Notes</a></li>
</ul>
"""
"""
<a id='1'></a>
# Basic Information

Before getting deeper into the comparison, it is very important to know some general information.

<b><font size = "4.5em">Keras :-</font></b> Actually Keras is not a framework on it's own. It sits on top of TensorFlow, Theano or CNTK.
<br>
<b><font size = "4.5em">PyTorch :-</font></b> On the other hand, PyTorch is a Deep Learning Framework developed by Facebook's AI research group.
"""
"""
<a id='2'></a>
# Factors for Comparison
<b><h3>1. Level of API</h3></b>
Keras has a high level of API as it is capable on running on top of the other frameworks. So it is easier to use and less code are needed.<br>
PyTorch has a low level API and hence it becomes little difficult to use and more code is required for the similar task but it gives better control to the programmer.<br>
You will clearly see the difference in the examples below.

<b><h3>2. Speed</h3></b>
Keras has comparitively less speed and is much slower than PyTorch. PyTorch has a very high speed due to which it is generally suitable for large datasets.

<b><h3>3. Debugging</h3></b>
Keras has very less debugging capabilites but PyTorch has very high debugging capabilities. One reason for that is as in PyTorch you have to write each and every step so you know the code properly and can debug the error easily but this is not so in the case of Keras.

<b><h3>4. Ease of code</h3></b>
As already told Keras has a high level API so we can do many jobs in just a single line of code and is very easy to read and understand.
Whereas PyTorch consist many lines of code and it is comparitively difficult to understand.

<b><h3>5. Community</h3></b>
Keras has a small community support but the good news is you don't have to worry because there are lots of tutorials and solutions available in the net as many people start with the Keras only.
PyTorch on the other hand has great community support and active development is going on.

<b><h3>6. Popularity</h3></b>
Although both these frameworks are very popular but still if we compare then Keras is much more popular than PyTorch.
"""
"""
<img src="https://www.stickergiant.com/blog/wp-content/uploads/2017/10/Lets-Code-This-Home-Depot-Kiss-Cut-2017.jpg" style = "padding: 2% 0;width:50rem;height:25rem;border-radius:20%">
Don't worry if you don't get it because now we will dive deeper into the comparison and lets start coding to explain and verify each of the factors in detail.<br>
Here I will be comparing these two frameworks by creating one <b>Artificial Neural Network(ANN) Model</b> and one <b>Convolutional Neural Network(CNN) Model</b>. 
"""
"""
<a id='3'></a>
In this kernel I will be using data from Titanic: Machine Learning from Disaster competition and Digit Recognizer competition for comparsion but here I will not explain each and every thing which is required for this competiton like data pre processing steps, EDA, and the exact models, I will only explain those things more which needs for comparison. Also please don't consider these models as the final models for these competitions as these are just demo models for comparison and will not give high accuracy.<br>
But don't worry, I have created notebooks for these competitions which gives a high accuracy and all the steps are explained in great detail.
<div class="row">
    <div class="col-sm-6">
    <div class = "card">
      <div class = "card-body" style = "width: 20rem; ">
        <h5 class = "card-title"  style = "font-size: 1.2em;"align="center">Full Explanation of CNN with high accuracy model</h5>
          <img src="https://storage.googleapis.com/tfds-data/visualization/fig/mnist-3.0.1.png" class = "card_img-top" style = "padding: 2% 0;width:22rem;height:15rem;"  alt="...">
        <p class="card-text" style = "font-size: 1.0em;text-align: center "><b>Digit Recognizer: Detailed Explanation of CNN</b></p>
        <a href = "https://www.kaggle.com/utcarshagrawal/digit-recognizer-detailed-explanation-of-cnn" class = "btn btn-info btn-lg active"  role = "button" style = "color: white; margin: 0 15% 0 25%" data-toggle = "popover" title = "Click">Click here</a>
      </div>
    </div>
  </div>
    <div class="col-sm-6">
      <div class="card">
        <div class="card-body" style="width: 20rem;">
          <h5 class = "card-title"  style = "font-size: 1.2em; " align="center" > Non Deep Learning Based method on Titanic Dataset </h5>
            <img src = "https://miro.medium.com/max/650/1*mzhf9OccFn7DeVHI9dPiLQ.jpeg" class = "card_img-top" style = " padding: 2% 0;width:22rem;height:15rem;"  alt="...">
          <p class="card-text" style = "font-size: 1.0em;text-align: center "><b>Titanic:Spark ML Magic + EDA & Feature Engineering</b></p>
          <a href = "https://www.kaggle.com/utcarshagrawal/titanic-spark-ml-magic-eda-feature-engineering" class = "btn btn-info btn-lg active"  role = "button" style = "color: white; margin: 0 15% 0 25%" data-toggle = "popover" title = "Click">Click here</a>
        </div>
      </div>    
    </div>
   </div>    
    
"""
"""
### Now the first step is to import the required libraries. Here I have separated libraries for Keras, PyTorch and some common libraries. 
"""
"""
<a id = '4.1'></a> 
# Importing Common Libraries
"""
import numpy as np 
import pandas as pd 
import os
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelBinarizer
from sklearn.preprocessing import StandardScaler
import numbers
"""
<a id = '4.2'></a> 
## Importing Keras Libraries
"""
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D, BatchNormalization
from keras.optimizers import RMSprop, SGD
from keras.preprocessing.image import ImageDataGenerator
from keras.utils.np_utils import to_categorical
"""
<a id = '4.3'></a> 
## Importing Pytorch Libraries
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.autograd import Variable
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms
from torchvision.utils import make_grid
"""
<a id='5'></a>
# 1. COMPARISON BASED ON ANN
"""
"""
<a id='5.1'></a>
# Reading the data
The next step is to read the data. As I told earlier I will be creating two models for comparison. The first is ANN Model for which I will be using Titanic: Machine Learning from Disaster competition data. Generally people prefer a non deep learning based approach for this dataset but deep learning based approach also works well.<br> 
For non deep learning based approach of this competition [go here](#3).
"""
df1 = pd.read_csv("../input/titanic/train.csv")
df1.head()
val = ['Pclass', 'Sex', 'Embarked', 'SibSp', 'Parch']
plt.figure(figsize=(15,15))
plt.subplots_adjust(right=1.5)
for i in range(5):
    plt.subplot(2,3,i+1), sns.countplot(x=val[i], hue='Survived', data = df1)
    plt.legend(['Not Survived', 'Survived'], loc='upper center', prop={'size': 10})
    plt.title('Count of Survival in {} Feature'.format(val[i]), size=10, y=1.05)
"""
### For more EDA of this dataset [click here](https://www.kaggle.com/utcarshagrawal/titanic-spark-ml-magic-eda-feature-engineering#Exploratory-Data-Analysis).
"""
"""
<a id='5.2'></a>
# Data Preparation
Before working on the model data preprocessing is one of the most important step. But preparing the data hugely depend on which type of framework you are using to create a model. There might be few steps which could be exactly same but there could be few different steps also.<br>
As we cannot directly apply this data to predict whether the person survived or not, so we will do some data preparation but I will not do all the steps because I am not here interested in increasing the accuracy.<br> 
### If you want to see all the feature engineering steps with explanations [click here](https://www.kaggle.com/utcarshagrawal/titanic-spark-ml-magic-eda-feature-engineering#Feature-Engineering).
"""
# Sex Encoding
binar = LabelBinarizer().fit(df1.loc[:, "Sex"])
df1["Sex"] = binar.transform(df1["Sex"])
# Embarked Encoding
df1["Embarked"] = df1["Embarked"].fillna('S')
df_Embarked = pd.get_dummies(df1.Embarked)
df1 = pd.concat([df1, df_Embarked], axis=1)
#Family
df1['Family'] = df1['SibSp'] + df1['Parch'] + 1
df1['Alone'] = df1['Family'].apply(lambda x : 0 if x>1 else 1 )
#Age
df1['Age'] = df1['Age'].fillna(-0.5)
features = ["Pclass","Sex", "Age", "C", "Q", "S", "Alone"]

X_train = df1[features].values
Y_train = df1["Survived"].values
# Feature Scaling
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
"""
Here in this dataset all the pre processing steps are same for both Keras and PyTorch model. So we don't have anything to compare here but this doesn't always happen. In my next example, you will see the difference.
"""
"""
<a id='5.3'></a>
# ANN MODEL
So we have applied all the steps and now its time to create our model. Here you will observe the difference clearly. First I will make a Keras model and then a PyTorch model.
"""
"""
## ANN KERAS MODEL
"""
model1 = Sequential()
model1.add(Dense(100, input_dim=7, activation="relu"))
model1.add(Dropout(0.5))
model1.add(Dense(200, activation="relu"))
model1.add(Dropout(0.5))
model1.add(Dense(1, activation="sigmoid"))
"""
Here you can easily see how each and every step almost took just one line. This is the major advantage of Keras and if you know little bit about Keras models, then you will observe that it is not difficult to understand also. Now lets talk about PyTorch model.
"""
"""
## ANN PYTORCH MODEL
"""
class ANN(nn.Module):
    def __init__(self):
        super(ANN, self).__init__()
        self.features = nn.Sequential(
            nn.Linear(7,100),
            nn.ReLU(inplace=True),
            nn.Dropout(p=0.5),
            nn.Linear(100,200),
            nn.ReLU(inplace=True),
            nn.Dropout(p=0.5),
            nn.Linear(200,2),
        )
    def forward(self, x):
        x = self.features(x)
        
        return x
    
model2 = ANN()
"""
Here you can observe the difference clearly. Keras on one side is more easily readable and understandable than PyTorch. Both the models are exactly same (Dense layer in Keras is same as Linear layer in PyTorch, just names are different) rest all the parameters are same but syntax of both the models are hugely different. Although for beginners PyTorch initially might be difficult but once you start understanding then it will become your favourite framework. 
"""
"""
<a id='5.4'></a>
# Optimizers
"""
#For Keras Model

optimizer1 = SGD(lr = 0.01, momentum = 0.9)
# Compiling our model
model1.compile(optimizer = optimizer1, loss = 'binary_crossentropy', metrics = ['accuracy'])

#For PyTorch Model

# Define the optimier
optimizer2 = torch.optim.SGD(model2.parameters(), lr=0.01)
# Define our loss function
criterion = nn.CrossEntropyLoss()
"""
Here we are using SGD optimizer for both the models just the syntax is different. But in PyTorch we have to also define our loss function whereas in Keras we don't have to explicity define it but we write our loss function when we compile our Keras model.
"""
batch_size=64
epochs=15
"""
<a id='5.5'></a>
## So we have completed all the pre processes. Now its time to train our model
"""
"""
The major difference in the code comes here. Training a Keras model is just a one line step and it is very simple as all the things will be printed automatically like loss, accuracy etc. But in PyTorch it is a long method. The advantage of this is you will know your code properly and understand better what the system is doing in the backend. Also you have to give print command to whatever you want to print in your own way.
"""
# Keras Model
model1.fit(X_train, Y_train, batch_size = batch_size, epochs = epochs, verbose=2)
# PyTorch Model
batch_no = len(X_train) // batch_size
train_loss_plot, train_acc_plot = [], []
train_loss = 0
train_loss_min = np.Inf
for epoch in range(epochs):
    for i in range(batch_no):
        start = i*batch_size
        end = start+batch_size
        x_ten = Variable(torch.FloatTensor(X_train[start:end]))
        y_ten = Variable(torch.LongTensor(Y_train[start:end])) 
        
        # Prevent accumulation of gradients
        optimizer2.zero_grad()
        # Make predictions
        output = model2(x_ten)
        loss = criterion(output,y_ten)
        #backprop
        loss.backward()
        optimizer2.step()
        
        values, labels = torch.max(output, 1)
        num_right = np.sum(labels.data.numpy() == Y_train[start:end])
        train_loss += loss.item()*batch_size
        
    train_loss = train_loss / len(X_train)
    train_loss_plot.append(train_loss)
    train_acc_plot.append((num_right / len(Y_train[start:end])))
    if train_loss <= train_loss_min:
        print("Validation loss decreased ({:6f} ===> {:6f})".format(train_loss_min,train_loss))
        torch.save(model2.state_dict(), "model.pt")
        train_loss_min = train_loss

    print('')
    print("Epoch: {} \tTrain Loss: {} \tTrain Accuracy: {}".format(epoch+1, train_loss,num_right / len(Y_train[start:end]) ))
metrics=pd.DataFrame(model1.history.history)
plt.plot(metrics['loss'], label='Keras_loss')
plt.plot(train_loss_plot, label='PyTorch_loss')
plt.legend()
plt.show()
plt.plot(metrics['accuracy'], label='Keras_acc')
plt.plot(train_acc_plot, label='PyTorch_acc')
plt.legend()
plt.show()
"""
This is the difference of PyTorch and Keras. Ease of code is one of the factor for choosing Keras but on the other hand PyTorch is very flexible. Here please don't consider the accuracy level as these are not great models and both of these can achieve better accuracy by changing the hyperparameters. Now let us talk about comparison based on CNN where we will cover some left over points.
"""
"""
<a id='6'></a>
# 2. COMPARISON BASED ON CNN
"""
"""
<a id='6.1'></a>
# Reading the data
"""
"""
As I have already told I will be using Digit Recognizer data for comparison based on CNN, so first lets read the data. 
"""
df2 = pd.read_csv('../input/digit-recognizer/train.csv')
df3 = pd.read_csv('../input/digit-recognizer/test.csv')
"""
<a id='6.2'></a>
# Data Preparation
"""
"""
### If you want to know the details of each and every step in Keras Framework [go here](#3).
"""
"""
## Data Preparation for Keras Model
"""
Y_train = df2["label"]
X_train = df2.drop(labels = ["label"],axis = 1).values

X_train = X_train/255.0
X_test = df3/255.0

X_train = X_train.reshape(-1,28,28,1)
X_test = X_test.values.reshape(-1,28,28,1)

Y_train = to_categorical(Y_train, num_classes = 10)

X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train, test_size = 0.1, random_state=35)
"""
## Data Preparation for Pytorch Model
"""
targets_np = df2.label.values
features_np = df2.loc[:, df2.columns != 'label'].values/255
test_np = df3.values/255

features_np = features_np.reshape(-1,1,28,28)
test_np = test_np.reshape(-1,1,28,28)
fake_labels = np.zeros(test_np.shape)
fake_labels = torch.from_numpy(fake_labels)

features_train, features_test, target_train, target_test = train_test_split(features_np, targets_np, test_size=0.1, random_state=35)

featuresTrain = torch.from_numpy(features_train)
targetsTrain = torch.from_numpy(target_train)

featuresTest = torch.from_numpy(features_test)
targetsTest = torch.from_numpy(target_test)

test_data = torch.from_numpy(test_np)

train = torch.utils.data.TensorDataset(featuresTrain, targetsTrain)
test = torch.utils.data.TensorDataset(featuresTest, targetsTest)
submission_data = torch.utils.data.TensorDataset(test_data, fake_labels)

train_loader = torch.utils.data.DataLoader(train, batch_size = batch_size, shuffle = True)
test_loader = torch.utils.data.DataLoader(test, batch_size = batch_size, shuffle = True)
submission_loader = torch.utils.data.DataLoader(submission_data, batch_size = batch_size, shuffle = False)
"""
Previously, you saw that the preprocessing steps was same for both the frameworks. But here this is not so. Initially, normalizing, reshaping and splitting the data are same for both but afterthat we have huge difference. In PyTorch we can't apply the numpy data directly but first we have to convert it in tensor then we make a tensor dataset and then combines a dataset and a sampler, and provides single- or multi-process iterators over the dataset.   
"""
"""
<a id='6.3'></a>
# **CNN MODEL COMPARISON**
As you saw the difference in ANN Model and now you will clearly see the difference in the CNN Model also.
"""
"""
## KERAS MODEL
### For detailed explanation of each layer and to know exactly what each layer does in the backend and also to know the perfect model to get high score [go here](#3).
"""
model3 = Sequential()

model3.add(Conv2D(filters=32, kernel_size=(5,5), padding='Same', activation='relu', input_shape=(28,28,1)))
model3.add(BatchNormalization())
model3.add(Conv2D(filters=32, kernel_size=(5,5), padding='Same', activation='relu'))
model3.add(BatchNormalization())
model3.add(MaxPool2D(pool_size=(2,2)))
model3.add(Dropout(0.25))

model3.add(Flatten())
model3.add(Dense(256, activation='relu'))
model3.add(BatchNormalization())
model3.add(Dropout(0.5))
model3.add(Dense(10, activation='softmax'))
"""
## PYTORCH MODEL
"""
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 32, kernel_size=(5,5), stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(inplace=True),
            nn.Conv2d(32, 32, kernel_size=(5,5), stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
          
        self.classifier = nn.Sequential(
            nn.Dropout(p = 0.5),
            nn.Linear(32 * 12 * 12, 512),
            nn.BatchNorm1d(512),
            nn.ReLU(inplace=True),
            nn.Dropout(p = 0.5),
            nn.Linear(512, 10),
        )
    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        
        return x

model4 = CNN()
"""
<a id='6.4'></a>
# Optimizer
"""
#For Keras Model

optimizer3 = RMSprop(lr=0.001, rho=0.9, epsilon=1e-08, decay=0.0)
# Compiling our model
model3.compile(optimizer = optimizer3 , loss = "categorical_crossentropy", metrics=["accuracy"])

#For PyTorch Model

# Define the optimier
optimizer4 = optim.RMSprop(model4.parameters(), lr=0.001)
# Define our loss function
criterion = nn.CrossEntropyLoss()
"""
<a id='6.5'></a>
## So we have completed all the pre processes. Now its time to train our model
"""
"""
As I previously told how this step has a huge difference. But here you will observe one difference that shows the flexibility of PyTorch and that is "cuda()". If we want a particular computation to be performed on the GPU, we can instruct PyTorch to do so by calling cuda() on our data structures (tensors). So this way we can apply our model on large datasets and the model can be trained on GPU very fastly.
"""
model3.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, validation_data = (X_val, Y_val))
if torch.cuda.is_available():
    model4 = model4.cuda()
    criterion = criterion.cuda()
    
model4.train()

steps = 0
print_every = 50
train_losses, test_losses = [], []

for e in range(epochs):
    running_loss = 0
    for images, labels in train_loader:
        steps += 1
        if torch.cuda.is_available():
            images = images.cuda()
            labels = labels.cuda()
        # Prevent accumulation of gradients
        optimizer4.zero_grad()
        # Make predictions
        log_ps = model4(images.float())
        loss = criterion(log_ps, labels.long())
        #backprop
        loss.backward()
        optimizer4.step()
        
        running_loss += loss.item()
        if steps % print_every == 0:
            test_loss = 0
            accuracy = 0

            # Turn off gradients for validation
            with torch.no_grad():
                model4.eval()
                for images, labels in test_loader:
                    if torch.cuda.is_available():
                        images = images.cuda()
                        labels = labels.cuda()
                    log_ps = model4(images.float())
                    test_loss += criterion(log_ps, labels.long())

                    ps = torch.exp(log_ps)
                    # Get our top predictions
                    top_p, top_class = ps.topk(1, dim=1)
                    equals = top_class == labels.view(*top_class.shape)
                    accuracy += torch.mean(equals.type(torch.FloatTensor))

    train_losses.append(running_loss/len(train_loader))
    test_losses.append(test_loss/len(test_loader))

    print("Epoch: {}/{}.. ".format(e+1, epochs),
          "Training Loss: {:.4f}.. ".format(train_losses[-1]),
          "Test Loss: {:.4f}.. ".format(test_losses[-1]),
          "Test Accuracy: {:.4f}".format(accuracy/len(test_loader)))
metrics2=pd.DataFrame(model3.history.history)
plt.plot(metrics2['loss'], label='Keras_loss')
plt.plot(train_losses, label='PyTorch_loss')
plt.legend()
plt.show()
plt.plot(metrics2['val_loss'], label='Keras_val_loss')
plt.plot(test_losses, label='PyTorch_val_loss')
plt.legend()
plt.show()
"""
<a id='6.6'></a>
# Predicting on test set
"""
results = model3.predict(X_test)
def make_prediction(data):
    images, labels = next(iter(data))
    if torch.cuda.is_available():
        images = images.cuda()
        labels = labels.cuda()

    img = images
    # Turn off gradients to speed up this part
    with torch.no_grad():
        logps = model4(images.float())

    ps = torch.exp(logps)
make_prediction(submission_loader)
"""
Similar to the previous model, here also you can observe the difference of ease of code and simplicity. But in this case you can also observe the flexibility and the speed of PyTorch model. Again here I am saying please don't compare the accuracy as these are not perfect models and we can achieve very high accuracy using any of the frameworks. 
"""
"""
<a id='7'></a>
# When to use which?
Now the last thing which is left is when to use which framework.<br>
Although after comparing so many things we still can't say which is better. Each one have its pros and cons and it ultimately depends on your need and expectations.Some suitable situations to use:-
<ul>
    <li><h2>Keras:-</h2></li>
    <ol>
        <li>Quick Prototype</li>
        <li>Small dataset</li>
        <li>Best for begineers</li>
    </ol>
    <li><h2>PyTorch:-</h2></li>
    <ol>
        <li>Flexibility</li>
        <li>Large dataset</li>
        <li>Fast Speed</li>
        <li>Debugging Capabilities</li>
    </ol>
<ul>
    
        
"""
"""
<a id='8'></a>
# Bonus Part(Digit Recognizer)
To achieve a very high score of this Digit Recognizer competition, if you want to use Keras framework then [go here](#3). For PyTorch Model I have not created any notebook so I have written the code here:-
"""
class CNN_2(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(inplace=True),
            nn.Conv2d(32, 32, kernel_size=3, stride=1, padding=1),
            nn.BatchNorm2d(32),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
            nn.Conv2d(32, 64, kernel_size=3, padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            nn.Conv2d(64, 64, kernel_size=3, padding=1),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2)
        )
          
        self.classifier = nn.Sequential(
            nn.Dropout(p = 0.5),
            nn.Linear(64 * 7 * 7, 512),
            nn.BatchNorm1d(512),
            nn.ReLU(inplace=True),
            nn.Dropout(p = 0.5),
            nn.Linear(512, 512),
            nn.BatchNorm1d(512),
            nn.ReLU(inplace=True),
            nn.Dropout(p = 0.5),
            nn.Linear(512, 10),
        )
    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        
        return x
"""
<a id='9'></a>
# Ending Notes
So here we come to an end to a long comparison between the two most famous deep learning frameworks. I hope now you will be able to select the correct framework according to your need. One more thing I want to say is that I am not an expert in PyTorch so if I have done any mistake in PyTorch code or anywhere else also please comment below so that I can correct it. <br>

I hope that this notebook will not only help you for comparison but also it will help you in the coding part for both the Keras and PyTorch users. <br>

As I already told for much detailed information for non deep learning based method on Titanic dataset and for CNN explanation using Keras you can refer my other two notebooks, [go here](#3).<br>

Also if you have any doubt you can comment below and I will definately try to help you.<br>

Also, please join this discussion and tell your views which framework do you prefer and why:- <a href = "https://www.kaggle.com/getting-started/175487" class = "btn btn-info btn-lg active"  role = "button" style = "color: white;" data-toggle = "popover" title = "Click">Click here</a>

<font color='red'><h3>Thanks a lot for having a look at this notebook.<br> I would like to get an appreciation from you with an upvote. Please upvote and give review if you liked the kernel.</h3></font>
"""