# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/yusufdalva/TensorFlow_Practice/blob/master/fundamentals/Basic_ConvNets.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="JUYx6b473Wul" colab_type="text"
# # Basic Convolutional Neural Network (CNN) Example
# This notebook shows the use of Convlutional layers and Pooling Layers in a Neural Network implemented with Keras, using Fashion MNIST dataset. 

# + [markdown] id="VKACeHtT4QuP" colab_type="text"
# ### Necessary imports
# Tensorflow, Matplotlib and numpy are the necessary imports for this notebook.

# + id="W4oXSrZB230p" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 34} outputId="8ebfff2b-3a3a-40cf-ebe7-2e4eae94e40e"
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
# Checking Tensorflow version
print('Curreent version of Tensorflow: ', tf.__version__)

# + [markdown] id="pmtP0j3-6Msk" colab_type="text"
# ## Loading the dataset
# At this step the Fashion MNIST dataset will be loaded for use.

# + id="wPPvLEzE6GK4" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 204} outputId="4d1c7671-1708-4bf4-bf7e-e293a106b672"
fashion_mnist = tf.keras.datasets.fashion_mnist
(train_samples, train_labels), (test_samples, test_labels) = fashion_mnist.load_data()
assert train_samples.shape[0] == train_labels.shape[0] # Check for training samples the one-to-one relationship between samples and labels
print('Number of training samples: ', train_samples.shape[0])
assert test_samples.shape[0] == test_labels.shape[0] # Check for test samples the one-to-one relationship between samples and labels
print('Number of test samples: ', test_samples.shape[0])
assert train_samples.shape[1:] == test_samples.shape[1:] # Check the compatibility of data samples for test set and training set
print('Shape of a data sample: ', train_samples.shape[1:])
# Normalizing the data
train_samples = train_samples / 255.0
test_samples = test_samples / 255.0

# + [markdown] id="oRqRjYLR8O2f" colab_type="text"
# ## Base model
# To observe the effect of convolutional layers, a base model that contains 2 hidden layers is used. Softmax activation is used in the output layer.

# + id="Irp2vDSx70FR" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 697} outputId="e48e5134-82e7-4ebf-9bf8-e30c2a8ba5cd"
# Constructing the graph for the base model
epoch_count = 20
dnn_in = tf.keras.Input(shape = train_samples.shape[1:])
X = tf.keras.layers.Flatten()(dnn_in)
X = tf.keras.layers.Dense(units = 128, activation = 'relu')(X)
X = tf.keras.layers.Dense(units = 64, activation = 'relu')(X)
dnn_out = tf.keras.layers.Dense(units = 10, activation = 'softmax')(X)
dnn_model = tf.keras.Model(inputs = dnn_in, outputs = dnn_out)
# Compiling the model
dnn_model.compile(loss = 'sparse_categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
# Fittibg the model
base_history = dnn_model.fit(x = train_samples, y = train_labels, epochs = epoch_count)

# + [markdown] id="F1mMEoFC2Vkt" colab_type="text"
# ### Evaluating the base model
# In order to compare with the CNN later, the base model is evaluated with the accuracy metric. The training statistics are also visualized here.

# + id="JQR7GWVq_A99" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 68} outputId="918a2822-3b3c-43e2-a09f-83e4ad761b5d"
base_test_metrics = dnn_model.evaluate(x = test_samples, y = test_labels)
print('Loss in test set: ' + '{:.4f}'.format(base_test_metrics[0]))
print('Accuracy in test set: ' + '{:.2f}'.format(base_test_metrics[1]))

# + id="DDvsg3gq32Fn" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 370} outputId="71c243a3-2013-4c5c-cf9b-8420d2611c4f"
fig, ax = plt.subplots(1,2, figsize = (15,5))
ax[0].plot(range(1, epoch_count + 1), base_history.history['loss'])
ax[0].set_xlabel('epochs')
ax[0].set_ylabel('Training Loss')
ax[0].set_title('Change in Training Loss')
ax[1].plot(range(1, epoch_count + 1), base_history.history['accuracy'])
ax[1].set_xlabel('epochs')
ax[1].set_ylabel('Training accuracy')
ax[1].set_title('Change in Training Accuracy')
fig.suptitle('Change in loss and accuracy in base model (DNN with no convolutions)', fontsize=16)
fig.show()

# + [markdown] id="lIcVBn0-5vSM" colab_type="text"
# ## The Convolutional Model (CNN)
# As a comparison with the other model, a neural network that has the sequence </br>
# CONV->MAX POOL->CONV->FLATTEN->DENSE->DENSE->DENSE(softmax) will be implemented. The following diagram shows the structure of the network: </br>
# ![image](https://drive.google.com/uc?export=view&id=1sy8b0e55S-xejK2e0hU6PEgsf47Kf6Gd)
#
#

# + id="AtD2xSj28DSZ" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 425} outputId="596e9d7d-435f-4bd3-9e4e-9eda647de474"
train_samples = train_samples.reshape(train_samples.shape[0], 28, 28, 1)
test_samples = test_samples.reshape(test_samples.shape[0], 28, 28, 1)
cnn_in = tf.keras.layers.Input(shape = train_samples.shape[1:])
X = tf.keras.layers.Conv2D(filters = 32, kernel_size = 3)(cnn_in) # Kernel size = 3 is equivalent to 3x3 convolution window
X = tf.keras.layers.MaxPool2D(pool_size = 2, strides = 2)(X) # Pool size = 2 is equivalent to 2x2 pooling window
X = tf.keras.layers.Conv2D(filters = 16, kernel_size = 3)(X)
X = tf.keras.layers.Flatten()(X)
X = tf.keras.layers.Dense(units = 128, activation = 'relu')(X)
X = tf.keras.layers.Dense(units = 64, activation = 'relu')(X)
cnn_out = tf.keras.layers.Dense(units = 10, activation = 'softmax')(X)
# Constructing the model graph
cnn_model = tf.keras.Model(inputs = cnn_in, outputs = cnn_out)
# Summary of the model and its parameters
cnn_model.summary()

# + [markdown] id="UC9g7bsFAkb6" colab_type="text"
# Here note that after a convolutional layer the dimensions are found with the following formula:</br>
# $dim = \large{\lfloor{\frac{dim_{in} + \, 2\, \cdot \, padding - \, size_{filter}}{stride} + 1}\rfloor}$</br>
# This rule is valid for each of the dimensions that convolution is applied.

# + [markdown] id="AnYe8jodCR64" colab_type="text"
# ### Training the CNN
# At this stage the CNN will be trained and then evaluated with the same metrics as the base model given above.

# + id="ZGqhGeAXCQO9" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 697} outputId="7728fd8b-0e7f-42a2-8f12-3a65bd3b9a2d"
# Compiling the model
cnn_model.compile(loss = 'sparse_categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
# Fitting the model to training data
cnn_history = cnn_model.fit(x = train_samples, y = train_labels, epochs = epoch_count)

# + id="IicVmHV4EDpW" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 68} outputId="33b44670-166a-4f73-8b12-1b077e96c3ef"
cnn_test_metrics = cnn_model.evaluate(x = test_samples, y = test_labels)
print('Loss in test set for CNN: ' + '{:.4f}'.format(cnn_test_metrics[0]))
print('Accuracy in test set for CNN: ' + '{:.2f}'.format(cnn_test_metrics[1]))

# + id="I2-n_n9aEL8Y" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 370} outputId="a81d4b87-71a9-4251-d49b-c59b2ac87ee1"
fig, ax = plt.subplots(1,2, figsize = (15,5))
ax[0].plot(range(1, epoch_count + 1), cnn_history.history['loss'])
ax[0].set_xlabel('epochs')
ax[0].set_ylabel('Training Loss')
ax[0].set_title('Change in Training Loss')
ax[1].plot(range(1, epoch_count + 1), cnn_history.history['accuracy'])
ax[1].set_xlabel('epochs')
ax[1].set_ylabel('Training accuracy')
ax[1].set_title('Change in Training Accuracy')
fig.suptitle('Change in loss and accuracy in convolutional model (CNN)', fontsize=16)
fig.show()

# + [markdown] id="y0x4vUR9F1xB" colab_type="text"
# ## Comparison
# For a more brief comparison, the plots for the DNN and CNN are shown as one. Since CNN extracts features first and then learrns these features, it was expected to perform better. This can be observed even in an easy dataset like Fashion MNIST, which both of the models can learn easily. 

# + id="jvI-tf6sGOcy" colab_type="code" colab={"base_uri": "https://localhost:8080/", "height": 370} outputId="58a0b05f-68fa-46ab-ef8c-a0662a829e9b"
fig, ax = plt.subplots(1,2, figsize = (15,5))
ax[0].plot(range(1, epoch_count + 1), cnn_history.history['loss'], label = 'CNN')
ax[0].plot(range(1, epoch_count + 1), base_history.history['loss'], label = 'Base Model')
ax[0].legend()
ax[0].set_xlabel('epochs')
ax[0].set_ylabel('Training Loss')
ax[0].set_title('Change in Training Loss')
ax[1].plot(range(1, epoch_count + 1), cnn_history.history['accuracy'], label = 'CNN')
ax[1].plot(range(1, epoch_count + 1), base_history.history['accuracy'], label = 'Base Model')
ax[1].legend()
ax[1].set_xlabel('epochs')
ax[1].set_ylabel('Training accuracy')
ax[1].set_title('Change in Training Accuracy')
fig.suptitle('Comparison of CNN and Base Model (with no onvolutions)', fontsize=16)
fig.show()

StarcoderdataJupyterScriptsDedupFiltered