Applying Data Augmentation to Handwritten
Arabic Numeral Recognition Using Deep Learning
Neural Networks
Akm Ashiquzzaman1 , Abdul Kawsar Tushar1 , and Ashiqur Rahman2
1

arXiv:1708.05969v4 [] 27 Sep 2017

2

Computer Science and Engineering Department, University of Asia Pacific, Dhaka, Bangladesh
Computer Science and Engineering Department, Bangladesh University of Engineering and Techology, Dhaka, Bangladesh
{zamanashiq3, tushar.kawsar, ashiqbuet14}@gmail.com

Abstract—Handwritten character recognition has been the
center of research and a benchmark problem in the sector of
pattern recognition and artificial intelligence, and it continues to
be a challenging research topic. Due to its enormous application
many works have been done in this field focusing on different
languages. Arabic, being a diversified language has a huge scope
of research with potential challenges. A convolutional neural
network model for recognizing handwritten numerals in Arabic
language is proposed in this paper, where the dataset is subject to
various augmentation in order to add robustness needed for deep
learning approach. The proposed method is empowered by the
presence of dropout regularization to do away with the problem
of data overfitting. Moreover, suitable change is introduced
in activation function to overcome the problem of vanishing
gradient. With these modifications, the proposed system achieves
an accuracy of 99.4% which performs better than every previous
work on the dataset.
Keywords—Data Augmentation, Dropout, ELU, Deep Learning, Neural Network.

I. I NTRODUCTION
Automatic character recognition, also known as optical
character recognition (OCR), has very high commercial and
pedagogical importance and has been receiving a profound
interest from researchers over the past few years. It is the key
method in check reading, textbook digitalization, data collection from forms, and in other numerous kinds of automated
data extraction. And this process is being applied to different
languages all over the world.
Arabic is the fifth most spoken language in the world. And it
is also one of the official language of United Nation from 1973.
Along with 290 million native speakers, it is spoken by 422
million speakers in 22 countries [1]. Arabic is the major source
of vocabulary for languages Turkish, Uighur, Urdu, Kazakh,
Kurdish, Uzbek, Kyrgyz, and Persian. And alphabets of many
other languages like Persian, Syriac, Urdu, Turkish are quite
similar to Arabic [2].
The past works of OCR of hand written alphabets and
numerals were concentrated on Latin languages. Gradually
other languages are also coming to the fore. But Arabic still
has many unexplored fields of study. Maximum work done on
this language was to detect printed characters [3]; however,
detection of hand written alphabets are more challenging

Fig. 1. Handwritten Arabic digits and corresponding inverted images [5]

than printed characters. Some work has been done on Arabic
handwritten digit recognition via patterns leanred from other
related languages [4]. The interesting point of note regarding
Arabic is, though the words as well as characters are written
from right to left, the numerals are written from left to right.
Fig. 1 depicts the ten Arabic numeral classes.
Das et al. [6] have devised an method to recognize handwritten Arabic numerals by multilayer perceptron (MLP) with
a considerable accuracy. In [5] a convolutional neural network
(CNN) model is used with dropout based on the model [6] that
achieves a better accuracy on the same dataset. In this paper,
we propose a modification to the same method devised in [5].
We introduce data augmentation to make learning more robust
against overfitting. Exponential linear unit (ELU) is introduced
instead of Rectified Linear Unit (ReLU) to fix the vanishing
gradient problem.

Fig. 2. CNN used in [5]

II. BACKGROUND
The method devised in [6] to recognize Arabic numerals
uses MLP as a classifier. It uses a set of 88 features; among
them 72 are shadow feature and 16 are octant features. To
train this MLP a huge data set of 3000 handwritten sample is
used which is obtained from CMATERDB 3.3.1 [7]. Each of
these pictures is scaled to the size of 32 × 32. For uniform
training each pixel is redefined in gray scale.
The MLP used in [6] has a single hidden layer along
with input and output layer. And this single hidden layer is
enough to classify the given data set [8]. The setting of layers
for the MLP method is shown in Fig. 4. The width of the
network is determined through trial and error to get a good
bias and minimal variance. Supervised learning is undertaken
and performed over 2000 samples. Simple back-propagation
algorithm is used for training.

Fig. 4. MLP used by Das et al. [6].

The result was evaluated by cross validation, and each
time the number of neurons are varied to get a perfect biasvariance combination. By exchanging the number of neurons
arbitrarily between training data set and test data set error can
be brought very close to zero, but this will generate worse
performance when tested on samples outside of sample dataset.
This condition is known as “overfitting” [9] and this is because
the model gets too much acquainted to the training data set
and cannot generalize over a broader spectrum as a result. The
number of neurons in hidden layer is finally fixed to 54 that
gives an accuracy of 93.8% in recognizing Arabic numerals.
Das et al. split the total data set at a ratio 2:1 for training
and testing. The images are normalized before feeding into
the network. Before feeding data into MLP the images were
transformed into a simple one dimensional vector.
The model described in [5] uses the same dataset as [6]
but adopts the method of CNN for numeral recognition. The
images are kept in the original form for CNN; however, color
of the images are inverted before feeding. Fig. 5 shows the
final condition of images before feeding in CNN model. Their
model consists of two layers of convolution with kernels of
size 5x5 and 3x3, respectively. The fully connected network
consists of a hidden layer and one final layer with 10 neurons
for each numeral class as shown in Fig. 2. They use ReLU
as activation function and categorical cross entropy for error
calculation. Softmax function is used to get the final result
from output layer. Total dataset is split at ratio 2:1 for training
and validation purpose. This model gives an accuracy of 97.4%
over validation dataset.

Fig. 3. Proposed CNN

TABLE I
S OME WELL KNOWN ACTIVATION FUNCTIONS [10] [11]

Name

Plot

Equation

f (x) =

Logistic

Derivative
f 0 (x) = f (x) (f (x) − 1)

1
e−x +1

tanh

f (x) = tanh(x)

Arctan

f (x) = tan−1 (x)

ReLU


 0
f (x) =
 x

ELU


 α(ex − 1)
f (α, x) =
 x

f 0 (x) =


 0
f 0 (x) =
 1

f or x < 0
f or x ≥ 0

III. P ROPOSED M ETHOD
In this section we discuss the changes that we bring to the
method described in [5] to improve the accuracy. Two key
changes are brought about - we introduce data augmentation
to the CMATERDB 3.3.1 dataset, and change the activation
function from ReLU to ELU. The pictorial presentation of our
model is given in Fig. 3.
Data augmentation transforms our base data. In our case,
it simply takes our dataset of images and transforms it by
rotation, color variation or by adding noise. It makes our model
more robust against overfitting and numerically increases the
size of the dataset. We’ve apply ZCA-whitening as augmentation, images are also rotated in a range of 10 degree. The
images are shifted both horizontally and vertically to an extent
of 20% of the original dimensions randomly. The images
are zoomed randomly up to 10%. During this augmentation

Fig. 5. (a) Original Data. (b) Final Data after processing

f 0 (x) = 1 − f (x)2

f or x < 0
f or x ≥ 0

1
1+x2

f or x < 0
f or x ≥ 0


 f (α, x) + α
f (α, x) =
 1
0

f or x < 0
f or x ≥ 0

process the points outside the boundaries are filled according
to the nearest point.
The ReLU is sometimes plagued with the gradient vanishing
problem. From Table I it is seen that for region x < 0 the
derivation of ReLU is 0, and due to this in this region the
updating of weight vector stops. And this stops the learning
process. The ELU function can prevent this condition as its
derivative does not become zero in any point on the curve.
Furthermore, it assures a smooth learning .
In our model, we have four convolution layers, each with
ELU as an activation function. The kernel size is determined
through trial and error, and the window of 3 × 3 gives maximum accuracy. Next to the final layer of convolution, we define
a pooling layer doing max-pool with a pool size of 2×2. After
max-pooling the convoluted images are flattened by squashing
2D convoluted data and fed into the fully connected layers.
The final output layer contains 10 output nodes representing
10 classes of numerals. Softmax function is used to calculate
final result from the output layer. This function calculates the
probability of each class from the ELU value of each output
layer neuron [12].
Both the convolution and fully connected layers have a
dropout rate of 25% to prevent data overfitting problem. In
this method, in each epoch 25% neurons in each layers do not
update their weight vectors, and the effect of these neurons
are removed from network. If neurons are dropped, they are
prevented from co-adapting too much with the training set and
reduces overfitting.

TABLE II
P ERFORMANCE C OMPARISON OF P ROPOSED M ETHODS AND M ETHODS
D ESCRIBED IN [5] AND [6]

Method Name

Accuracy

Method-1

93.8

Method-2

97.4

Proposed Method

99.4

IV. E XPERIMENT
The success of any machine learning method largely depends on the size and correctness of dataset used. For deep
learning the effect of data set is even more vital. The CMATERDB 3.3.1 Arabic handwritten digit dataset is used [7].
It contains 3000 separate handwritten numeral image. Each of
them is 32x32 pixel RGB image. Similar to the process in [5],
we invert the images before feeding into CNN. As a result the
numerals are in white foreground on the backdrop of black
background. For the past works done on OCR, it is observed
through activation visualization that edges are a very important
feature in character recognition, and black background makes
the edge detection easier.
We train the proposed CNN model with CMATERDB
Arabic handwritten digit dataset and test against the test data
sample of same dataset. Similar to the process implementation
described in [5], our CNN have increased 4, 977, 290 instead
of total 75, 383 trainable parameters such as weights and biases. Model is implemented in Keras [13]. Experimental model
was implemented in Python using Theano [14] and Keras
libraries. The model is trained for 100 epoch with different
kernel sizes. The batch size is 128 for both training and
testing. Categorical crossentropy is used as the loss function
in this model. Adadelta optimizer [15] is used to optimize
the learning process. Among the 3000 images, 2500 are used
for training and 500 is used in validation. We have used a
computer with CPU intel i5-6200U CPU @ 2,30GHz and @
4GB ram. Nvidia Geforce GTX 625M dedicated graphics is
used for faster computation, i.e CUDA support for accelerated
training is adopted.

Fig. 6. Some misclassified images during training. Blue denotes actual class
and red denotes predicted class.

connected layers in later phase of the proposed model to
enhance feature extraction and recognition. The result was
significantly improved from the previous methods because of
the introduction of data augmentation during training. Data
augmentation virtually makes the image transformed into new
set of features for the neural network to detect and recognize.
It also shifts and zooms the training images, making the image
decentralized for the kernels to recognize them from various
positions.
Fig. 6 denotes some of the misclassified images during
training. It demonstrates some of misclassified images, which
are actually impossible to recognize manually due to morphological decomposition of figure shape. Table III shows
the class-wise classification of the test images, which denotes
the final accuracy rate of 99.40% . The columns denote true
classes and the rows denote predicted classes. The cell content
denotes the count of predicted classes.
VI. C ONCLUSION

V. R ESULT A ND D ISCUSSION
Table II shows the result of different models applied to
the same CMATERDB dataset. Method-1 denotes the method
proposed in [6] and Method-2 denotes the method proposed
in [5].
Our proposed method of CNN gives better accuracy than
both the methods. The prominent attribute of the deep learning
is that deep embedded layers recognize the features and cascade them into the final output prediction to cast classification.
This is the attribute that has contributed to deep learning
method being used in problems in image recognition and
disease prediction [16], among others. In our proposed model,
the CNN layers have been increased in number from the
model used by [5]. We have also added two other fully

The OCR is a benchmark problem in pattern recognition
and has wide commercial interest. The work is based on the
dataset CMATERDB 3.3.1. Das et al. performed the first work
done on this dataset while using MLP in recognition. This
model gains an accuracy of 93.8%. The work in [5] is based
on the work of [6] and uses CNN as its model. This model
uses dropout regularization and ReLU as activation function,
and for the final output softmax function is used in the output
layer. This model achieves an accuracy of 97.3%. This paper
proposes a modification to the model proposed in [5]. We add
data augmentation to prevent overfitting, as well as change the
activation function from ReLU to ELU to prevent vanishing
gradient problem and make the learning more uniform. After
adopting all these changes, the proposed model achieves an

TABLE III
C ONFUSION M ATRIX

0

1

2

3

4

5

6

7

8

9

0

50

0

0

0

0

0

0

0

0

0

1

0

50

0

0

0

0

0

0

0

0

2

0

0

49

1

0

0

0

0

0

0

3

0

0

0

50

0

0

0

0

0

0

4

0

0

0

0

50

0

0

0

0

0

5

0

0

0

0

0

50

0

0

0

0

6

0

2

0

0

0

0

48

0

0

0

7

0

0

0

0

0

0

0

50

0

0

8

0

0

0

0

0

0

0

0

50

0

9

0

0

0

0

0

0

0

0

0

50

accuracy of 99.4% which is better than any of the previous
works on this dataset.
ACKNOWLEDGMENT
The authors would like to thank department of Computer
Science and Engineering, University of Asia Pacific for providing support in this research.
R EFERENCES
[1] “World arabic language day,” accessed: 2017-05-27. [Online]. Available:
http://www.unesco.org/new/en/unesco/events/prizes-and-celebrations/
celebrations/international-days/world-arabic-language-day/
[2] A. Boucenna, “Origin of the numerals,” arXiv preprint math/0606699,
2006.

[3] A. Amin, “Off-line arabic character recognition: the state of the art,”
Pattern recognition, vol. 31, no. 5, pp. 517–530, 1998.
[4] A. K. Tushar, A. Ashiquzzaman, A. Afrin, and M. Islam, “A novel
transfer learning approach upon hindi, arabic, and bangla numerals using
convolutional neural networks,” arXiv preprint arXiv:1707.08385, 2017.
[5] A. Ashiquzzaman and A. K. Tushar, “Handwritten arabic numeral
recognition using deep learning neural networks,” in Imaging, Vision
& Pattern Recognition (icIVPR), 2017 IEEE International Conference
on. IEEE, 2017, pp. 1–4.
[6] N. Das, A. F. Mollah, S. Saha, and S. S. Haque, “Handwritten
arabic numeral recognition using a multi layer perceptron,” CoRR, vol.
abs/1003.1891, 2010. [Online]. Available: http://arxiv.org/abs/1003.1891
[7] “Google coe archieve - long-term storage for google code project hosting.” https://code.google.com/archive/p/cmaterdb/downloads, accessed:
2017-05-30.
[8] M. Kubat, “Neural networks: a comprehensive foundation by simon
haykin, macmillan, 1994, ISBN 0-02-352781-7,” Knowledge Eng.
Review, vol. 13, no. 4, pp. 409–412, 1999. [Online]. Available:
http://journals.cambridge.org/action/displayAbstract?aid=71037
[9] P. M. Domingos, “Bayesian averaging of classifiers and the overfitting
problem,” in Proceedings of the Seventeenth International Conference
on Machine Learning (ICML 2000), Stanford University, Stanford, CA,
USA, June 29 - July 2, 2000, 2000, pp. 223–230.
[10] D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate
deep network learning by exponential linear units (elus),” arXiv preprint
arXiv:1511.07289, 2015.
[11] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on
machine learning (ICML-10), 2010, pp. 807–814.
[12] G. E. Hinton and R. R. Salakhutdinov, “Replicated softmax: an undirected topic model,” in Advances in neural information processing
systems, 2009, pp. 1607–1614.
[13] F. Chollet, “keras,” https://github.com/fchollet/keras, 2015.
[14] Theano Development Team, “Theano: A Python framework for
fast computation of mathematical expressions,” arXiv e-prints, vol.
abs/1605.02688, May 2016. [Online]. Available: http://arxiv.org/abs/
1605.02688
[15] M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv
preprint arXiv:1212.5701, 2012.
[16] A. Ashiquzzaman, A. K. Tushar, M. Islam, J.-M. Kim et al., “Reduction
of overfitting in diabetes prediction using deep learning neural network,”
arXiv preprint arXiv:1707.08386, 2017.

