Optical Character Recognition (OCR) for Telugu:
Database, Algorithm and Application
Konkimalla Chandra Prakash*, Y. M. Srikar*, Gayam Trishal*, Souraj Mandal*, Sumohana S. Channappayya

arXiv:1711.07245v1 [] 20 Nov 2017

Indian Institute of Technology Hyderabad, Kandi, Telangana - 502285
Abstract—Telugu is a Dravidian language spoken by more
than 80 million people worldwide. The optical character
recognition (OCR) of the Telugu script has wide ranging
applications including education, health-care, administration
etc. The beautiful Telugu script however is very different from
Germanic scripts like English and German. This makes the use
of transfer learning of Germanic OCR solutions to Telugu a
non-trivial task. To address the challenge of OCR for Telugu,
we make three contributions in this work: (i) a database of
Telugu characters, (ii) a deep learning based OCR algorithm,
and (iii) a client server solution for the online deployment of
the algorithm. For the benefit of the Telugu people and the
research community, we will make our code freely available at
https://gayamtrishal.github.io/OCR_Telugu.github.io/.
Index Terms—OCR, Telugu, Convolutional neural network,
Deep learning, Document Recognition.

I. I NTRODUCTION
Telugu is the official language of Telangana and Andhra
Pradesh. It ranks third by the number of native speakers
in India, fifteenth in the Ethnologue list of most-spoken
languages worldwide. It has a complex orthography with a
large number of distinct character shapes (estimated to be
of the order of 10,000) composed of simple and compound
characters formed from 16 vowels (called achus) and 36
consonants (called hallus). Optical Character Recognition
(OCR) is the mechanical or electronic conversion of images
of typed, handwritten or printed text into machine-encoded
text. The availability of huge online collection of scanned
documents justifies the necessity for an OCR system, but the
complex script and grammar make the problem a challenging
one.
The Indian subcontinent has more than 18 constitutionally
recognized languages with several scripts but commercial
products in Optical Character Recognition (OCR) are almost
non-existent. In South India, Telugu and Kannada languages
have similar scripts. The complexity is huge in case of
Telugu because of the large number of output classes possible
and the inter class variability. Further there aren’t any OCR
systems for Telugu that make use of the advances in deep
learning which has motivated us to build one. An OCR
system has a huge impact as a part of real life applications
than along with a word processor. In the literature, other
attempts to Telugu OCR have neither shown results on a
large dataset [14] nor considered all possible character and
vattu combinations which exist in the language. Here, we
describe a novel end-to-end approach for Telugu OCR.

* - Equal Contribution

Apart from data, classifier selection also has a significant
impact on an OCR system. Before deep learning was used,
feature learning was a critical step in the design of any
classifier because feeding raw data would not lead to the targeted results. Therefore, classification is generally performed
after the difficult process of appropriate feature selection that
distinguishes classes. The advent of Convolutional Neural
Networks (CNNs) has paved the way for automated feature
learning. Also, the strong generalization capability of this
multi-layered network has pushed the classification performance beyond human accuracy. Due to these reasons we
have used a CNN based classifier in our OCR system.
The rest of the paper is structured as follows. Section II
talks about previous works on OCR of Telugu and Kannada.
Section III briefly describes the methodology and novelties
introduced in this paper. Sub-section III (A) talks about the
proposed dataset. Sub-section III (B) presents the architectural details of the proposed CNN framework and the overall
model for classification of characters. Sub-section III (C)
gives an in-depth explanation of the prepossessing steps and
segmentation algorithm.
II. R ELATED W ORK
Optical character recognition (OCR) has been one of the
most studied problems in pattern recognition. Until recently,
feature engineering was the dominant approach that used features like Wavelet features, Gabor features, Circular features,
Skelton features etc; [1][2][3] followed by SVM or boosting
based classifiers. But the success of CNNs motivated us to
use them for Telugu character recognition.
The first reported work on OCR for Telugu can be dated
back to as early as 1977 by Rajasekharan, and Deekshatulu
which used used features that encode the curves that trace
a letter, and compare this encoding with a set of predefined
templates [12]. It was able to identify 50 primitive features
and proposes a two-stage syntax-aided character recognition
system. The first attempt to use neural networks was made
by M.B. Sukhaswami et.al., which trains multiple neural
networks, pre-classify an image based on its aspect ratio and
feed it to the corresponding network[4]. It demonstrated the
robustness of a Hopfield network for the purpose of recognition of noisy Telugu characters. Later work on Telugu OCR
primarily followed the featurization classification paradigm
[13].
The work by Jawahar et.al., [14] describes a bilingual
Hindi-Telugu OCR for documents containing Hindi and
Telugu text. It is based on Principal Component analysis
followed by support vector regression. They report an overall accuracy of 96.7% over an independent test set. They

perform character level segmentation offline by their data
collecting tools. However, they have only considered 330
distinct classes.
The work by Rakesh and Trevor [5] on Telugu OCR using
convolutional neural networks is also fascinating. They used
50 fonts in four styles for training data each image of size
48x48. However, they not consider all possible outputs (only
457 classes) of CNN. The work by Kunte and Samuel [20] on
Kannada OCR employs a 2 stage classification system that
is similar to our approach. They have first used wavelets for
feature extraction and then two stage multi layer perceptrons
for the task of classification. They have divided the characters
into seperate sub classes but have not considered all possible
combinations. We describe our approach next.
III. P ROPOSED M ETHODOLOGY
A general OCR system can be divided into four stages.
The first stage is skew correction in which tilt in the image
will be adjusted followed word segmentation and binarization
after which each document is converted into seperate words
and binarized. This is followed by character segmentation in
which word is divided into characters and a classifier is used
to classify the characters. The classifier engine is created by
training on a dataset with labels. Our paper introduces novelties in the dataset, classifier and pre-processing techniques.

are rotated or if they share common region when projected
on X-axis or X-axis.
Inspired by the success of deep learning, we have explored
CNNs to classify the characters and proposed a new architecture for the same. CNN is a type of feed-forward neural
network or a sequence of multiple layers which is inspired
by biological processes. It eliminates the dependency on
hand-crafted features and directly learns useful features from
data itself. It is a combination of both feature extractor
and classifier and mainly consists convolutional, pooling and
fully connected layers
General Telugu character consists of two main parts - the
main character and the vattu/gunintham as shown in Figure
1. Using a single CNN would be futile because of the huge
number of classes arising from various permutations of main
character, vattu and gunintham. Hence we have used a 2
cnn architecture for classifying the character. First CNN is
for identifying the main character and the second CNN for
identifying the vattu and/or gunintam present along with the
main character. The architectures for both the CNNs has been
shown in Figure 2.

A. Dataset
The choice of the dataset is the key for OCR systems.
Unfortunately, there are very few works on Telugu character
datasets. The work by Pramod et al,. [17] has 1000 words and
on an average of 32 images per category. They used the most
frequently occuring words in Telugu but were unable to cover
all the words in Telugu. Later works were based on character
level [18][19][20]. The dataset by Hastie [6] has 460 classes
and 160 samples per class which made up 76000 images. But
these works have not utilized all the possible combinations of
vattu and guninthas. Here, we propose a dataset which takes
into consideration all possible combinations of vattu and
guninthas with 17387 categories and nearly 560 samples per
class. All the images are of size 32x32. There are 6,757,044
training samples, 972,309 validation samples and 1,934,190
test samples which add upto 1 million images (10 GB).
Each character has been augmented with 20 different fonts
downloaded from [16], 5 different sizes, random rotations,
additive gaussian noise and spatial transformations. Our
dataset is novel because unlike other datasets which only
take into account the commonly occuring permutations of
characters and vattus, we have spanned the entire telugu
alphabets and their corresponding vattu and guninthas.
B. Classifier
The performance of an OCR system depends hugely on
the performance of its classifier. Previous works on Telugu
OCR have done the character level segmentation based on
histogram along X and Y direction. Assuming that the
histogram method for segmentation would work perfectly,
they have used SVM based classifiers for classification of
characters. But we have observed that in real scenarios,
histogram method fails to properly segment out the vattu
and main character together. It also fails when the characters

Figure 1: Main character and vattu.
C. Pre Processing and Segmentation
Pre-processing consists of skew correction in which tilt in
the image is adjusted and binarization after which the image
is binarized and segmented into individual words. This is
followed by character level segmentation and classification.
1) Skew Correction: When an image is captured by using
a phone’s camera, there is no guarantee that the text is
perfectly horizontal. But our segmentation algorithm is based
on the fact that there is no skew in the image. This makes
skew correction a matter of utmost importance. We have used
a straight line hough transform based technique for correcting
skew. Output of skew correction is shown in Figure 3. The
algorithm is explained in 1.
Algorithm 1: Skew correction
Image = erode(Image);
edge map = canny(Image);
h,a,d = hough transform(edge map);
AnglePeaks = hough line peaks(h, a, d);
for angle in AnglePeaks do
categorize the angles into 4 bins: -(-90 to -45), (-45
to 0), (0 to 45) and (45 to 90).
end
RotationAngle = mean(mode(bins));

(a) Architecture 1 for main character.

(b) Architecture 2 for vattu.

Figure 2: Architecture of 1st CNN (main character) and 2nd CNN (vattu and gunintham.)

(a) Input image.

(a) Input image.

(b) Otsu’s thresholding.

(b) Skew corrected image.

Figure 3: (a) Input image which contains skew, (b) Image
after skew correction
2) Binarization: Having a robust binarization algorithm is
essential for any OCR system. We have observed that images
taken by phone often have regions of shadow due to improper
lighting conditions. Most of the previous works have used
Otsu’s [16] algorithm for binarizing their text document.
Otsu’s thresholding has given us poor results because the
shadow region, due to similarity in intensity level with that
of characters was being binarized as foreground region.
We used a modified version of Otsu’s thresholding for our
binarization. The first step is to remove noise present in the
image. We have applied the morphological closing algorithm
for that task. We have also tried to use a mode based
thresolding algorithm. It was removing false predictions
to some extent but was also decreasing the thickness of
characters. After denoising, the logical OR is taken between
the denoised image and that of the mode based thresolding
algorithm. The output of Otsu’s method and our method has
been compared in Figure 4.
3) Word region segmentation: Segmentation for documents is well developed in case of English. Many papers
in the literature use Maximally Stable Extremal Regions
(MSER) for segmentation of characters in English. But this

(c) Our thresholding.

Figure 4: (a) Input greyscale image, (b) Output of Otsu’s
thresholding and (c) Output of our thresholding.

solution as such cannot be directly extended to Telugu,
because most of the dheergam and vattus get seperated if
MSER is directly applied. Due to these reasons we made
small modifications to MSER to take into consideration
dheergas and vattus.
In order to eliminate the possibility of dheergam and vattu
being segmented separately we merged the nearby characters
into one word. For this we dilated the output of MSER
to get words. The variable for extent dilation is calculated
approximately on the fly from the distribution of character
sizes in the image. The amount of dilation is different along
x and y axis so that it doesn’t merge with other words.
Algorithm explained in 2. Results are shown in Figure 5.
4) Character level segmentation: Every character from
the word is segmented by using Connected Components
Algorithm in Image Processing. After binarization of the
image we apply the algorithm to separate all the letters and
vattus. The letters and vattus are returned in the form of
components(groups of binary pixels). In the process minor

Algorithm 3: Character Segmentation

Algorithm 2: Our MSER Variant
Regions = MSER(Image);
FilteredRegions = [ ] ;
for R ∀ Regions do
e = height(R)/width(R);
aC = area(R)/boundingBoxArea(R);
if (e>0.1) and (e<10) and (aC>0.2) then
FilteredRegions.append(R);
end
end
Hist = hist(boundingBoxArea(FilteredRegions));
avgArea = averageArea(modeBin(Hist));
sideLength = sqrt(avgArea);
finImx = dilatex(RegImage,0.7*sideLength);
finalImage = dilatey(finImx,0.2*sideLength);
boxes = boundingBox(contours(finalImage));

(a) Input image

BinarizedWord = Binarize(WordRegion);
Components = ConnComps(BinarizedWord);
for Ci ∀ Components do
for Cj ∀ Components do
if (Ci overlaps Cj in both horizontal and
vertical direction) then
Group Ci with Cj which has maximum
common overlapping distance.
end
end
end
for C ∀ Components do
TextRegions[C] = C;
Pos[C] = [C.minh, C.minw, C.maxh, C.maxw] ;
end

(b) MSER Regions

Figure 6: Character Segmentation

(c) Dilation of Regions

(d) Word Proposals

Figure 5: (a) Input image for Word Segmentation, (b) All
segmented MSER regions are coloured white, (c) After
finding approximate side length and dilating, (d)
Segmented words from contour regions.
blobs are also removed from the components. In Telugu
language some vattus are not connected with the base letter
itself. So for connecting the base letter with its vattu we
measured the overlapping distance in horizontal and vertical
direction and grouped them together. Results are shown in
Figure 6.
5) Mobile Application: With the exploding usage of Mobile phones and Apps, we have developed an app that
deploys the OCR solution. We have built an Android App
for Online Image to text conversion with Industry standard
(MVP Architecture) which works on any Android (4.4+).
We used client-server based communication where the client
(App user) requests the server with an image and the server
responds to the app with a html file. The theme is specifically

Figure 7: UI of Telugu OCR App
made keeping in mind that old age or low vision people can
use it easily. The App uses camera or gallery for images.
This App will also be made public.
IV. R ESULTS
Initially, we only trained with a small subset of the existing
data and found that the performance of the classifier was
not upto the mark. Accuracy for character CNN and vattu

CNN were 95.8 and 91.1 %. Training on the entire data has
drastically improved the classifier’s performance. Accuracies
are mentioned in Table I. We have used our end to end OCR
system on real data and the results were good.
Batch size was set as 500 because of large training
data size. Initially, we trained our network using SGD +
momentum optimizer. Even after 70-80 epochs, the accuracy
was not satisfactory. By using adam optimizer, accuracy
improved just after 10 epochs. There was no decrease in
the validation accuracy from epoch 10 to epoch 15, so we
halted the training process. Our model was trained on GTX
1060 with 16GB RAM.
Table I: CNN accuracies.
Architecture
Cifar [21]
Cifar [21]
Lenet [22]
Lenet [22]
Ours
Ours

Type
Character
vattu
Character
vattu
Character
vattu

Accuracy
98.60
95.46
98.62
95.59
98.74
96.10

V. C ONCLUSION AND F UTURE W ORK .
We have presented a solution for Telugu OCR that includes
a database, algorithm and an application. We have spanned
the entire Telugu language while creating the dataset, so there
isn’t any further possibility of increase in data. Segmentation
algorithm can be improved to an extent that every character
is segmented together with its vattu and gunintham. Network
accuracy cam be further improved to make the classifier
better. This proposed work can be further extended to other
languages with the scope of having a common OCR system
for all the languages of India.
VIII. R EFERENCES
[1] Babu, Arja Rajesh. OCR for Printed Telugu Documents. Diss. Indian Institute of Technology Bombay Mumbai, 2014.
[2] Hu, Peifeng, et al. "Recognition of gray character using
gabor filters." Information Fusion, 2002. Proceedings of the
Fifth International Conference on. Vol. 1. IEEE, 2002.
[3] Ramanathan, R., et al. "Robust feature extraction
technique for optical character recognition." Advances in
Computing, Control, & Telecommunication Technologies,
2009. ACT’09. International Conference on. IEEE, 2009.
[4] Rao, P. V. S., and T. M. Ajitha. "Telugu script
recognition-a feature based approach." Document Analysis
and Recognition, 1995., Proceedings of the Third International Conference on. Vol. 1. IEEE, 1995.
[5] Achanta, Rakesh, and Trevor Hastie. "Telugu
OCR Framework using Deep Learning." arXiv preprint
arXiv:1509.05962 (2015).
[6] LeCun, Yann, et al. "Gradient-based learning applied
to document recognition." Proceedings of the IEEE 86.11
(1998): 2278-2324.
[7] Bastien, Frédéric, et al. "Theano: new features
and speed improvements." arXiv preprint arXiv:1211.5590
(2012).
[8] Cireşan, Dan, and Ueli Meier. "Multi-column deep
neural networks for offline handwritten Chinese character
classification." Neural Networks (IJCNN), 2015 International
Joint Conference on. IEEE, 2015.

[9] Otsu, Nobuyuki. "A threshold selection method from
gray-level histograms." IEEE transactions on systems, man,
and cybernetics 9.1 (1979): 62-66.
[10] Wolf, Christian, J-M. Jolion, and Francoise Chassaing. "Text localization, enhancement and binarization in
multimedia documents." Pattern Recognition, 2002. Proceedings. 16th International Conference on. Vol. 2. IEEE, 2002.
[11] Karatzas, Dimosthenis, et al. "ICDAR 2015 competition on robust reading." Document Analysis and Recognition
(ICDAR), 2015 13th International Conference on. IEEE,
2015.
[12] Rajasekaran, S. N. S., and B. L. Deekshatulu. "Recognition of printed Telugu characters." Computer graphics and
image processing 6.4 (1977): 335-360.
[13] Negi, Atul, Chakravarthy Bhagvati, and B. Krishna.
"An OCR system for Telugu." Document Analysis and
Recognition, 2001. Proceedings. Sixth International Conference on. IEEE, 2001.
[14] Jawahar, C. V., MNSSK Pavan Kumar, and SS Ravi
Kiran. "A bilingual OCR for Hindi-Telugu documents and
its applications." Document Analysis and Recognition, 2003.
Proceedings. Seventh International Conference on. IEEE,
2003.
[15] http://fonts.siliconandhra.org/
[16] Otsu, Nobuyuki. "A threshold selection method from
gray-level histograms." IEEE transactions on systems, man,
and cybernetics 9.1 (1979): 62-66.
[17] Sankar K, Pramod, C. V. Jawahar, and Raghavan
Manmatha. "Nearest neighbor based collection OCR." Proceedings of the 9th IAPR International Workshop on Document Analysis Systems. ACM, 2010.
[18] Prasanth, L., et al. "Elastic matching of online
handwritten Tamil and Telugu scripts using local features."
Document Analysis and Recognition, 2007. ICDAR 2007.
Ninth International Conference on. Vol. 2. IEEE, 2007.
[19] Babu, Arja Rajesh. OCR for Printed Telugu Documents. Diss. Indian Institute of Technology Bombay Mumbai, 2014.
[20] Kunte, R. Sanjeev, and RD Sudhaker Samuel. "An
OCR system for printed Kannada text using two-stage
Multi-network classification approach employing Wavelet
features." Conference on Computational Intelligence and
Multimedia Applications, 2007. International Conference on.
Vol. 2. IEEE, 2007.
[21] Krizhevsky, Alex, and G. Hinton. "Convolutional
deep belief networks on cifar-10." Unpublished manuscript
40 (2010).
[22] LeCun, Yann. "LeNet-5, convolutional neural networks." URL: http://yann. lecun. com/exdb/lenet (2015).

