Handwritten text recognition is challenging because of the virtually infinite ways a human can write the same message. Our fully convolutional handwriting model takes in a handwriting sample of unknown length and outputs an arbitrary stream of symbols. Our dual stream architecture uses both local and global context and mitigates the need for heavy preprocessing steps such as symbol alignment correction as well as complex post processing steps such as connectionist temporal classification, dictionary matching or language models. Using over 100 unique symbols, our model is agnostic to Latin-based languages, and is shown to be quite competitive with state of the art dictionary based methods on the popular IAM and RIMES datasets. When a dictionary is known, we further allow a probabilistic character error rate to correct errant word blocks. Finally, we introduce an attention based mechanism which can automatically target variants of handwriting, such as slant, stroke width, or noise.
@cite_30: This paper proposes an end-to-end framework, namely fully convolutional recurrent network (FCRN) for handwritten Chinese text recognition (HCTR). Unlike traditional methods that rely heavily on segmentation, our FCRN is trained with online text data directly and learns to associate the pen-tip trajectory with a sequence of characters. FCRN consists of four parts: a path-signature layer to extract signature features from the input pen-tip trajectory, a fully convolutional network to learn informative representation, a sequence modeling layer to make per-frame predictions on the input sequence and a transcription layer to translate the predictions into a label sequence. We also present a refined beam search method that efficiently integrates the language model to decode the FCRN and significantly improve the recognition results. We evaluate the performance of the proposed method on the test sets from the databases CASIA-OLHWDB and ICDAR 2013 Chinese handwriting recognition competition, and both achieve state-of-the-art performance with correct rates of 96.40 and 95.00 , respectively.
@cite_29: In this paper we demonstrate a modified topology for long short-term memory recurrent neural networks that controls the shape of the squashing functions in gating units. We further propose an efficient training framework based on a mini-batch training on sequence level combined with a sequence chunking approach. The framework is evaluated on publicly available data sets containing English and French handwriting by utilizing a GPU based implementation. Speedups of more than 3x are achieved in training recurrent neural network models which outperform state of the art recognition results.
@cite_23: Recurrent neural networks (RNNs) with Long Short-Term memory cells currently hold the best known results in unconstrained handwriting recognition. We show that their performance can be greatly improved using dropout - a recently proposed regularization method for deep architectures. While previous works showed that dropout gave superior performance in the context of convolutional networks, it had never been applied to RNNs. In our approach, dropout is carefully used in the network so that it does not affect the recurrent connections, hence the power of RNNs in modeling sequences is preserved. Extensive experiments on a broad range of handwritten databases confirm the effectiveness of dropout on deep architectures even when the network mainly consists of recurrent and shared connections.
@cite_2: We present a technique of using Deep Neural Networks (DNNs) for offline character recognition of Telugu characters. We construct DNNs by stacking Auto-encoders that are trained in a greedy layer-wise fashion in an unsupervised manner. We then perform supervised fine-tuning to train the entire network. We provide results on Consonant and Vowel Modifier Datasets using two and three hidden layer DNNs. We also construct an ensemble classifier to increase the classification performance further. We observe 94.25 accuracy for the two hidden layer network on Consonant data and 94.1 on Vowel Modifier Dataset which increases to 95.4 for Consonant and 94.8 for Vowel Modifier Dataset after combining classifiers to form an ensemble classifier of 4 different two hidden layer networks.
@cite_29 proposed hybrid RNN-HMM for English offline handwriting recognition. They introduced a new variant of a LSTM memory cell by using a scalar multiple for every gate in each layer of the RNN. The scaling technique for LSTM gates reduced Character Error Rate (CER) by 0.3 @cite_23 proposed Multidimensional RNN using dropout to improve offline handwriting recognition accuracy by 3 Dewan and Srinivasa @cite_2 and @cite_30 used CNNs for offline character recognition of Telugu and Chinese characters respectively. @cite_2 used auto encoders, where the model was trained in a greedy layer wise fashion to learn weights in an unsupervised fashion, then fine-tuned with supervised data.