arXiv:1801.09321v1 [cs.CV] 29 Jan 2018

Document Image Classification with Intra-Domain
Transfer Learning and Stacked Generalization of
Deep Convolutional Neural Networks
Arindam Das

Saikat Roy

Ujjwal Bhattacharya

CDA DVS
Valeo India Pvt. Ltd.
Chennai, India
arindam.das@valeo.com

Institute for Informatics
University of Bonn
Bonn, Germany
saikatroy@uni-bonn.de

CVPR Unit
Indian Statistical Institute
Kolkata, India
ujjwal@isical.ac.in

Abstract—In this work, a region-based Deep Convolutional
Neural Network framework is proposed for document structure
learning. The contribution of this work involves efficient training
of region based classifiers and effective ensembling for document
image classification. A primary level of ‘inter-domain’ transfer
learning is used by exporting weights from a pre-trained VGG16
architecture on the ImageNet dataset to train a document classifier on whole document images. Exploiting the nature of region
based influence modelling, a secondary level of ‘intra-domain’
transfer learning is used for rapid training of deep learning
models for image segments. Finally, stacked generalization based
ensembling is utilized for combining the predictions of the base
deep neural network models. The proposed method achieves
state-of-the-art accuracy of 92.2% on the popular RVL-CDIP
document image dataset, exceeding benchmarks set by existing
algorithms.

I. I NTRODUCTION
Documents can be classified into various classes based on
their text contents and/or their structural properties. During
a manual search for a particular document from a large
collection of documents, knowledge about the type or structure
of the document helps reduce the time necessary for the search.
However, the automatic accomplishment of the same is a
challenging task. In an early study [1], it was observed that a
real-life documents can be viewed in different ways, in both
geometric and logical structure spaces. The authors observed
that effective understanding of the document structure can
be realized through the use of an expert system and pattern
classification methods.
Automatic classification of document images is an effective
initial step of various Document Image Processing (DIP) tasks
such as document retrieval, information extraction and text
recognition, among others. The performance of a DIP system
may be enhanced through efficient initial classification of its
input documents into a number of pre-determined categories.
Automatic classification also has a significant role in indexing
the documents of a Digital Library. It has been seen that any
large volume of documents from different categories can be
better organized provided these are first classified into several
categories based on their structures [2].

With increased usage of digital means of document exchange and the growth in volume of digitized documents,
the requirement of an automatic system capable of understanding their logical structures is felt significantly for better
management of said documents stores including their efficient
archiving, retrieval, information mining and so on. However,
the development of an automatic system for classification of
arbitrary document images into repective true categories is
computationally a non-trivial task in contrast to the case of
human beings. The complexity of this task is increased due
to inter-class similarity and intra-class variability issues in
documents. For example, an advertisement may look like a
news item or a form may be very dense with items arranged
in multiple columns while another form may have only a few
items with each item in a distinct horizontal row and the rows
are separated wide apart. This fact gets further clarified from
the examples shown in Fig. 1.
Intuitively, from the perspective of a classifier, documents
can be characterized by their text content or structural information. [3]. Document classifiers based on text contents
use optical character recognition (OCR) techniques to extract
the texts in the document image and thus are susceptible to
the errors of the OCR method. On the other hand, there are
OCR systems which follow a structural analysis approach
by first determining the class of the document image based
on which an appropriate OCR module is employed [4]. The
effectiveness of such systems hinge on that of the structure
classification techniques employed and hence speaks for the
utility of structure learning as an important aspect of document
understanding.
A. Contribution
The proliferation and subsequent effectiveness of deep
learning techniques in a wide spectrum of tasks in machine
learning over the last decade has been spoken of ad-nauseum
in literature [5], [6]. Deep neural networks have been used
to provide unparalleled results in multiple domains including
document image classification. In the present work, deep
convolutional neural networks (DCNN) are used for automatic

(a)

(b)

(c)

(d)

(e)

(f )

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(p)

Fig. 1. Three samples from each of the sixteen classes of the RVL-CDIP database after resizing, preserving only document structure: (a) Letter, (b) Memo,
(c) Email, (d) Folder, (e) Form, (f ) Handwritten, (g) Invoice, (h) Advertisement, (i) Budget, (j) News, (k) Presentation, (l) Scientific Publication (m)
Questionnaire, (n) Resume, (o) Scientific Report (p) Specification.

understanding the structural aspects of a document for the
purpose of classification. While DCNN based approaches are
not new to this area, [7]–[9], the present study distinguishes
itself by studying the rapid training of effective document
region based classifiers. To achieve the same, multiple levels
of transfer learning are used. The unique nature of the region
based approach to document classification is utilized to make a
case for both inter-domain and intra-domain transfer learning
for this problem. Integration of the predictions from region
based classifiers are done by performing a thorough study into
meta-classification using stacked generalization. Finally, the
proposed techniques are validated experimentally by achieving state-of-the-art results on the popular RVL-CDIP dataset
containing 400,000 document image samples of 16 different
categories.
II. R ELATED W ORK
Several studies of automatic document classification have
been done in recent past. A related survey work can be found
in [3]. An approach for automatic generation of a decision
tree for logical labeling of business letters was proposed in
[10]. A 3-step approach based on certain GTree was studied
in [11] for partitioning a document image into its logical
objects. INFOCLAS, an early prototype system for indexing
and classifying printed business letters, was described in [12].
In [13] a machine learning based approach was proposed for

automatic discovery of the knowledge, that is, the relevant
features for document classifiers.
Further, in [14] a modified X-Y tree was used to describe
a document page and its hierarchical structure provided a
fixed length feature vector for an multilayer perceptron (MLP)
for classification. In [15], structural classifiers in addition to
k-Nearest Neighbours (kNN) and MLP classifiers was used
for classification of form document images. In [16], Hidden
Markov Models (HMM) were considered to be robust and
suitable for handling uncertainties and noise in input document
image samples. In [17], interval coding was considered to compute spatial layout of input document images and the resulting
fixed length feature vector was used for its classification based
on such an HMM.
In another study [2], a supervised classifier was trained
using given examples from each underlying class and exploited
visual similarity of document layout structure for their classification. Similarly [17] also proposed document classification
based on the layout similarity. In contrast, [18] used a recursive
representation of document structure to preserve relationship
among its different parts. Also recently, in [19], certain multiscale run length histograms for representation of document
images and a generative classifier model for efficient classification was proposed.
Deep learning techniques which have been popular across
multiple domains such as object recognition have also been

Fig. 2. VGG16 Architecture with Softmax layer replaced used as the base
model for each classifier

Fig. 3. Illustration of characteristic differences between the Header and Footer
regions in Scientific Publication and Advertisement classes

used in document structure learning tasks. Deep Convolutional
Neural Networks (DCNN) are deeper variants of convolutional
neural networks [20] and have recently exhibited their exceptional performance in object recognition tasks [21], [22]. In
[7], a deep CNN architecture with rectified linear units was
trained using dropout [23] for a 10-class document image classification task, popularly known as the Tobacco3482 dataset
[24]. Later, in [8], the concept of Transfer Learning [25] was
used to improve the recognition accuracy on the same standard
dataset by using a CNN pre-trained on a ImageNet dataset
[26]. Another work in the area which used transfer learning
includes [9] which also introduced region based modelling
and introduced the larger 16 class RVL-CDIP dataset. In [27],
results of a committee of lightweight supervised layerwise
trained models on the Tobacco3482 achieved decent results
without any transfer learning. In [28], extensive exploration
was done on the variation of components such as architectures, image size, aspect ratio preservation, spatial pyramidal
pooling, training set size among others using the AlexNet
architecture and the RVL-CDIP and ANDOC datasets. In
[29], run-length and fisher vector representations trained on an
MLP were compared to AlexNet and GoogLeNet architectures
on the RVL-CDIP dataset with deep models showing better
performance. In comparison, [30] concentrated on speed by
replacing the fully connected portion of the VGG architecture
with extreme learning machines (ELM).

found use in the task of document image classification utilizing
images containing just the structural framework of document
without much, if any, content being legible.
From [28]–[30], it was shown that the VGG16 model [32]
performs better on a document classification task than other
DCNN models. This information is utilized by us to select
the VGG16 model for our base classifier model for this task
with only the output layer replaced by one compatible with
our dataset. The general architecture is shown in Fig. 2 with
the Adam Optimizer [33] being used for training along with
a learning rate decay tuned based on the accuracy on the
validation set. The initial weights used are transferred from
a model trained on the ImageNet object recognition dataset.

III. P ROPOSED M ETHOD
A. Deep Convolutional Neural Network Architecure
DCNNs are currently one of the most popular models for
deep learning. DCNNs use many of the ideas used by conventional MLPs such as feed-forward connections, non-linear
activations, gradient descent and backpropagation. However,
their concept of shared weights and more recently activation
functions like the rectified linear unit, f (x) = max(0, x) make
them resistant to the vanishing gradient problem and effective
as deep learning architectures. Also their multi-filter weight
sharing concept help them discover robust features in input
spaces like images and audio hence making them powerful
supervised classifiers. [21], [31] As seen earlier, they have

B. Region based modelling for Document Classification
A combination of holistic and region based modelling for
document image classification was introduced in [9]. The
general idea consists of training multiple machine learning
models to capture influences of holistic as well as regionspecific visual cues of various document classes. For example,
a scientific publication or letter contains distinctively informative header sections than, say, an advertisement as seen in
Fig 3. In contrast, the central region of a memo can be very
different from that of a resume. This supports the fact that
a modelling technique governed by combination of features
from multiple document image regions can be effective in
their classification. As in [9] and [27], the various sections
used for our purposes in this work include Holistic (entire
image), Header, Footer, Left Body and Right Body.
C. Intra-Domain Transfer Learning of Region DCNNs
Transfer Learning involves the transfer of experience obtained by a machine learning model in one domain into another related domain [25]. While document classification and
object classification apparently seem like divergent domains,
architectures trained on the 1000 class ImageNet dataset have
proven to function as generalized feature extractors. Region
based training of document recognizers on an architecture
such as the VGG architectures creates an infeasible learning
problem due to the sheer training time of such a model.

dends by needing only a few iterations of fine tuning to provide
excellent results. In this work, each region based model was
only fine tuned for 4 iterations after L2 transfer with merely a
decaying learning rate. In contrast the VGG16-Holistic model
took 25 iterations after an L1 transfer to provide equivalent
convergence. This demonstrates a significant benefit for the
intra-domain transfer learning approach.
D. Stacked Generalization Schemes for Model Aggregation

Fig. 4. Flowchart of Proposed Model for Document Image Classification with
L1 and L2 Transfer Learning

For example, each VGG16 model contains more than 130
million trainable parameters. This makes training from scratch
(VGG16 on ImageNet weights) on each region-based model
be of a prohibitive computational cost. However, the nature
of training region based models creates a unique scenario that
can be used to bypass this predicament.
In this work, a VGG16 model, trained on ImageNet, is used
as initial weights of our holistic model thus constituting an
initial level-1 (L1) transfer of weights. The L1 transfer, of
course originates from a different domain and is the regular
inter-domain form of transfer learning. However, the holistic
model trained on whole images of the RVL-CDIP dataset and
can be thought of as a generalized document feature extractor.
The training sets for the region based models, although containing images of document regions and at a different scale,
are still essentially images of documents. Thus, this concept
is utilized by setting up another level (L2) of transfer learning
in which the region based models are initialized with weights
from the holistic model instead of the original VGG16 model.
The DCNN models trained are illustrated in Fig 4 and can
be summarized as follows:
– VGG16-Holistic: Trained with whole images from the
RVL-CDIP dataset. The initial weights transferred (L1)
from the VGG16 model trained on the ImageNet dataset.
– VGG16-Header/Footer/LeftBody/RightBody: Trained
with the Header/Footer/Left Body/Right Body section of
RVL-CDIP images. The initial weights are transferred
(L2) from the fully trained VGG16-Holistic model.
The L2 transfer is shown to demonstratively provide divi-

The strategy of stacking or stacked generalization was introduced by Wolpert [34]. It is an ensemble learning technique
for machine learning models designed to produce a reduced
generalization error than the same of its individual base
models. The idea has been used to a limited extent in literature
to combine DCNN models. It was shown in [27] that stacked
generalization performs significantly better than the simple
winner-takes-all approaches. Stacked generalization works by
training a meta-classifier on predictions of the base classifiers
on a hold-out dataset to learn the final set of predictions.
In the present study, the region-based DCNN models were
considered to be the base models for stacked generalization.
The concatenation of the base class softmax predictions on
the validation set for each base model are used to train the
meta-classifier while the same on the test sets are used to get
the final prediction.
Generalizing, let Qji be the c-dimensional vector consisting
of predicted probability values of the ith classifier for the j th
data sample corresponding to all of the underlying c classes. If
f (.) be a meta-classifier and n is the number of base classifier
models, then a meta-classifier can be said to learn the mapping
f : Rc×n → R
The domain of f , that is, the feature space for the metaclassifier is basically just an aggregate space of the softmax
outputs for each base DCNN model. Infact, the meta-feature
for a sample j is simply Qj1 ∧ Qj2 ∧ ... ∧ Qjn where ∧
represents concatenation of the prediction vectors from the
base classifiers.
For this work, c = 16 and n = 5 then the three schemes
studied here for generation of the meta classifier are detailed
below.
The meta-classifiers used in the proposed method include:
1) Linear Regression: Involves learning a linear mapping
of the input to predict the output.
2) Ridge Regression: Linear regression with l2regularization.
3) K-Nearest Neighbours (kNN): Instance space learning
algorithm which classifies a point by the majority of the
labels of its k nearest neighbours. Values of k tested
were 32, 64 and 128.
4) Support Vector Machine (SVM): SVMs are popular
supervised learning models which maximum margin
separators. Here, an SVM with an RBF kernel was used.
5) Bootstrap Aggregating (with SVM): Also known as
bagging, it is a meta-classifier which works by sampling
subsets (bags) of the input data and training multiple
base classifiers (SVM in this case) and aggregating

TABLE I
C OMPARISON OF ACCURACY BETWEEN H ARLEY ET AL . AND P ROPOSED
M ETHOD (1 DECIMAL POINT ) ON THE RVL-CDIP T EST S ET
DCNN Model
Holistic
Header
Footer
Left Body
Right Body
Ensemble of Models

Harley et al.
89.8
84.9
79.4
82.7
79.5
89.3

Proposed Work
91.1
86.0
81.2
85.2
82.2
92.2

TABLE II
C OMPARISON OF

ACCURACIES ON RVL-CDIP OF BEST MODELS FROM
DIFFERENT TECHNIQUES

Publication

Accuracy

Harley et al. [9]

89.80%

Tensmeyer et al. [28]

89.31%

Tensmeyer et al.

90.94%

Csurka et al. [29]

90.70%

Afzal et al. [30]

90.97%

Proposed Work

91.11%

Proposed Work

92.21%

Comments
Document section-based models
with weight transfer from Alexnet
with max voting ensemble
Uses AlexNet architecture with
spatial pyramidal pooling, without
transfer learning
Same architecture as above with
images resized to 384 × 384 and
aspect ratio preservation
The GoogLeNet architecture was
used with ImageNet based transfer
learning
Weights transfer from VGG-16
with MLP layers replaced by ELM
Trained on full document images
with weights transfer from VGG16 trained on Imagenet
MLNN based stacking of holistic
& region-based models with inter
and intra-domain weights transfer

Fig. 5. Comparison of Accuracies for different meta-classifiers in Stacked
Generalization

dataset can be seen in Fig. 1. The dataset is subdivided into
Training, Validation and Test Sets each containing 320000,
40000 and 40000 images respectively.
B. Preprocessing
The preprocessing for holistic and section-based datasets
were only marginally different. The images for the Holistic
dataset were initially resized to 224 × 224. For the regionbased images, the regions were extracted exactly as in [9], for
the ease of comparison. The extracted regions were further
resized to 224 × 224. Following the resizing, all datasets were
standardized and the single image channel was duplicated to
3 channels for VGG16 compatibility.
C. Evaluation

results. Here, 30 bootstrap samples each with 7500
training examples were used.
6) Extreme Learning Machine (ELM): ELMs (described
simplistically) involve randomly assigning weights in
the hidden units and train extremely fast compared to
traditional neural networks. A hidden layer of 100 units
was used in our experiments.
7) Multilayer Neural Network (MLNN): A 3-layer fully
connected neural network (256-256-16) with ReLU activation units and heavy dropout rate of 0.75 to control
overfitting between layers trained using Adam optimizer.
In case of regression techniques used under this scheme,
a simple argmax(.) was used on the output in a multivariate
regression problem. The meta-classifiers were chosen to be
as lightweight as possible, so as to not add further to the
computational cost of training the system.
IV. E XPERIMENTS

AND

R ESULTS

A. Dataset
The developed model was tested on a subset of the IITCDIP Test Collection known as the RVL-CDIP dataset. The
dataset consists of scanned grayscale images of documents
from lawsuits against American Tobacco companies and is
segregated into 16 categories or classes. A sample of the

It is seen in Table II that the VGG16-Holistic model with
L1 transfer by itself achieves a state of the art accuracy on the
RVL-CDIP dataset with Adam optimizer and gradual learning
rate decay based on the validation set accuracy. Of course,
the stacked generalized ensemble performs even better setting
a benchmark at 92.21%. However, if the region models were
trained simply with L1 transfer, the gain could be argued to
be too less given the computational cost of training multiple
large models (each model 130 million parameters).
As mentioned earlier, while the holistic model took about
25 iterations after the L1 transfer to converge, the VGG16Region models took only 4 iterations after L2 tranfer to show
comparable results. The performance can be further evaluated
by referring to Table I. Every region based model in this work
after only a few iterations of training (or fine tuning) easily
surpasses the slightly smaller AlexNet models trained in [9] by
L1 type transfer learning. The effectiveness of the L2 transfer
significantly reduces training time for the region based models
and eases the computational cost of training multiple models,
while preserving benefits characterized by the gain in accuracy.
Finally, extensive experiments into meta-classifiers for
stacked generalization demonstrate that fully connected
MLNNs otherwise known as MLPs to be the clear favorite as
the second level meta-classifier. The second best performance,
perhaps unsurprisingly is by Boosting, since it is itself an

ensemble technique utilizing the second best individual metaclassifier in this work, that is SVMs. Fig. 5 demonstrates
the comparative predictive performance of the various metaiclassifiers.
V. C ONCLUSION
In this work, rapid training of region based DCNNs has been
explored with multiple levels of transfer learning. The similarity between region based and the holistic input spaces facilitates the usage of intra-domain transfer learning in addition to
the general inter-domain variant. This allows fast convergence
and feasibility of region based models in what would otherwise
be an unnecessarily unwieldy machine learning problem. Also,
a thorough experimentation of stacked generalization has
been performed with multiple meta-classification algorithms.
Finally, a state-of-the-art result was not only set on the holistic
model at 91.11% but also on the final stacked generalized
model at 92.21% accuracy on the RVL-CDIP dataset thus
representing significant gain over existing methods. In the
future, faster but less accurate models such as [30] could be
explored as with region based intra-domain transfer. As crossdomain extensions of this work, the multi-domain trained
DCNN models can be expected to act as generalized text
detectors and be the starting points for training models for
text segmentation in domains such recognizing text in signs
and number plates in Automated Driving Assistance Systems
(ADAS) among others.
R EFERENCES
[1] Y. Tang, C. Suen, C. Yan, and M. Cheriet, “Document analysis and
understanding: A brief survey,” Proc. of Int. Conf. on Document Analysis
and Recognition, pp. 17–31, 1991.
[2] C. Shin, D. Doermann, and A. Rosenfeld, “Classification of document
pages using structure-based features,” Int. J. Doc. Anal. and Recognition,
vol. 3, no. 4, pp. 232–247, 2001.
[3] N. Chen and D. Blostein, “A survey of document image classification:
problem statement, classifier architecture and performance evaluation,”
Int. Journal of Document Analysis and Recognition, vol. 10, no. 1, pp.
1–16, 2007.
[4] E. Appiani, F. Cesarini, A. Colla, M. Diligenti, M. Gori, S. Marinai,
and G. Soda, “Automatic document classification and indexing in highvolume applications,” Int. Journal of Document Analysis and Recognition, vol. 4, no. 2, pp. 69–83, 2001.
[5] J. Schmidhuber, “Deep learning in neural networks: An overview,”
Neural networks, vol. 61, pp. 85–117, 2015.
[6] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521,
no. 7553, pp. 436–444, 2015.
[7] L. Kang, J. Kumar, P. Ye, Y. Li, and D. Doermann, “Convolutional
neural networks for document image classification,” 22nd International
Conference on Pattern Recognition (ICPR), pp. 3168–3172, 2014.
[8] M. Z. Afzal, S. Capobianco, M. I. Malik., S. Marinai, T. M. Breuel,
A. Dengel, and M. Liwicki, “Convolutional neural networks for document image classification,” Proc. of the 13th Int. Conf. on Document
Analysis and Recognition (ICDAR), pp. 1111–1115, 2015.
[9] A. W. Harley, A. Ufkes, and K. G. Derpanis, “Evaluation of deep
convolutional nets for document image classification and retrieval,” in
Document Analysis and Recognition (ICDAR), 2015 13th International
Conference on. IEEE, 2015, pp. 991–995.
[10] A. Dengel, “Initial learning of document structure,” Proc. of Int. Conf.
on Document Analysis and Recognition, pp. 86–90, 1993.
[11] A.Dengel, “About the logical partitioning of document images,” Nevada
University, Nevada (United States), Tech. Rep., 1994.

[12] R. Hoch, “Using ir techniques for text classification in document analysis,” in SIGIR ’94: Proceedings of the Seventeenth Annual International
ACM-SIGIR Conference on Research and Development in Information
Retrieval, B. W. Croft and C. J. van Rijsbergen, Eds. London: Springer
London, 1994, pp. 31–40.
[13] M. Junker and R. Hoch, “Evaluating ocr and non-ocr text representations
for learning document classifiers,” 12th International Conference on
Document Analysis and Recognition, pp. 1060–1066, 1997.
[14] F. Cesarini, M. Lastri, S. Marinai, and G. Soda, “Encoding of modified
x-y trees for document classification,” Proc. of the 6th Int. Conf. on Doc.
Anal. and Recog., pp. 1131–1136, 2001.
[15] P. Héroux, S. Diana, A. Ribert, and E. Trupin, “Classification method
study for automatic form class identification,” Proc. of the 14th Int. Conf.
on Patt. Recog., pp. 926–929, 1998.
[16] L. R. Rabiner, “A tutorial on hidden markov models and selected
applications in speech recognition,” in Readings in Speech Recognition,
A. Waibel and K.-F. Lee, Eds. Morgan Kaufmann Publishers Inc.,
1990, pp. 267–296.
[17] J. Hu, R. Kashi, and G. Wilfong, “Comparison and classification of
documents based on layout similarity,” Information Retrieval, vol. 2,
no. 2-3, pp. 227–243, 2000.
[18] M. Diligenti, P. Frasconi, and M. Gori, “Hidden tree markov models
for document image classification,” IEEE Trans. Pattern Analysis and
Machine Intelligence, vol. 25, no. 4, pp. 519–523, 2003.
[19] A. Gordo, F. Perronnin, and E. Valveny, “Large-scale document image
retrieval and classification with runlength histograms and binary embeddings,” Pattern Recognition, vol. 46, no. 7, pp. 1898–1905, 2013.
[20] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
applied to document recognition,” Proceedings of the IEEE, vol. 86,
no. 11, pp. 2278–2324, 1998.
[21] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
with deep convolutional neural networks,” in Advances in Neural Information Processing Systems, 2012, pp. 1106–1114.
[22] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”
in Proceedings of the IEEE conference on computer vision and pattern
recognition, 2015, pp. 1–9.
[23] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958,
2014.
[24] J. Kumar, P. Ye, and D. Doermann, “Structural similarity for document
image classification and retrieval,” Pattern Recognition Letters, vol. 43,
pp. 119–126, 2014.
[25] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. on
Knowl. and Data Eng., vol. 22, no. 10, pp. 1345–1359, 2010.
[26] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet:
A large-scale hierarchical image database,” Proc. of IEEE Conf. on
CVPR, pp. 248–255, 2009.
[27] S. Roy, A. Das, and U. Bhattacharya, “Generalized stacking of
layerwise-trained deep convolutional neural networks for document
image classification,” in 23rd International Conference on Pattern
Recognition (ICPR). IEEE, 2016, pp. 1273–1278.
[28] C. Tensmeyer and T. Martinez, “Analysis of convolutional neural networks for document image classification,” arXiv preprint
arXiv:1708.03273, 2017.
[29] G. Csurka, D. Larlus, A. Gordo, and J. Almazan, “What is the right
way to represent document images?” arXiv preprint arXiv:1603.01076,
2016.
[30] M. Z. Afzal, A. Kölsch, S. Ahmed, and M. Liwicki, “Cutting the error
by half: Investigation of very deep cnn and advanced training strategies
for document image classification,” arXiv preprint arXiv:1704.03557,
2017.
[31] T. N. Sainath, A.-r. Mohamed, B. Kingsbury, and B. Ramabhadran,
“Deep convolutional neural networks for lvcsr,” in Acoustics, Speech
and Signal Processing (ICASSP), 2013 IEEE International Conference
on. IEEE, 2013, pp. 8614–8618.
[32] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
[33] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,”
arXiv preprint arXiv:1412.6980, 2014.
[34] D. H. Wolpert, “Stacked generalization,” Neural Networks, vol. 5, pp.
241–259, 1992.

