Recent advancements in the area of Computer Vision with state-of-art Neural Networks has given a boost to Optical Character Recognition (OCR) accuracies. However, extracting characters text alone is often insufficient for relevant information extraction as documents also have a visual structure that is not captured by OCR. Extracting information from tables, charts, footnotes, boxes, headings and retrieving the corresponding structured representation for the document remains a challenge and finds application in a large number of real-world use cases. In this paper, we propose a novel enterprise based end-to-end framework called DeepReader which facilitates information extraction from document images via identification of visual entities and populating a meta relational model across different entities in the document image. The model schema allows for an easy to understand abstraction of the entities detected by the deep vision models and the relationships between them. DeepReader has a suite of state-of-the-art vision algorithms which are applied to recognize handwritten and printed text, eliminate noisy effects, identify the type of documents and detect visual entities like tables, lines and boxes. Deep Reader maps the extracted entities into a rich relational schema so as to capture all the relevant relationships between entities (words, textboxes, lines etc) detected in the document. Relevant information and fields can then be extracted from the document by writing SQL queries on top of the relationship tables. A natural language based interface is added on top of the relationship schema so that a non-technical user, specifying the queries in natural language, can fetch the information with minimal effort. In this paper, we also demonstrate many different capabilities of Deep Reader and report results on a real-world use case.
@cite_0: In this paper, I summarize research in document layout analysis carried out over the last few years in our laboratory. Correct document layout analysis is a key step in document capture conversions into electronic formats, optical character recognition (OCR), information retrieval from scanned documents, appearance-based document retrieval, and reformatting of documents for on-screen display. We have developed a number of novel geometric algorithms and statistical methods. Layout analysis systems built from these algorithms are applicable to a wide variety of languages and layouts, and have proven to be robust to the presence of noise and spurious features in a page image. The system itself consists of reusable and independent software modules that can be reconfigured to be adapted to different languages and applications. Currently, we are using them for electronic book and document capture applications. If there is commercial or government demand, we are interested in adapting these tools to information retrieval and intelligence applications.
@cite_19: 
@cite_1: Abstract Geometric matching under uncertainty is a long-standing problem in computer vision. This paper presents a simple and efficient branch-and-bound algorithm for finding globally optimal solutions to geometric matching problems under a wide variety of allowable transformations (translations, isometries, equiform transformations, others) and a wide variety of allowable feature types (point features, oriented point features, line features, line segment features, etc.). The algorithm only requires an implementation of the forward transformation (model-to-image) and an error model to be supplied. Benchmarks and comparisons of the algorithm in comparison with alignment and Hough transform methods are presented.
@cite_17: This paper deals with the transfer of knowledge on invoice document layout and extraction strategies, collected by users of the invoice recognition software smartFIX over several years of productive use, to other user's systems. The results of a project analyzing this 'treasure' of knowledge and putting it to use in the smartFIX system are presented. The evaluation shows that this transfer of knowledge using state-of-the-art techniques in transfer learning achieves significantly higher initial recognition rates than the unaugmented system, delivering instant economic advantages by reducing accountant personnel workload.
A relevant problem for extraction is that of document template identification for which several techniques have been proposed. Bruel developed a matching technique that was more robust to noise in the template instance @cite_1 , and subsequently to capture the layout of a complex document @cite_0 . In [9], the authors generate a hierarchical representation of documents and templates using XY-trees and utilize tree edit distance to identify the correct template for a new instance. Hamza @cite_19 utilize a bottom-up aproach which involves keyword extraction followed by clustering to generate high level structures. The document is represented as a graph with these high level structures as nodes and edges indicative of spatial relationships. Documents are mapped via graph probing, which is an approximation of graph edit distance. @cite_17 have developed an invoice reading system, where documents are again represented by a graph but nodes are split into key-nodes and data-nodes, corresponding to keywords and values (numbers, dates etc). Graph probing is used to find the correct template. Extraction is done by examining the nearby words of every data-node.