Source Printer Identification from Document Images Acquired using Smartphone
Sharad Joshi Suraj Saxena Nitin Khanna
cs.CV eess.IV

IEEEexample:BSTcontrol





Source Printer Identification 
from Document Images Acquired using Smartphone
    Sharad Joshi, Suraj Saxena, Nitin Khanna, Member, IEEE
Sharad Joshi, Suraj Saxena, and Nitin Khanna are with the Multimedia Analysis and Security (MANAS) Lab, Electrical Engineering, Indian Institute of Technology Gandhinagar (IITGN), Gujarat, 382355 India. E-mail: {nitinkhanna}@iitgn.ac.in
    
=======================================================================================================================================================================================================================================================================================================



empty


Vast volumes of printed documents continue to be used for various important as well as trivial applications. 
Such applications often rely on the information provided in the form of printed text documents whose integrity verification poses a challenge due to time constraints and lack of resources.
Source printer identification provides essential information about the origin and integrity of a printed document in a fast and cost-effective manner.
Even when fraudulent documents are identified, information about their origin can help stop future frauds.
If a smartphone camera replaces scanner for the document acquisition process, document forensics would be more economical, user-friendly, and even faster in many applications where remote and distributed analysis is beneficial.
Building on existing methods, we propose to learn a single CNN model from the fusion of letter images and their printer-specific noise residuals.
In the absence of any publicly available dataset, we created a new dataset consisting of 2250 document images of text documents printed by eighteen printers and acquired by a smartphone camera at five acquisition settings.
The proposed method achieves 98.42% document classification accuracy using images of letter `e' under a 5×2 cross-validation approach.
Further, when tested using about half a million letters of all types,
it achieves 90.33% and 98.01% letter and document classification accuracies, respectively, thus highlighting the ability to learn a discriminative model without dependence on a single letter type.
Also, classification accuracies are encouraging under various acquisition settings, including low illumination and change in angle between the document and camera planes.


Printer Classification, Convolutional Neural Network, Forgery Detection, Printer Dataset, Smartphone.



§ INTRODUCTION

Usage of digital documents has increased sharply in the last decade.
However, security issues, cost of transition, and acceptability by the workforce restrict a complete transition from printed to digital documents.
Such restrictions have encouraged continued usage of printed documents in many financial and administrative dealings such as agreements, deeds, business communication, and record-keeping.
So, there is co-existence of digital and printed documents.
As per a global forest product facts and figures 2018 report provided by the Food and Agriculture Organization of the United Nations, production of printing and writing paper was 96 million tonnes in 2018 <cit.> and has been steady since 2014.
The humongous volume of printed documents requires fast and accurate digital systems to predict their origin and integrity.
Information about the source printer can provide useful information about the origin and integrity of a printed document to an investigator.

The problem of attributing the source printer to a printed document has been studied extensively in the literature using digital methods <cit.>.
Two main approaches to source printer identification include (1) extrinsic methods based on embedding a signal in the printed document <cit.> and (2) intrinsic methods that exploit artifacts introduced by the combination of various electro-mechanical parts of a printer <cit.>.
Apart from being costly and complex, extrinsic solutions require access to the printer before the document is printed, which may not be feasible as manufacturers are not legally bound to integrate such solutions into their printers.
On the other hand, intrinsic solutions only require sample document(s) printed from the printer and
can be used to investigate documents printed in the past.


All the existing methods in the literature make use of a reference scanner to acquire a digital image of the printed document.
On the other hand, smartphones with an in-built camera have become very common.
As compared to scanners, smartphones are compact, easy to use, and can be quickly deployed to acquire and transmit document images in a remote working environment.
Most importantly, they are light-weight gadgets that human beings have become accustomed to carry along almost all the time.
The document analysis community has recently started working towards replacing batch scanners by smartphone cameras <cit.>.
In document forensics, a very recent approach proposed a method for source identification of colored images printed by color laser printers <cit.>.
There are significant differences between the working of color and `black-and-white' (grayscale) printers.
So, the method proposed in <cit.> cannot be used on grayscale documents.
In this work, we explore the scenario whereby a smartphone camera can replace the reference scanner used to convert text documents printed by black toner into digital images for source identification (Figure <ref>).

Traditional source printer identification methods relied on handcrafted features extracted from texture patterns <cit.>.
Recent works have shown that the convolutional neural network (CNN) can replace the traditional pattern recognition pipeline for attributing the source printer of printed text documents <cit.>.
In this work, we propose a combination of native letter image and its noise residual representation into a two-channel image, which allows learning a single CNN model.
Our experimental analysis shows that this strategy works better than learning multiple CNNs in parallel, one for each type of noise residual proposed in <cit.>.
We use a series of experiments to show the efficacy of the proposed two-channel CNN-based approach.
The major contributions of this work include the following: 

    
  * To the best of our knowledge, this is the first approach for source printer identification of printed text documents using a smartphone camera.
    
  * Introduces a single CNN approach, using a two-channel combination of letter image and its noise residual, capable of learning a single model.
    
  * Introduces a new smartphone acquired dataset consisting of 2250 images of text documents printed from eighteen printers under five different types of settings.
    
  * Performs better than state-of-the-art methods achieving 98.42%, 94.70%, 98.12%, 97.86%, and 97.69% page-level classification accuracy using a 5×2 cross-validation approach with dataset acquired at 0^∘, +5^∘, -5^∘,  free-hand, and low illumination settings, respectively.
    
  * Achieves 90.33% letter-level and 98.01% document-level classification accuracy when tested on about half million letter images; thus paving the way for using all letter types and intra-page forensic tasks.


We discuss the existing methods for source printer identification in Section <ref>.
The proposed system has been described in Section <ref>. Section <ref> discusses an evaluation of the proposed method using a new smartphone acquired dataset.
At last, we discuss the outcomes of the proposed work and draw out the conclusions in Section <ref>.



§ RELATED WORK


The problem of source printer identification is well defined in the literature and has been addressed in detail in the last decade <cit.>.
Imperfections in the printing process lead to artifacts in the printed document, which are invisible to the naked eye <cit.>.
These artifacts may be observed in the zoomed versions of printed letters (Figure <ref>).
Early methods relied on chemical and microscopic analysis. They require an expert's supervision, are time-consuming, and may even damage the printed document under investigation <cit.>.
The whole process has the potential to be automated using digital methods; speed depends upon the availability of computer hardware and does not require an expert's supervision.
Since printers treat text and images differently, source printer identification from printed text and printed images need to be addressed separately.
Since this work deals with text documents, only text-based methods are discussed here.
For a review of image-based methods, please refer <cit.>.
Apart from the methods for source printer identification, some methods have been proposed to classify the printing techniques, i.e., laser, inkjet, or photocopier <cit.>.
We classify the text-based methods into texture-based, noise residual-based, deep learning-based, and other methods as follows.



 §.§ Texture-based Methods

Researchers at Purdue University observed the appearance of light and dark lines perpendicular to the direction of the paper movement inside the printer. They termed it as banding, which acted as one of the earliest intrinsic signature used in a digital system for source printer identification <cit.>.
The efficacy of the method was shown on documents scanned at high resolution (2400 dpi).

Consequently, texture-based methods gained popularity as they provide encouraging performance on documents scanned at lower resolutions (600 dpi).
At a high level of abstraction, most of these methods can be seen as following the typical pattern recognition pipeline. Such methods use one or more hand-crafted features extracted from either all occurrences of a specific letter type (like `e') or non-overlapping rectangular blocks obtained by segmenting the printed text documents.
These features are used to learn a suitable classifier model, which predicts printer labels for each letter or a group of letters in the document under test. 
Majority voting on these labels predicts the printer for the whole document.
All occurrences of the letter `e’ were used in <cit.> to extract gray-level co-occurrence matrices (GLCM) followed by calculation of 22 statistical features.
They used a 5-nearest neighbor classifier to predict printer labels.
The authors also extended this technique for other classifiers, namely, support vector machine
(SVM) <cit.>, and Euclidean distances <cit.>.
Tsai et al. <cit.> used a combination of discrete wavelet transform (DWT) and GLCM based features extracted from all occurrences of a specific Chinese character followed by classification using SVM.
The same authors extended this method to a decision-fusion model-based approach for source printer identification, which applies a feature selection approach on a variety of features, including local binary pattern (LBP), GLCM and DWT <cit.>.
Ferreira et al. <cit.> used statistical features extracted from GLCM extended in multiple directions and scales as well as convolutional texture gradient filter (CTGF) which is a new feature descriptor based on filtering textures with a specific gradient. These features are extracted from all occurrences of `e' followed by SVM as the classifier.

All the methods discussed so far, extract features from only a particular letter type (usually the most frequent letter type).
Authors in <cit.> proposed a single-classifier-based system that used features based on local tetra patterns (LTrP).
The whole system is designed such that a single SVM model can be applied to all the letters printed on a test document.
Recently, the same authors introduced a printer-specific local texture descriptor (PSLTD) <cit.> that can be extracted from all the letters on a test page.
In contrast to previous methods, this method provides promising performance when the font type of letters in the test document is not present in the train documents. Thus, acting as the first step towards a font-independent descriptor.




 §.§ Noise Residual-based Methods

Some methods are based on extracting features from either only noise residual or a combination of native letter image and its noise residual.
Elkasrawi et al.  <cit.> used statistical features extracted from the noise residual of each text line, followed by SVM as a classifier.
The method in <cit.> is extended in <cit.> by including more features extracted from noise residual obtained after applying a spatial filter, Wiener filter, and Gabor filter on native letter images (of a specific character type).



 §.§ Deep Learning-based Methods

Ferreira et al. <cit.> proposed a data-driven approach using six convolutional neural networks (CNN) trained in parallel.
Three CNNs learn models using all occurrences of letter `e', their mean filtered versions, and their median filtered versions.
The learned models are used to extract features that are concatenated and used as input to a one-vs-one SVM, which predicts printer labels for all images of letter `e'.
Similarly, a set of predictions are obtained for all images of letter `a'  using the other three CNNs and a one-vs-one SVM.
Majority voting on predicted letter labels provides printer labels for the whole document.

Authors in <cit.> proposed a noise residual which improves the performance using the CNN architecture utilized in <cit.>.
Also, they showed improvement in performance using printer-specific data-augmentation and a spatial pyramid pooling (SPP) layer <cit.>.
The proposed method builds on this method and aims to reduce computational complexity, along with performance improvement.
However, data-augmentation and use of SPP layer significantly increase computation and memory requirements for a fixed size training data <cit.>.
So, we do not consider the data-augmentation and SPP layer in the proposed method.



 §.§ Other Methods

Kee et al. <cit.> applied principal component analysis (PCA) on all occurrences of `e'. They used the mean letter image and top p eigenvectors to characterize a printer profile.
Zhou et al. <cit.> proposed a text-independent approach using a specially designed and patented equipment to scan subtle textures on printed pages.
On the other hand, all other methods discussed here work with standard office scanners.
Another category of methods based on printer-specific geometric distortion has also been discussed in the literature <cit.>.
Though promising, these methods require a reference soft copy of the printed document and document image scanned at high resolution (1200 dpi).



§ PROPOSED METHOD







In this paper, we propose to make use of a single CNN based method to learn a model that could find the source printer of printed documents using camera-acquired images of those documents.
The proposed method builds on the findings of <cit.> and <cit.>.
Figure <ref> depicts the overview of the proposed method.
First module in the proposed method processes the document image to obtain two-channel letter images suitable for learning a printer discriminating CNN model. 
Then, an appropriate CNN architecture is designed. Finally, printer labels are predicted for a given test document.



 §.§ Pre-processing


At first, a reference smartphone is used to acquire images of all the printed text documents.
The default setting of most smartphones save the acquired image as a .jpg file, which is a three-channel RGB image.
Though, it would be worth exploring the impact of using an RGB document image instead of its gray-scale version. To reduce complexity, we assume that the black toner-based printer artifacts of our interest would be conveniently acquired in the gray-scale version.
So, in this work, we convert all RGB documents into their gray-scale versions.

All occurrences of letter `e' printed on a page are extracted using Matlab's in-built optical character recognition (OCR) in the form of a a rectangular image tightly bound around the printed letter (first row of Figure <ref>).
The hypothesis is that the printing process introduces certain device-specific artifacts, which can be described by the difference between the printed letter image and its ideally printed version (termed here as the noise residual).
Also, the native letter and noise residual images have been shown to carry complementary information (Table 1 of <cit.>).
So, we estimate a noise residual image corresponding to each extracted letter image.
An ideal printed letter image without such artifacts is expected to have equal toner spread in the interior region (i.e., a constant pixel intensity), and the region containing the white background must not contain any black toner.
However, zoomed letter images (Figure <ref>) show that the black toner spreads at the border of a letter.
Thus, differentiation between a letter's interior region and its background is challenging.

In the absence of any other reliable source of information, we estimate the noise introduced during the printing and image-acquisition process.

We estimate an ideally printed letter image as described in <cit.>.
First, we segment the letter image (I) into three regions, i.e., flat (F), edge (E), and background (B) using its intensity histogram as follows <cit.>:

    F = 
        { I(p)  I(p) ∈[0,αμ ]   and   ∀ p∈ I }


    E = 
        { I(p)  I(p) ∈(αμ,βμ]   and   ∀ p∈ I }


    B = 
        { I(p)  I(p) ∈(βμ,max(I)]   and   ∀ p∈ I }

Here, max(I) denote the maximum intensity value in I.
μ denotes the mean of all pixel intensities in I. α and β are experimentally chosen constants which are fixed for all images across experiments.
Next, a three-level ideal image (D) is estimated from the input image as follows <cit.>: 

    D(p) =  
        
            median(F),      I(p) ∈(0,αμ] 
      
            median(E),     I(p) ∈(αμ,βμ] 
     
            median(B),     I(p) ∈(βμ,max(I)]

The native letter image (I) is subtracted from the ideal image (D) to obtain the noise residual image (N). 
Figure <ref> shows samples of native letter images along with corresponding ideal and noise residual images.
Similar to the approach in <cit.>, both images are converted into two square images of fixed dimensions given by the patch size (hyperparameter).
Images having a larger dimension are center-cropped, i.e., extra rows or columns are removed from either side, whereas shorter dimensions are padded by zeros.
Our preliminary experiments suggested that the choice of padding by either zeros or the maximum possible pixel (i.e., 255 for an 8-bit image) did not have any significant impact on the classification accuracy.
However, since the pixel values in noise residual images are valued as `0' when there is no error, we chose padding by zeros.
Further, the letter images and their noise residuals are combined into 2-channel images such that CNN can learn a single model from them.



 §.§ Model Learning



The 2-channel letter images are used to train a single CNN model.
The CNN architecture used in <cit.> has been adapted for this problem as it has been shown to work well with small patches of letter images performing better than AlexNet <cit.> and GoogLeNet <cit.>.
It is adapted to include a batch normalization (BN) layer <cit.> to improve the speed of learning and an activation layer (ReLu) to introduce non-linearity that leads to improvement in classification accuracy by allowing the network to learn more discriminating models.
While designing the CNN architecture, we found that a combination of BN and ReLu performed better than using only either of them (Figure <ref>).

Unlike document images acquired by a scanner, the information contained in camera acquired images may be corrupted due to lower resolution, uneven lighting (even with on-camera flash <cit.>), blur, and perspective distortion <cit.>.
Further, the printer forensic traces may also be corrupted due to camera processing steps, which are optimized for general scene images and not for document forensic analysis.
These include sensor array (which induces sensor noise), demosaicing, and compression <cit.>.
To overcome such unwanted noise, our architecture consists of a larger number of smaller sized filters (i.e., 3 × 3) in each convolutional layer which can help extract a larger variety of filters from smaller image regions (filters are depicted in Figure <ref>).
The details of the CNN architecture are reported in Table <ref>.
Batch size is fixed at 100 whereas, Adam optimizer <cit.> is used with a learning rate of 0.001, decay of 0.0005 and cross-entropy loss.
The CNN model is trained for 50 epochs, and the model with the lowest validation loss is selected.




We visualize the 50 filters learned by the first convolutional layer in Figure <ref>.
Since the input to the CNN is a two-channel image, the first layer has 50 filters of dimensions 3 × 3 × 2, i.e., a total of 100 filters (50 × 2).
We show each filter channel separately arranged as a 5 × 10 matrix structure.
The model learns a variety of filters using both camera and scanner acquired images.
Most filters learned corresponding to native letter images acquired by smartphone camera are structurally similar to those learned from their scanner acquired version, with some variations in relative values.
This shows that the proposed method learns similar features from both types of data, thus showing the suitability of replacing a scanner with camera for document image-acquisition.
We emphasize that the models learned using both camera and scanner acquired document images contain the same printed text.



















 §.§ Prediction

A printed document under test goes through the pre-processing steps discussed in Section <ref>.
Then the CNN model learned in Section <ref> using train data is used to predict the source printer labels of all letters (or all occurrences of a specific letter type) printed on the document.
A majority voting over the predicted labels of all letters printed on the document provides its source printer label.


§ EXPERIMENTAL EVALUATION


Since there is no publicly available dataset of printed text documents acquired by a smartphone camera, we created a new dataset for this purpose.
The proposed system has been extensively evaluated using a series of experiments.
We have also compared with several baseline methods including the state-of-the-art data driven system CNN-{S^raw, S^med, S^avg}_a,e <cit.>, single classifier method CC-RS-LTrP-PoEP <cit.>, PSLTD <cit.>, and three variations of DenseNet architecture <cit.>.



 §.§ Dataset Details and Experimental Settings


We prepared a dataset by acquiring a total of 2250 images of text documents from Moto G3 smartphone with five different types of acquisition settings.
There are two types of variations considered in acquisition settings: (a) 
variation in relative angle between the camera and document planes (parallel (0^∘), up-tilt (+5^∘), down-tilt (-5^∘), and free-hand)
and (b) change in illumination (low illumination).
It is worth noting here that the relative angle of 0^∘ may not necessarily provide the same perspective in the document image as that acquired by a scanner.
This perspective distortion makes the characters printed farther from the camera look smaller, and some printed lines may not look parallel to each other <cit.>.
Eighteen printers listed in Table I of <cit.> printed twenty-five pages containing random text in Cambria font (English language) on white A-4 sheets using black colored toner.
These are the same documents used to create our scanner acquired
dataset <cit.> (for more details about the printed documents, please refer <cit.> and <cit.>).
Their visual appearance is grayscale, but, when a smartphone acquires images, it saves them as three-channel RGB Images.
The smartphone acquires each page only once (with a particular acquisition setting) using the default smartphone setting with HDR mode set to off. 
Except for the free-hand scenario, the smartphone acquires document images in a fixed indoor setup, where we fixed the location of the smartphone and documents using an appropriate physical structure. Specifically, we designated an area to place each document (i.e., a printed page) to be acquired and placed a selfie stick on top of it.
The selfie stick held our reference smartphone such that the angle between document and smartphone could be adjusted and fixed as per our requirements. The setup allowed us to acquire multiple versions of the dataset at an angle of 0, 5, and -5 degrees.
We created this setup to reduce the amount of human mishandling error and to analyze the impact of acquisition settings in a better manner.
We also acquired images in low lighting conditions at an angle of 0 degrees to analyze its impact on classification accuracy.

In contrast, the free-hand setting mimics a real-world scenario where a “common” camera-user tries to adjust the camera such that the document image is of good perceptual quality. So, in this scenario, there are no restrictions on the specific placement of the phone relative to a document.
Also, the relative angle between the camera and document planes may not be consistent over the acquisition of multiple printed documents.

Unless stated otherwise, all experimental results have been carried out using a 5 × 2 cross-validation approach on our dataset acquired at 0 degrees. For the first five-folds, out of 25 documents per printer, 12 printed documents are used in training and 13 in testing.
Similarly, for the rest of the five folds, train and test data are exchanged, such that there are 13 documents in training and 12 in testing.
Thus, there are a total of 450 document images acquired for each setting and we split them almost equally into train and test for each fold.
Each document contains about 2300 letters (about 300 occurrences of letter `e') and there are about 1 million letters overall.
The train and test sets are disjoint for each fold.
Moreover, the classification accuracy stats reported here are mean, median, and standard deviation (σ) values of accuracies obtained over all ten folds.








 §.§ Effect of Patch Size


First, we analyze the effect of different patch sizes on the performance of the proposed system.
For this purpose, we computed the median height and width of all occurrences of the letter `e' in our dataset acquired at 0 degrees, which are 31 and 26, respectively.
Based on this observation, we select the range of patch sizes from 22 to 32.

The results in Table <ref> show that the proposed system performs well with patches of all sizes.
Moreover, patches of size 32 × 32 and 30 × 30 provide the highest average classification accuracy of 98.42% when averaged over ten folds used in a 5 × 2 cross-validation approach.
However, the standard deviation (σ) of classification accuracies over the ten folds is smaller using the patch size as 30 × 30, i.e., 0.66.
So, for the rest of the experiments discussed in this section, we fix the patch size as 30 × 30 while experimenting with the proposed system.






































 §.§ Combination of Native and Noise Residual Images


We evaluate the impact of learning CNN models using only native letter images, only noise residual images, a combination of both, and two parallel CNNs from native and noise residual images.

As expected, in general, the classification accuracy increases with the number of training pages as shown in Table <ref>.
All experiments have been carried out using five iterations with varying documents in train and test.
However, the five combinations are fixed across all methods for a given number of training pages.
The results show that two-channel concatenation of native and residual images works consistently better than using two CNNs in parallel with different number of training pages.
Moreover, learning a CNN model from only noise residual versions performs the worst among all variants.
We also compare the number of CNN model parameters used in each variant. 
The native, noise residual, and the two-channel approaches require almost the same, i.e., about 955,000 model parameters.
Wehereas, the two parallel CNNs-based approach requires almost double the number of model parameters i.e., about 1,910,000.

The results in Table I of <cit.> confirmed that using the scanner-acquired public dataset of <cit.>, there is a consistent improvement in classification accuracies (for all the ten folds) by using a combination of native and noise residual versions as compared to using only native images.
But, the experiments with our camera-acquired dataset show that the proposed CNN architecture provides almost similar performance with only native letter images as well as with the two-channel combination of native images and their noise residual versions.
Specifically, the 2-channel approach works slightly better when training pages equal to 4, 6, 8 and 10 documents per printer.
Also, an evaluation of all types of letter images provides 98.01% and 97.87% average classification accuracy for the 2-channel and native only approaches, respectively.
One possible reason for this reduced impact of noise residual could be the extra noise introduced during the acquisition process by the camera as compared to a scanner.
Nonetheless, we use the two-channel approach while discussing the rest of the experiments highlighting the use of a camera in place of a scanner.










 §.§ Comparison with State-of-the-art Methods


We compare the proposed system with state-of-the-art methods as well as a popular state-of-the-art CNN architecture for image classification tasks, i.e., DenseNet <cit.>.
All accuracy results are reported using a 5 × 2 cross-validation approach.
The results in Table <ref> show that the proposed method outperforms existing methods.
Also, the proposed method performs better than three variations of the DenseNet architecture, i.e., DenseNet121, DenseNet169, and DenseNet201, which contain 121, 169, and 201 layers, respectively.
Apart from achieving the highest classification accuracy, the standard deviation is also the lowest with the proposed method, thus outlining its consistency in comparison to other methods.
For a fair comparison, the input to DenseNet is the two-channel image, which is the same as the input to the proposed CNN architecture.
The experiment on DenseNet was run using the in-built option provided by Keras <cit.> using the minimum allowed patch size, i.e., 32 × 32.



























































































































 §.§ Effect of Acquisition Settings on Classification Accuracy


We evaluate the proposed system under five acquisition settings on the nature of the surroundings and the image acquisition technique used by an investigator.
The first variation pertains to the change in the relative angle between the camera and document planes.
Here, we assume three possibilities: (a) the camera is placed exactly parallel to the document, (b) camera is at some angle relative to the document, and (c) the angle is not fixed and may arbitrarily vary while capturing both train and test documents.
The second variation is related to the amount of illumination available at the time of capturing the images.
Here, we use a much lower illumination than the previous scenarios.
The results show that the proposed method works consistently well for all the above scenarios (Table <ref>), including the free-hand scenario, which is the easiest to implement in a practical scenario without any constraints.
As expected, the average classification accuracy is highest using the controlled setting where camera and document planes are almost parallel to each other while it is lowest at 94.70% for the down-tilt (-5^∘) setting.
The confusion matrix for the free-hand scenario (Tables <ref>) shows that the proposed system works consistently for identifying the documents printed by all the printers in our dataset.
The proposed method correctly classifies all test documents printed by eight out of eighteen printers over all ten folds in the 5 × 2 cross-validation.

However, mis-classifications are higher using the down-tilt (-5^∘) setting because the proposed system extracts fewer letters.
Specifically, about 18,486 occurrences of letter `e' are extracted from down-tilt (-5^∘) setting as compared to 51,894 and 75,060 from up-tilt (5^∘) and free-hand, respectively.
This is a limitation of the letter extraction stage which currently uses a default OCR.
The decrease in OCR accuracy is a result of increase in perspective distortion <cit.> as well as misalignment of camera and document planes.
Flatbed scanners do not suffer from this limitation as the documents and scanning surface are perfectly aligned.
However, since the investigator can control the image acquisition process, such a scenario may be avoided even without the use of a more recent and improved letter extraction approach.
We emphasize that the same printed documents are used for training and testing in experiments with all the acquisition settings.





















 §.§ Experiment using All Types of Letters


The single-classifier approach introduced in <cit.> showed that it is possible to learn a single classifier model using all letter types.
This approach is particularly useful when a specific type of letters printed on the available documents is insufficient to train a discriminative classifier model, as shown in Table V of <cit.>.
We analyze the ability of the proposed method to learn a single CNN model from all types of letters in Table <ref>.
Results show that the proposed method can achieve performance similar to using only a single letter type (i.e., `e').
Moreover, the average character level accuracy on about half a million letters is 90.33%.
Such a high accuracy highlights that the CNN model is attributing the artifacts in a vast majority of letter images to the source printer, thus further strengthening the argument of using a camera in place of a scanner.
Further, it helps pave the way for intra-page forensic tasks such as detection of forgery created by either printing in empty spaces or pasting a small printed chit on portions of a text document <cit.>.



 §.§ Experiment on Scanner Dataset


We also analyze the performance of the proposed method on our scanner acquired dataset <cit.>.
We use the experimental settings used to report the same-font and cross-font results in Table IV of <cit.>.
The results (Table <ref>) show that the proposed method performs equivalent to the state-of-the-art methods for the same-font scenario (i.e., font type of letters in test data is present in train data) by correctly classifying test document images from all printers.
But, it fails with cross-font scenario (i.e., font type of letters in test data is absent in train data) similar to existing data-driven methods of <cit.> and <cit.>.
It is expected as the proposed method is dependent upon CNN to learn appropriate features based on the training data.






 §.§ Variations of Proposed CNN Architecture



We evaluate the effect of average and max pooling operations and various activation functions.

We compare commonly used activation functions, including exponential linear unit (ELU), rectified linear unit (ReLU), parametric rectified linear unit (PReLU), and hyperbolic tangent (Tanh).
The results in Table <ref> show that the proposed method provides similar performance with all combinations.
However, a combination of Tanh activation function and average pooling performs slightly better than others providing 98.81% average classification accuracy.





§ CONCLUSION


We proposed an approach to identify the source printer of a printed document via its camera acquired image.
The use of smartphones achieves significant letter and document level classification accuracy that would be useful in many practical scenarios as they provide more flexibility to end-users.
Our method uses a two-channel combination of the native letter image and its noise residual image to learn a single CNN model.
We use a series of experiments to show the efficacy of the proposed method on a newly created dataset.
Experiments on the scanner dataset in <cit.> (Table 1) highlighted a consistent improvement
due to a combination of native and noise-residual images.
In contrast, experiments on our camera acquired document dataset revealed that the noise-residual image does not have a significant impact on the performance.
One possible reason for this could be that the printer-specific noise residual estimation from camera acquired data is already corrupted by unwanted noise due to camera artifacts and blur.
Nonetheless, the proposed method performs better than state-of-the-art methods, including a popular state-of-the-art image classification method (i.e., DenseNet <cit.>) providing an average classification accuracy of 98.42 % using a 5 × 2 cross-validation protocol.

We also evaluated the performance of the proposed method under varying image acquisition scenarios, including the effect of angle between the camera and document planes, and intensity of indoor lighting.
Results indicate that the performance is almost consistent across all variations considered in this work, except down tilt scenario.
The analysis revealed that increased perspective distortion degraded letter extraction performance with the down-tilt dataset.
The proposed method achieves an average letter-level classification accuracy of 90.33% by learning a single CNN model from about 1 million letter images consisting of all types of letters printed in a specific font type and size.
Such a high accuracy highlights the strong ability of the proposed method to extract printer-specific signatures from camera-acquired letter images and paves the way for certain types of intra-page forgery detection.
Also, 98.01% of average page-level accuracy is obtained, which lowers the dependence on sufficient availability of a specific letter type.

Similar to previous data-driven approaches, the proposed method fails under the cross-font scenario.
Toner level may impact printer signature, so in real-world applications, it is expected that the training data is available over the entire printer toner cycle.
Several other scenarios may impact a printer's signature, including printer age and characteristics of printing paper.

Future work may include a large scale evaluation of source printer attribution using camera acquired document images in the wild.
Also, the impact of other variations in text like font size and other languages may be explored.










	



§ ACKNOWLEDGMENT


This material is based upon work partially supported by a grant from the Department of Science and Technology (DST), New Delhi, India, under Award Number ECR/2015/000583 and Visvesvaraya PhD Scheme, Ministry of the Electronics & Information Technology (MeitY), Government of India. 
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies.



IEEEtran

