title: Automatic Scanned Document Image Enhancement
body: <p>I'm developing automatic image enhancement based on Microsoft paper <a href="https://www.microsoft.com/en-us/research/uploads/prod/2016/11/Digital-Signal-Processing.pdf" rel="nofollow noreferrer">Whiteboard scanning and image enhancement</a> </p>
  
  <p>in the "White balancing and image enhancement" section they provide the steps for the enhancement:</p>
  
  <p><strong>First: they estimate the background of the scanned document or the detected white Board:</strong></p>
  
  <blockquote>
    <p><strong>1.</strong> "Divide the whiteboard region into rectangular cells. The cell size should be roughly the same as what we expect
    the size of a single character on the board (15 by 15 pixels in our implementation)."</p>
  </blockquote>
  
  <p>then</p>
  
  <blockquote>
    <p><strong>2.</strong> "Sort the pixels in each cell by their luminance values. Since the ink absorbs the incident light, the luminance of
    the whiteboard pixels is higher than stroke pixels’. The whiteboard color within the cell is, therefore, the color with
    the highest luminance. In practice, we average the colors of the pixels in the top 25 percentile in order to reduce
    the error introduced by sensor noise"</p>
  </blockquote>
  
  <p>then </p>
  
  <blockquote>
    <p><strong>3.</strong> "Filter the colors of the cells by locally fitting a plane in the RGB space. Occasionally there are cells that are
    entirely covered by pen strokes, the cell color computed in Step 2 is consequently incorrect. Those colors are
    rejected as outliers by the locally fitted plane and are replaced by the interpolated values from its neighbors."</p>
  </blockquote>
  
  <p>My problem is with the second and the third steps:</p>
  
  <p><strong>How they get the luminace value, should I convert the input image to YUV color space and get the luminace value from the Y channel or just work on RGB color space?</strong></p>
  
  <p><strong>How to fit a local plane in the RGB space ?</strong></p>
  
  <p>Here is my python code that I tried to make cell from input image, get the luminance value from YUV color space and a simple result that seems incorrect compared by the result they get in the paper.</p>
  
  <p><strong>Python Code:</strong></p>
  
  <pre><code>import cv2
  import numpy as np
  
  
  
  ## Return List of cells from a given Image
  def SubImage(image):
      Cells = []
      CellRows = []
      for i in range(0,rows/CellSize):
          subIm = image[i*CellSize:(i+1)*CellSize,:]
          CellRows.append(subIm)
      for img in CellRows:
          for i in range(0,cols/CellSize):
              subIm = img[:,i*CellSize:(i+1)*CellSize]
              Cells.append(subIm)
      return Cells
  
  
  ## Sort luminosity Value
  def GetLuminance(Cells):
      luminance = []
      for cel in Cells:
          luminance.append(cel.max())
      return luminance
  
  
  ## Estimate the background color of the white board
  def UniformBackground(CelImage,img,luminance):
      a = 0
  
      for c in range(0,len(CelImage)):
          cel = CelImage[c]
          for i in range(0,cel.shape[0]):
              for j in range(0, cel.shape[1]):
                  cel[i,j] = min(1,cel[i,j]/ luminance[c])
      for i in range(0,rows/CellSize):
          for j in range(0,cols/CellSize):
              img[i*CellSize:(i+1)*CellSize,j*CellSize:(j+1)*CellSize] = CelImage[a]
              a = a + 1
  
  if __name__ == '__main__':
      img = cv2.imread('4.png')
      CellSize = 15
      rows,cols,depth = img.shape
  
  
      if (rows%CellSize !=0):
          rows = rows - rows%CellSize
  
      if (cols%CellSize !=0):
          cols = cols - cols%CellSize
  
      yuvImg = cv2.cvtColor(img, cv2.COLOR_BGR2YUV)
      # Get cells from Y channel
      CellsY = SubImage(yuvImg[:,:,0])
      CellsB = SubImage(img[:,:,0])
      CellsG = SubImage(img[:,:,1])
      CellsR = SubImage(img[:,:,2])
  
      # Get Luminance From Y cells
      LuminanceY = GetLuminance(CellsY)
  
      # Uniform Background
      UniformBackground(CellsB, img[:,:,0], LuminanceY)
      UniformBackground(CellsG, img[:,:,1], LuminanceY)
      UniformBackground(CellsR,img[:,:,2], LuminanceY)
  
      #bgrImg = cv2.cvtColor(imgB, cv2.COLOR_GRAY2BGR)
      #print imgB
      cv2.imwrite('unifrom.jpg',img)
  </code></pre>
  
  <p><strong>Input white Board image:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/rlzWx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rlzWx.png" alt="white Board image"></a></p>
  
  <p><strong>output Image:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/U2VFS.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/U2VFS.jpg" alt="Output image"></a></p>
  
  <p><strong>Expected Output:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/VLq5B.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/VLq5B.png" alt="expected Output"></a></p>