Best approach for 2D Grid Image Segmentation I'm working on a project where I need to extract text from grocery discount flyers like the Costco announcement below (retrieved in a random google search, Costco is not the deal here):



If I just run OCR (like with Tesseract in python):

import cv2
import pytesseract
img = cv2.imread('costco.jpg')
text = pytesseract.image_to_string(img)
print(text)


I get:


  Cadbury Chocolate
  
  variety pack packet
  
  ere $12.99 i rom hagst 31026 2012
&gt; 
&gt; Je laa
&gt; + a
&gt; 
&gt; Wrigley’s Excel Gum variety
&gt; 
&gt; Backol 24
&gt; 
&gt; $13.79 fom agus 26.202
  
  OFF
  
  Solon Extra virgin olive oil [...]


Which is a lot noisy.

My guess is that splitting the image to its base squares enchances the recognition.

However, I'm confused on how to do it. I can classify images using a CNN, but am not sure about object recognition.

Should I have a sliding window and train several "grid box" objects on a generic CNN and then provide this window data to be classified? How to adapt to distinct object window sizes?
This is a really cool problem. You already have a working model here are a few different ways of going forward with the project.


Grouping text based on locality. "no segmentation"  Text region extraction in a document image based on the Delaunay tessellation 
Segmentation  Multiscale Edge-Based Text Extraction from Complex Images 
Training a map of the image then extracting text form the map.  Multi Oriented Text Detection 


Training the model to detect boxes followed by extracting the text from each box seems like a very smart direction to move in. This paper talks about the former.  automatic image segmentation and edge detection  Good luck!