Binarization of image for varying background and foreground colors for OCR I am trying to develop an automated OCR system for images obtained from the internet. The images mostly consist of computer generated text imposed on a background, so the OCR should produce near perfect results. However, the texts vary in color, and the background its imposed on can be of different colors too or even photos. The OCR tool often generates garbage when the colors are different enough, so I have to binarize the image properly, cleaning up the text content.

I found a promising article here, and implemented the algorithm using Python OpenCV. However, it seems to work poorly for most computer generated text. I also tried some local binarization algorithms, but they also fail to separate most text colors, like red on an orange background, the details being lost once converted to grayscale.

Is there an alternative image processing algorithm for this task ?
This problem is well studied. It is termed scene text recognition. Of course, the state-of-the-art methods involve deep learning. In particular, I like the works, which use a bi-directional LSTM together with features extracted from Convolutional Neural Networks. This allows for end-to-end training and frees the segmentation or binarization requirement. In particular, I like the work:


  Reading Scene Text in Deep Convolutional Sequences - 
  Pan He, Weilin Huang, Yu Qiao1, Chen Change Loy2, Xiaoou Tang2
  AAAI 2016