How to achieve image straightening as cam scanner does?
Geometric Transformation

Supose that $f$ is an image, defined over $(w,z)$ coordinate system, undergoes geometric distortion to produce an image $g$, defined over $(x,y)$ coordinate system. To perform this operation is used this

$$(x,y) = T\{(w,z)\} $$

So, one of most commmonly used forms of spatial transformation is the affine transform (Wolberg [1990]). It can be written in the matrix form as

$$ [x \ y \ 1] = [w \  z \ 1] \ \mathbf{T} = [w \  z \ 1] 
\left[ { \begin{array}{ccc}
         t_{11} &amp; t_{12} &amp; 0 \\ 
         t_{21} &amp; t_{22} &amp; 0 \\ 
         t_{31} &amp; t_{32} &amp; 1 \\ 
        \end{array}} \right]$$

This transformation can scale, rotate, translate or shear a set of points, depending on the values choosen for the elements of $\mathbf{T}$

Your case: Perspective Transformation

For perspective transformation, you need a 3x3 transformation matrix. Straight lines will remain straight even after the transformation. To find this transformation matrix, you need 4 points on the input image and corresponding points on the output image. Among these 4 points, 3 of them should not be collinear.

Implementations

Python


OpenCV


OpenCV has a lot of methods to perform image warping and geometric transformation an you can get the examples in the docs page


scikit-image


scikit-image also have many examples in the documation page.

For python just choose the module more comfortable to you or use both.


MATLAB ® Image Processing Toolbox


As the others examples the Math Work's software have a lot of functions to perform geometric transformation. Beware that is an commercial software and it not have a strong programming language if you want make a little software.

References

Wolberg [1990] G. Wolberg. Digital Image Warping . IEEE Computer Society Press, 1990.

Gonzalez [1987] Gonzalez, R.C., P. Wintz, 1987. Digital Image Processing. 2nd Ed., Addison-Wesley, Reading:MA.

Futher Reading

This lecture is a good introduction to this process