Title: Hough transformation vs Contour detection for Rectangle recognition with perspective projection
Tags: image-processing;opencv
Question: I made rectangle detection work with contour detection and apply polygon with OpenCv to get location of the rectangle before adjusting the perspective projection. And it's working great. But some people in my group suggested Hough transformation instead. I wonder if there is any advantage of using Hough transformation for rectangle detection.
Update: I tried both of the methods. In my example, both methods worked fine after Canny edge detection. But since Hough transformation produces lines, we have to assume several things such as length of lines and connectability of the lines and should do additional computations such as searching connected lines and find corner points from the connected lines. Personally, I liked contour method better since its concept is simpler. With the method, you just search contours that can be approximated with closed and convex polygons with 4 corners and adjust the polygons for their perspective projections. That's about it.

Here is the accepted answer: What sort of results are you getting with contour detection so far? Got any examples?
Hough transform should work well for rectangle detection IFF you can assume that the sides of the rectangle are the most prominent lines in your image.
Then you can simply detect the 4 biggest peaks in hough space and you got your rectangle.
This works for example with a photo of a white sheet of paper in front of a dark background.
Ideally you would preprocess the image with blur, threshold, morphological operators to remove any small-scale structures before hough transform.
If there are multiple smaller rectangles or other sorts of prominent lines in your images, contour detection might be the better choice.
Some general advantages for the hough transform off the top of my head:
Hough transform can still work if part of the rectangle is obstructed or out of the frame.
Hough transform should be faster than contour detection, I guess?
Hough transform will ignore anything that is not a straight line, so you may have greater success with cluttered images. (if the rectangle sides are the most prominent lines)
In the end it probably depends on the input data. Got any examples?
Perhaps a combined approach would be best? see
Combining Hough Transform and Contour Algorithm for detecting Vehicles License-Plates
I did some experiments in using hough transform to detect rectangles a while back, you can see some preliminary results here:
http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&amp;t=14491&amp;start=9
Unfortunately that is all that exists at the moment, the project is currently on hiatus, eventually I hope to resume it when I am less busy.
I'd be very interested in your results in comparison.
(If you are doing perspective correction, also check out proportions of a perspective-deformed rectangle )
Comment for this answer: Yeah, I added a note about that. Rectangle detection with hough works well if the rectangle is the biggest and most prominent structure in your picture.
Comment for this answer: Thanks for your answer. Actually, my code is not much different from opencv sample [squares]( https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/cpp/squares.cpp?rev=4079) What I have in doubt is how can you actually detect rectanges with Hough transformation since that method is basically for line detection. I guess I have to go through all lines and find its connected lines and see if they are closed.

Here is another answer: Searching for contour detection with hough transform brought me to this SO.
To help future searchers, this blog post has a good walkthrough to do this with opencv:
http://opencv-code.com/tutorials/automatic-perspective-correction-for-quadrilateral-objects/
The concept:
1. Get the edge map - canny, sobel
2. Detect lines with Hough transform
3. Get the corners by finding intersections between lines.
4. Check if the approximate polygonal curve has 4 vertices with approxPolyDP
5. Determine top-left, bottom-left, top-right, and bottom-right corner.
6. Apply the perspective transformation with getPerspectiveTransform to get the  transformation matrix and warpPerspective to apply the transformation.
Comment for this answer: The link now leads to a spam site, try https://github.com/bsdnoobz/opencv-code/blob/master/quad-segmentation.cpp

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 2, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2012:00:04/21/12', 'FavoriteCount': 14.0, 'PostTypeId': 1, 'Tags': '<image-processing><opencv>', 'Title': 'Hough transformation vs Contour detection for Rectangle recognition with perspective projection', 'ViewCount': 24541, 'id': 10260519, 'user_AboutMe': '', 'user_AccountId': 390121.0, 'user_DisplayName': 'Tae-Sung Shin', 'user_Reputation': 19897.0, 'user_Views': 1696.0, 'user_id': 749973.0}}