Homography valid for patch within the image but not the entire image? The homography I have is non-degenerate (det!=0), and generated from a valid planar pose. When I use it to warp the four corners of image, it returns something like the following:
1 0
3 2

instead of something like
0 1
3 2

where 0 represents top-left corner, 1 top right, 2 bottom left, 3 bottom right. It doesn't follow the clock-wise order anymore, and it's twisted.
The weird thing is, if I apply it to a local patch within the image, i.e. where the plane is, the returned result is valid.
How can this happen? Shouldn't it always return a valid quadrangle?
Are you sure that the points are all in front of the camera (z &gt; 0), if you apply a a projection like $(x, y) = (x/z, y/z)$ then points in a 3d quadrilateral where vertices 0 and 1 have negative z, you could end with a 2d figure like you described