Visual Tracking / Object Tracking: Estimate the Direction the Whale Is Moving I am trying to find out which way the whale is moving. So I have an image of the whale swimming in the water and I want to find out where is it located and which direction does it swim (with respect to X/Y coordinates of the image).



For example here the angle is approximately 30 degrees in respect to X coordinate.



So my idea was to use canny edge detection to extract edges, construct a matrix from the points of the edges, calculate SVD and use the U matrix from SVD as a rotation matrix to get the angle.

So I have done something like this in python:




Where the first image is the starting image, the second one is the edges, and the third one is the scatterplot with points from the edges and the red arrow is a direction of the angle I got from the SVD. The angles I got

Angle 146.73
Angle 135.08


are not even close to real angles (which are approximately 30 and 85 degrees).



I have tested that I get a correct angle when I SVD the gaussian matrix, which I rotated by some degree.

This is my first post, so if you need some clarifications (code that I used, explanation why I think this approach will work or anything else), please let me know.



So am I on the right track to find where the whale is going (and just made some silly mistake) or should I use another idea?
I wouldn't rely on edges as they are not connected and foam/waves also generate them.

I would rely on the contrast between the colors of skin, water and foam, to get a solid blob.

Possibly a two step process: discard the more saturated pixels (the sea), then among the achromatic keep the dark ones. After experimenting I saw that erosion can be useful to get rid of unwanted residues.

In the end you should get a big blob, and determine its dominant orientation (equivalent ellipse of inertia, Feret diameter, most distant points...)

Unsaturated pixels:



Dark pixels:



Combination:



Eroded:



Blobs and ellipse of inertia:





As there is a risk of blob fragmentation, you can consider merging the largest blobs.