title: OpenCV : Object detection and tracking based on feature detection
body: <p>I have just ventured into computer vision and trying to demystify various intricacies of it. I am trying to augment the kalman filter with a surf feature detector. But I do not understand how to call and use the kalman method after the homography and bounded rectangle has been constructed on the detected frame using surf features. I have detected the features and extracted the keypoints using a reference image after comparing with  the incoming frames. Then I used the flann matcher. </p>
  
  <p>Now, is it feasible to use the kalman filter since I want to track the motion and get the predicted motion. I have searched a lot but have not found that surf features can be used with kalman filter. All I am getting are suggestions to use cvBlobs for tracking. But, theoretically kalman filters are used for tracking purposes. However, I am confused since the several implementations of video based tracking using surf shows that surf can be used for tracking in itself. But my question is</p>
  
  <ul>
  <li><p>If kalman filter cannot be used with surf, then how do I implement the moments to get the coordinate measurement values as I need the information for motion prediction.</p></li>
  <li><p>can surf be used with kalman filter for tracking and if yes how to implement it after the object has been detected and bounded with a rectangle using the following code.</p>
  
  <p>Example : the object to be tracked <img src="https://i.stack.imgur.com/cHGZN.jpg" alt="book1.png"> . Some frames <img src="https://i.stack.imgur.com/zXUhZ.jpg" alt="frame1"> <img src="https://i.stack.imgur.com/DnapC.jpg" alt="rame2"></p>
  
  <p>/* Object Detection and recognition from video*/</p>
  
  <p>int main()
      {
         Mat object = imread( "book1.png",CV_LOAD_IMAGE_GRAYSCALE  );</p>
  
  <pre><code>if( !object.data )
  {
      std::cout&lt;&lt; "Error reading object " &lt;&lt; std::endl;
      return -1;
  }
  
  //Detect the keypoints using SURF Detector
  int minHessian = 500;
  
  SurfFeatureDetector detector( minHessian );
  std::vector&lt;KeyPoint&gt; kp_object;
  
  detector.detect( object, kp_object );
  
  //Calculate descriptors (feature vectors)
  SurfDescriptorExtractor extractor;
  Mat des_object;
  
  extractor.compute( object, kp_object, des_object );
  
  FlannBasedMatcher matcher;        
  
  namedWindow("Good Matches");
  namedWindow("Tracking");
  
  std::vector&lt;Point2f&gt; obj_corners(4);
  
  //Get the corners from the object
  obj_corners[0] = cvPoint(0,0);
  obj_corners[1] = cvPoint( object.cols, 0 );
  obj_corners[2] = cvPoint( object.cols, object.rows );
  obj_corners[3] = cvPoint( 0, object.rows );
  
  char key = 'a';
  int framecount = 0;
  VideoCapture cap("booksvideo.avi");
  
  for(; ;)
  {
      Mat frame;
      cap &gt;&gt; frame;
      imshow("Good Matches", frame);
  
  
      Mat des_image, img_matches;
      std::vector&lt;KeyPoint&gt; kp_image;
      std::vector&lt;vector&lt;DMatch &gt; &gt; matches;
      std::vector&lt;DMatch &gt; good_matches;
      std::vector&lt;Point2f&gt; obj;
      std::vector&lt;Point2f&gt; scene;
      std::vector&lt;Point2f&gt; scene_corners(4);
      Mat H;
      Mat image;
  
      //cvtColor(frame, image, CV_RGB2GRAY);
  
      detector.detect( image, kp_image );
      extractor.compute( image, kp_image, des_image );
  
      matcher.knnMatch(des_object, des_image, matches, 2);
  
      //THIS  LOOP IS SENSITIVE TO SEGFAULTS
      for(int i = 0; i &lt; min(des_image.rows-1,(int) matches.size()); i++) 
      {
          if((matches[i][0].distance &lt; 0.6*(matches[i][4].distance)) &amp;&amp; ((int) matches[i].size()&lt;=2 &amp;&amp; (int) matches[i].size()&gt;0))
          {
              good_matches.push_back(matches[i][0]);
          }
      }
  
      //Draw only "good" matches
      drawMatches( object, kp_object, image, kp_image, good_matches, img_matches, Scalar::all(-1), Scalar::all(-1), vector&lt;char&gt;(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
  
      if (good_matches.size() &gt;= 4)
      {
          for( int i = 0; i &lt; good_matches.size(); i++ )
          {
              //Get the keypoints from the good matches
              obj.push_back( kp_object[ good_matches[i].queryIdx ].pt );
              scene.push_back( kp_image[ good_matches[i].trainIdx ].pt );
          }
  
          H = findHomography( obj, scene, CV_RANSAC );
  
          perspectiveTransform( obj_corners, scene_corners, H);
  
          //Draw lines between the corners (the mapped object in the scene image )
          line( img_matches, scene_corners[0] + Point2f( object.cols, 0), scene_corners[1] + Point2f( object.cols, 0), Scalar(0, 255, 0), 4 );
          line( img_matches, scene_corners[1] + Point2f( object.cols, 0), scene_corners[2] + Point2f( object.cols, 0), Scalar( 0, 255, 0), 4 );
          line( img_matches, scene_corners[2] + Point2f( object.cols, 0), scene_corners[3] + Point2f( object.cols, 0), Scalar( 0, 255, 0), 4 );
          line( img_matches, scene_corners[3] + Point2f( object.cols, 0), scene_corners[0] + Point2f( object.cols, 0), Scalar( 0, 255, 0), 4 );
  
      }
  
      //Show detected matches
      imshow( "Good Matches", img_matches );
      for( int i = 0; i &lt; good_matches.size(); i++ )
      { 
          printf( "-- Good Match [%d] Keypoint 1: %d  -- Keypoint 2: %d  \n", i,    good_matches[i].queryIdx, good_matches[i].trainIdx ); 
      }
  
      waitKey(0);
  </code></pre>
  
  <p>}</p>
  
  <p>return 0;</p>
  
  <p>}</p></li>
  </ul>