How to recognize an object from a small training set of images?
Few reasons i could think of are:


Size of your training set is very small. Larger training sets have always been the key for accuracy.
Each algorithm will have some drawback like SURF is not good at viewpoint change and illumination change. So if the test images vary from the training images in case of illumination and viewpoint, results on those data will be poor.I don't know how FLANN works.