Database into training and testing images
Some of the possible reasons would be:


Object recognition accuracy is not really well-defined. Instead accuracies are for particular datasets. 
What you are working with is called cross-validation, where you split your own dataset into testing and training datasets and then use the trained model to make predictions on the test dataset.
Depending on how you do the cross-validation and the final model, the accuracy tests may vary. For example, if (even by a random process) your training set contains well-conforming images and the test dataset contains images which are not even closely conforming to your model, then the accuracy would be low.


As for a solution to your dilemma, you should probably consider looking at the dataset and sampling in such a way that this situation can be avoided. Alternatively you could use a more generalizable algorithm and check for overfitting of the data via cross-validation again.