The use of computer vision techniques in post-harvest processing of agricultural products has increased considerably in recent years due to their non-destructive and rapid monitoring abilities. Image processing, combined with pattern recognition, has been applied in fruit sorting and classification. In this study, a Bag-of-Feature (BoF) model is used for the classification of 20 sweet and bitter almond varieties. Harris, Harris Laplace, Hessian, Hessian Laplace and Maximally Stable Extremal Regions (MSER) keypoint detectors along with a Scale Invariant Feature Transform (SIFT) descriptor are used in the BoF model. The k-means clustering method is applied for building a codebook from keypoint descriptors. The performance of 3 classifiers, which were k-Nearest Neighbour (k-NN), linear and chi-square Support Vector Machine (L-SVM and Chi-SVM, respectively) were compared using classification results in the model. It was observed that the Chi-SVM classifier outperformed the k-NN and L-SVM classifiers. Using the BoF model, it was possible to detect and classify sweet and bitter varieties with high overall accuracy. (C) 2017 IAgrE. Published by Elsevier Ltd. All rights reserved.