Sort out bad pictures of a dataset (k-means, clustering, sklearn)
I don't have the solution to your problem but here is a sanity check to perform prior to the final clustering, to check that the kind of features you extracted is suitable for your problem:

extract the histogram features for all the pictures in your dataset
compute the pairwise distances of all the pictures in your dataset using the histogram features (you can use sklearn.metrics.pairwise_distance)

np.argsort the raveled distances matrix to find the indices of the 20 top closest pairs of distinct pictures according to your features (you have to filter out the zero-valued diagonal elements of the distance matrix) and do the same to extract the top 20 most farest pairs of pictures based on your histogram features.
Visualize (for instance with plt.imshow) the pictures of top closest pairs and check that they are all pairs that you would expect to be very similar.
Visualize the pictures of the top farest pairs and check that they are all very dissimilar.
If one of those 2 checks fails, then it means that histogram of bag of SIFT words is not suitable to your task. Maybe you need to extract other kinds of features (e.g. HoG features) or reorganized the way your extract the cluster of SIFT descriptors, maybe using a pyramidal pooling structure to extract info on the global layout of the pictures at various scales.
I'm testing some things in image retrival and i was thinking about how to sort out bad pictures of a dataset. For e.g there are only pictures  of houses and in between there is a  picture of people and some of cars. So at the end i want to get only the houses.
At the Moment my approach looks like:

computing descriptors (Sift) of all pictures
clustering all descriptors with k-means
creating histograms of the pictures by computing the euclidean distance between the cluster centers and the descriptors of a picture
clustering the histograms again.

at this moment i have got a first sort (which isn't really good). Now my Idea is to take all pictures which are clustered to a center with len(center) > 1 and cluster them again and again. So the Result is that the pictures which are particular in a center will be sorted out. Maybe its enough to fit the result again to the same k-means without clustering again?!
the result isn't satisfying so maybe someone has got a good idea.
For Clustering etc. I'm using k-means of scikit learn.