KMeans clustering for Image Data I am trying to cluster the sample of Imagenet Dataset using K-Means clustering.

In this approach, I have used the below 2 approaches to get the optimal number of clusters.


Elbow method   From the Graph it seems like the best number of clusters is from 6 to 10. 
Silhouette score  


Cluster : 2 | Silhouette score : 0.036273542791604996
Cluster : 3 | Silhouette score : -0.00300691369920969
Cluster : 4 | Silhouette score : 0.0025101888459175825
Cluster : 5 | Silhouette score : -0.005924953147768974
Cluster : 6 | Silhouette score : -0.00808520708233118
Cluster : 7 | Silhouette score : -0.006091121584177017
Cluster : 8 | Silhouette score : -0.00549863139167428
Cluster : 9 | Silhouette score : -0.014739749021828175
Cluster : 10 | Silhouette score : -0.021131910383701324
Cluster : 11 | Silhouette score : -0.04057755321264267
Cluster : 12 | Silhouette score : -0.012825582176446915
Cluster : 13 | Silhouette score : -0.012340431101620197
Cluster : 14 | Silhouette score : -0.032936643809080124
Cluster : 15 | Silhouette score : -0.04154697805643082
Cluster : 16 | Silhouette score : -0.04323640838265419


Where as in Silhouette analysis it looks like only for cluster 4 it is showing better value. Rest of the clusters are seems like samples are wrongly assigned to wrong clusters.

In such cases, which metric needs to be considered?

Updates: 
I have reduced the dimension of the features using PCA. Below are the updated graphs for Elbow and Silhouette analysis. However I do not see any improvement in the clustering. As per Silhouette the samples are not being assigned to the closest cluster. 

Elbow 
 

Silhouette 


Updating the question with latest graphs, VGGNet has been replaced by Resnet50 



Code snippet: 

        for i in range(lower_range, upper_range):
            # For Elbow curve
            kmeans_cluster = KMeans(n_clusters = i)
            kmeans_cluster_fit = kmeans_cluster.fit(features_list_np)
            loss = kmeans_cluster_fit.inertia_
            error.append(kmeans_cluster_fit.inertia_)

            # For Silhoutte analysis
            preds = kmeans_cluster.fit_predict(features_list_np)
            score = silhouette_score(features_list_np, preds)
            score_list.append(score)

            sample_score = silhouette_samples(features_list_np, preds)
            sample_score_list.append(sample_score)

            print("Cluster : {} | Loss : {} | Silhoutte Score : {}".format(i, loss, score))

            zero_samples = 0
            positive_samples = 0
            negative_samples = 0

            for each_sample in sample_score:
                if each_sample == 0:
                    zero_samples += 1
                if each_sample &gt; 0:
                    positive_samples += 1
                if each_sample &lt; 0:
                    negative_samples += 1

            print("Cluster : {} | Silhouette sample distribution - Zero : {} | Positive : {} | Negative : {}".format(i, zero_samples, positive_samples, negative_samples))

            zero_list.append(zero_samples)
            pos_list.append(positive_samples)
            neg_list.append(negative_samples)


Also should the importance should be given to Silhouette Score or the Silhouette Samples. Because in some of the cases, Silhouette score is less but the number of samples having positive values are more. 

Thank you,  
KK
Neither, there is not enough discriminatory information in data (yet)

Dont squeeze the data until it tells you the truth. You can change the metric (malahobian distance for example) and the algo but you cant expect it to show miracles. 

Using elbow method, as you increase the number of clusters it will always become more homogenous. You dont have a "kink" indicating optimal clustering number.

And with Silhouete (be careful those are averaged Silhoute score per cluster, in other words for k=4 you have 4 scores indicating wether points are lying inside/border/should- be-other-cluster that are averaged) you get that all of the points (in average) lie on the border of the clusters, without clear distinction of the clusters (thats what 0 means).

Advice Find better quantitative representation of data, new feature, reduce noise etc...