Does the length of the text data affects the score of clustering I'm learning about clustering with Pythons Scikit-Learn lib.
I have a list of sentences (strings) . Im wondering, does the length of the string affects the silhouette_score.

For example, I have sentences from 2 words to 35 words, and i have tried number of clusters from 2 to 60, and the biggest silhouette_score that I get is around 7. Can that affect the silhouette_score?
Is it better to filter my data, so that I can select sentences that are much closer by number of words, for example, to set number of words from 20-25, or 5-10?

This is how my code looks like:

list_of_comments = data

#cv = TfidfVectorizer(analyzer = 'word', max_features = 6500, lowercase=True, preprocessor=None, tokenizer=None, stop_words = 'english')
cv = CountVectorizer(analyzer = 'word', max_features = 8000, lowercase=True, preprocessor=None, tokenizer=None, stop_words = 'english')  

x = cv.fit_transform(list_of_comments)

my_list = []
list_of_clusters = []
for i in range(2,35):

    kmeans = KMeans(n_clusters = i, init = 'k-means++', random_state = 0)
    kmeans.fit(x)
    my_list.append(kmeans.inertia_)

    cluster_labels = kmeans.fit_predict(x)

    silhouette_avg = silhouette_score(x, cluster_labels)*100
    print(round(silhouette_avg,2))
    list_of_clusters.append(silhouette_avg)
Yes, the length is very likely to influence any kind of similarity score. In general:


String containing very few words will have their highest similarity scores with other short strings with which they share one or two words in common. However if the words they contain are not common at all, they will have a lot of zero similarity scores (possibly only zeros).
Long strings (i.e. with many words) usually have low similarity scores, simply because they are unlikely to have a high proportion of their words in common. However compared to short strings they rarely have no word in common so they are much less likely to have lots of zero similarity scores.


As a result strings of similar length may tend to cluster together, depending on the data.