normalizing the word count vectors we will have to extend dist_raw to calculate the vector distancenot on the raw vectors but on the normalized ones insteaddef dist_norm( ) _normalized /sp linalg norm( toarray() _normalized /sp linalg norm( toarray()delta _normalized _normalized return sp linalg norm(delta toarray()this leads to the following similarity measurement==post with dist= this is toy post about machine learning actuallyit contains not much interesting stuff ==post with dist= imaging databases provide storage capabilities ==post with dist= most imaging databases safe images permanently ==post with dist= imaging databases store data ==post with dist= imaging databases store data imaging databases store data imaging databases store data best post is with dist= this looks bit better now post and post are calculated as being equally similar one could argue whether that much repetition would be delight to the readerbut from the point of counting the words in the poststhis seems to be right removing less important words let us have another look at post of its words that are not in the new postwe have "most""safe""images"and "permanentlythey are actually quite different in the overall importance to the post words such as "mostappear very often in all sorts of different contextsand words such as this are called stop words they do not carry as much informationand thus should not be weighed as much as words such as "images"that don' occur often in different contexts the best option would be to remove all words that are so frequent that they do not help to distinguish between different texts these words are called stop words as this is such common step in text processingthere is simple parameter in countvectorizer to achieve thisas followsvectorizer countvectorizer(min_df= stop_words='english'