x_train_lemma shape( x_train shape( as you can see from the outputlemmatization reduced the number of features from , (with the standard countvectorizer processingto , lemmatization can be seen as kind of regularizationas it conflates certain features thereforewe expect lemmatization to improve performance most when the dataset is small to illustrate how lemmatization can helpwe will use stratifiedshufflesplit for cross-validationusing only of the data as training data and the rest as test datain[ ]build grid search using only of the data as the training set from sklearn model_selection import stratifiedshufflesplit param_grid {' '[ ]cv stratifiedshufflesplit(n_iter= test_size= train_size= random_state= grid gridsearchcv(logisticregression()param_gridcv=cvperform grid search with standard countvectorizer grid fit(x_trainy_trainprint("best cross-validation score "(standard countvectorizer){ }format(grid best_score_)perform grid search with lemmatization grid fit(x_train_lemmay_trainprint("best cross-validation score "(lemmatization){ }format(grid best_score_)out[ ]best cross-validation score (standard countvectorizer) best cross-validation score (lemmatization) in this caselemmatization provided modest improvement in performance as with many of the different feature extraction techniquesthe result varies depending on the dataset lemmatization and stemming can sometimes help in building better (or at least more compactmodelsso we suggest you give these techniques try when trying to squeeze out the last bit of performance on particular task topic modeling and document clustering one particular technique that is often applied to text data is topic modelingwhich is an umbrella term describing the task of assigning each document to one or multiple topicsusually without supervision good example for this is news datawhich might be categorized into topics like "politics,"sports,"finance,and so on if each document is assigned single topicthis is the task of clustering the documentsas discussed in if each document can have more than one topicthe task topic modeling and document clustering