case studyclassifying reddit posts the following listing shows simple stemming algorithm called "snowball stemming these snowball stemmers can be language-specificso we'll use the english onehoweverit does support many languages listing the reddit data processing revised after data exploration stemmer nltk snowballstemmer("english"def wordstemmer(wordrow)stemmed [stemmer stem(wordfor word in wordrowreturn stemmed initializes stemmer from nltk library manual_stopwords [',',',')',',','(',' ',"' "," ' ",' ',"'ve",' ','#','','``',"' ","''",'!',' ',']','=','[',' ','&','%','*',',' ',' ',' ', ',' ',' ',' ',' ',' ',' ','--',"''",';','-',':'def data_processing(sql,manual_stopwords)#create pointer to the sqlite data execute(sqlrow[ and data {'wordmatrix':[],'all_words':[]row[ interwordmatrix [contain the interwordlist [title and text of the postrespectively we combine them into single text blob fetch data (reddit postsone wordrow tokenizer tokenize(row[ ]+"+row[ ]wordrow_lowercased lowercasearray(wordrowwordrow_nostopwords wordfilter(stopwords,wordrow_lowercasedtemporary word matrixwill become final word matrix after hapaxes removal wordrow_nostopwords wordfilter(manual_stopwords,wordrow_nostopwordswordrow_stemmed wordstemmer(wordrow_nostopwordsinterwordlist extend(wordrow_stemmedinterwordmatrix append(wordrow_stemmedget new topic row fetchone(wordfreqs nltk freqdist(interwordlists hapaxes wordfreqs hapaxes(for wordvector in interwordmatrixwordvector_nohapexes wordfilter(hapaxes,wordvectordata['wordmatrix'append(wordvector_nohapexesdata['all_words'extend(wordvector_nohapexesreturn data append correct word vector to final word matrix now we define our revised data preparation row fetchone(by one from sqlite database while row is not nonetokenizer nltk tokenize regexptokenizer( '\ +|[^\ \ ]+'remove manually added stop words from text blob loop through temporary word matrix stop words array defines terms to remove/ignore for subject in subredditsdata[subjectdata_processing(sql='''select topictitle,topictext,topiccategory from topics where topiccategory '''+"'"+subject+"'"manual_stopwords=manual_stopwordstemporary word list used to remove hapaxes later on make frequency distribution of all terms get list of hapaxes remove hapaxes in each word vector extend list of all terms with corrected word vector run new data processing function for both subreddits