text mining and text analytics table list of all pos tags (continuedtag meaning tag meaning nns nounplural nnp proper nounsingular nnps proper nounplural pdt predeterminer pos possessive ending prp personal pronoun prppossessive pronoun rb adverb rbr adverbcomparative rbs adverbsuperlative rp particle sym symbol uh interjection vb verbbase form vbd verbpast tense vbg verbgerund or present participle vbn verbpast participle vbp verbnon- rd person singular present vbz verb rd person singular present wdt wh-determiner wp wh-pronoun wppossessive wh-pronoun wrb wh-adverb pos tagging is use case of sentence-tokenization rather than word-tokenization after the pos tagging is complete you can still proceed to word tokenizationbut pos tagger requires whole sentences combining pos tagging and lemmatization is likely to give cleaner data than using only stemmer for the sake of simplicity we'll stick to stemming in the case studybut consider this an opportunity to elaborate on the exercise we now know the most important things we'll use to do the data cleansing and manipulation (text miningfor our text analyticslet' add the decision tree classifier to our repertoire decision tree classifier the data analysis part of our case study will be kept simple as well we'll test naive bayes classifier and decision tree classifier as seen in the naive bayes classifier is called that because it considers each input variable to be independent of all the otherswhich is naiveespecially in text mining take the simple examples of "data science,"data analysis,or "game of thrones if we cut our data in unigrams we get the following separate variables (if we ignore stemming and such)"data,"science,"analysis,"game,"of,and "thrones obviously links will be lost this canin turnbe overcome by creating bigrams (data sciencedata analysisand trigrams (game of thronesthe decision tree classifierhoweverdoesn' consider the variables to be independent of one another and actively creates interaction variables and buckets an interaction