draw from an unsupervised model should be taken with grain of saltand we recommend verifying your intuition by looking at the documents in specific topic the topics produced by the lda transform method can also sometimes be used as compact representation for supervised learning this is particularly helpful when few training examples are available summary and outlook in this we talked about the basics of processing textalso known as natural language processing (nlp)with an example application classifying movie reviews the tools discussed here should serve as great starting point when trying to process text data in particular for text classification tasks such as spam and fraud detection or sentiment analysisbag-of-words representations provide simple and powerful solution as is often the case in machine learningthe representation of the data is key in nlp applicationsand inspecting the tokens and -grams that are extracted can give powerful insights into the modeling process in text-processing applicationsit is often possible to introspect models in meaningful wayas we saw in this for both supervised and unsupervised tasks you should take full advantage of this ability when using nlp-based methods in practice natural language and text processing is large research fieldand discussing the details of advanced methods is far beyond the scope of this book if you want to learn morewe recommend the 'reilly book natural language processing with python by steven birdewan kleinand edward loperwhich provides an overview of nlp together with an introduction to the nltk python package for nlp another great and more conceptual book is the standard reference introduction to information retrieval by christopher manningprabhakar raghavanand hinrich schutzewhich describes fundamental algorithms in information retrievalnlpand machine learning both books have online versions that can be accessed free of charge as we discussed earlierthe classes countvectorizer and tfidfvectorizer only implement relatively simple text-processing methods for more advanced text-processing methodswe recommend the python packages spacy ( relatively new but very efficient and welldesigned package)nltk ( very well-established and complete but somewhat dated library)and gensim (an nlp package with an emphasis on topic modelingthere have been several very exciting new developments in text processing in recent yearswhich are outside of the scope of this book and relate to neural networks the first is the use of continuous vector representationsalso known as word vectors or distributed word representationsas implemented in the word vec library the original paper "distributed representations of words and phrases and their compositionalityby thomas mikolov et al is great introduction to the subject both spacy summary and outlook