besides looking at various approaches and modelswe also focus on important aspects in the machine learning pipeline including text pre-processingnormalizationand in-depth analysis of modelsincluding model interpretation and topic models the key idea here is to understand how we tackle problem like sentiment analysis on unstructured textlearn various techniquesmodels and understand how to interpret the results this will enable you to use these methodologies in the future on your own datasets let' get startedproblem statement the main objective in this is to predict the sentiment for number of movie reviews obtained from the internet movie database (imdbthis dataset contains , movie reviews that have been pre-labeled with "positiveand "negativesentiment class labels based on the review content besides thisthere are additional movie reviews that are unlabeled the dataset can be obtained from edu/~amaas/data/sentiment/courtesy of stanford university and andrew maasraymond dalypeter phamdan huangandrew ngand christopher potts this dataset was also used in their famous paperlearning word vectors for sentiment analysis proceedings of the th annual meeting of the association for computational linguistics (acl they have datasets in the form of raw text as well as already processed bag of words formats we will only be using the raw labeled movie reviews for our analyses in this hence our task will be to predict the sentiment of , labeled movie reviews and use the remaining , reviews for training our supervised models we will still predict sentiments for only , reviews in case of unsupervised models to maintain consistency and enable ease of comparison setting up dependencies we will be using several python libraries and frameworks specific to text analyticsnlpand machine learning while most of them will be mentioned in each sectionyou need to make sure you have pandasnumpyscipyand scikit-learn installedwhich will be used for data processing and machine learning deep learning frameworks used in this include keras with the tensorflow backendbut you can also use theano as the backend if you choose to do so nlp libraries which will be used include spacynltkand gensim do remember to check that your installed nltk version is at least >otherwisethe toktoktokenizer class may not be present if you want to use lower nltk version for some reasonyou can use any other tokenizer like the default word_tokenize(based on the treebankwordtokenizer the version for gensim should be at least and for spacythe version used was we recommend using the latest version of spacy which was recently released (version xas this has fixed several bugs and added several improvements you also need to download the necessary dependencies and corpora for spacy and nltk in case you are installing them for the first time the following snippets should get this done for nltk you need to type the following code from python or ipython shell after installing nltk using either pip or conda import nltk nltk download('all'halt_on_error=falsefor spacyyou need to type the following code in unix shell\windows command promptto install the library (use pip install spacy if you don' want to use condaand also get the english model dependency conda config --add channels conda-forge conda install spacy python - spacy download en