their treatment is highly dependent on context and domain systematic treatment of these cases is beyond the scope of this book the final category of string data is freeform text data that consists of phrases or sentences examples include tweetschat logsand hotel reviewsas well as the collected works of shakespearethe content of wikipediaor the project gutenberg collection of , ebooks all of these collections contain information mostly as sentences composed of words for simplicity' sakelet' assume all our documents are in one languageenglish in the context of text analysisthe dataset is often called the corpusand each data pointrepresented as single textis called document these terms come from the information retrieval (irand natural language processing (nlpcommunitywhich both deal mostly in text data example applicationsentiment analysis of movie reviews as running example in this we will use dataset of movie reviews from the imdb (internet movie databasewebsite collected by stanford researcher andrew maas this dataset contains the text of the reviewstogether with label that indicates whether review is "positiveor "negative the imdb website itself contains ratings from to to simplify the modelingthis annotation is summarized as two-class classification dataset where reviews with score of or higher are labeled as positiveand the rest as negative we will leave the question of whether this is good representation of the data openand simply use the data as provided by andrew maas after unpacking the datathe dataset is provided as text files in two separate foldersone for the training data and one for the test data each of these in turn has two subfoldersone called pos and one called neg arguablythe content of websites linked to in tweets contains more information than the text of the tweets themselves most of what we will talk about in the rest of the also applies to other languages that use the roman alphabetand partially to other languages with word boundary delimiters chinesefor exampledoes not delimit word boundariesand has other challenges that make applying the techniques in this difficult the dataset is available at example applicationsentiment analysis of movie reviews