mental image of representing text as "bag computing the bag-of-words representation for corpus of documents consists of the following three steps tokenization split each document into the words that appear in it (called tokens)for example by splitting them on whitespace and punctuation vocabulary building collect vocabulary of all words that appear in any of the documentsand number them (sayin alphabetical order encoding for each documentcount how often each of the words in the vocabulary appear in this document there are some subtleties involved in step and step which we will discuss in more detail later in this for nowlet' look at how we can apply the bag-of-words processing using scikit-learn figure - illustrates the process on the string "this is how you get ants the output is one vector of word counts for each document for each word in the vocabularywe have count of how often it appears in each document that means our numeric representation has one feature for each unique word in the whole dataset note how the order of the words in the original string is completely irrelevant to the bag-of-words feature representation figure - bag-of-words processing working with text data