In this post, I will introduce you to topic modeling in Python (or) topic identification, which you can apply to any text you encounter in the wild.
Using basic NLP(Natural Language Processing) models, we will identify topics from texts based on term frequencies.
We will learn a simple method – bag-of-words and then use pre-processing techniques like Lemmatization to find the topic of a Wikipedia article.
Bag-of-words is a basic method for finding topics in a text.
First, we need to create tokens using tokenization and then count up all the tokens.
The more frequent a word, the more important it might be.Can be a great way to determine the significant words in a text.
You might get the error “SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)” while importing word_tokenize and Counter.
1.Go to /Applications/Python 3.7 folder and run Install Certificates.cmd.This should resolve the SSL error.
2.Open Python Shell and run the commands I’ve given below.
Let’s build another bag-of-words counter using a Wikipedia article.
We will try doing the bag-of-words without looking at the full article text, and guessing what the topic is! If you’d like to peek at the title at the end, look at the title of the Wikipedia link here.
Note that this article text has had very little preprocessing from the raw Wikipedia database entry.
#Use the .most_common() method of bow_simple to print the 10 most common tokens.
As you can see, we could not find the topic of the article by using the Counter and tokenization.
Let’s do some preprocessing on the article and clean up to remove stop words like ‘the’, punctuation characters like quotes etc.
Preprocessing helps make for better input data, useful when performing machine learning or other statistical methods.
Tokenization to create a bag of words.
Shorten words to their root stems.
Removing stop words, punctuation, or unwanted tokens.
Good to experiment with different approaches.
Lets take the sample text , that we took for generating the bag-of-words, and apply preprocessing techniques. Note the output now and compare it with the initial output.
In : text = """The cat is in the box. The cat likes the box. The box is over the cat."""
Now, let’s apply the preprocessing techniques you’ve learned to help clean up text of the Wikipedia article for better NLP results.
We’ll need to remove stop words and non-alphabetic characters, lemmatize, and perform a new bag-of-words on cleaned text.
Let’s use the tokens created in the last Wikipedia parsing example: lower_tokens.
We will import the WordNetLemmatizer class from nltk.stem. Then, we will create a list called alpha_only that iterates through lower_tokens and retains only alphabetical characters. You can use the .isalpha() method to check for this.
Create another list called no_stops in which you remove all stop words, which are held in a list called english_stops.
Initialize a WordNetLemmatizer object called wordnet_lemmatizer and use its .lemmatize() method on the tokens in no_stops to create a new list called lemmatized.
Finally, create a new Counter called bow with the lemmatized words and show the 10 most common tokens.
See the output of this code, and the word with most occurences is debugging, and the Wikipedia article that we had was also on debugging !!
That’s it for now on topic modeling. We will learn about further advanced concepts of data science in my data science online course.
Feel free to comment on this article on topic modeling in the section below.