• +91-9872993883
• +91-8283824812
• info@ris-ai.com

# Build Autocorrect in Python ¶

Have you ever thought about how the autocorrect features works in the keyboard of a smartphone? Almost every smartphone brand irrespective of its price provides an autocorrect feature in their keyboards today. So let’s understand how the autocorrect features works. In this article, I will take you through how to build autocorrect with Python.

With the context of Machine Learning, autocorrect is based on Natural Language Processing. As the name suggests it is programmed to correct spellings and errors while typing.

Here I am using text from a book as Dataset named as Data.txt and Build Autocorrect in Python

First we call all necessary libraries :

In :
import pandas as pd
import numpy as np
import textdistance
import re
from collections import Counter


Now we read the Data.txt file as f:

In :
words = []
with open('Data.txt', 'r') as f:
file_name_data=file_name_data.lower()
words = re.findall('\w+',file_name_data)
# This is our vocabulary
V = set(words)
print(f"The first ten words in the text are: \n{words[0:10]}")
print(f"There are {len(V)} unique words in the vocabulary.")

The first ten words in the text are:
['the', 'project', 'gutenberg', 'ebook', 'of', 'moby', 'dick', 'or', 'the', 'whale']
There are 17647 unique words in the vocabulary.


In the above code, we made a list of words, and now we need to build the frequency of those words, which can be easily done by using the counter function in Python :

In :
word_freq_dict = {}
word_freq_dict = Counter(words)
print(word_freq_dict.most_common()[0:10])

[('the', 14703), ('of', 6742), ('and', 6517), ('a', 4799), ('to', 4707), ('in', 4238), ('that', 3081), ('it', 2534), ('his', 2530), ('i', 2120)]


## Relative Frequency of words¶

Now we want to get the probability of occurrence of each word, this equals the relative frequencies of the words:

In :
probs = {}
Total = sum(word_freq_dict.values())
for k in word_freq_dict.keys():
probs[k] = word_freq_dict[k]/Total


### Finding Similar Words ¶

Now we will sort similar words according to the Jaccard distance by calculating the 2 grams Q of the words. Next, we will return the 5 most similar words ordered by similarity and probability:

In :
def my_autocorrect(input_word):
input_word = input_word.lower()
if input_word in V:
return('Your word seems to be correct')
else:
similarities = [1-(textdistance.Jaccard(qval=2).distance(v,input_word)) for v in word_freq_dict.keys()]
df = pd.DataFrame.from_dict(probs, orient='index').reset_index()
df = df.rename(columns={'index':'Word', 0:'Prob'})
df['Similarity'] = similarities
return(output)


Let's check for similarity of word 'nevertheless' from the set of words :

In :
my_autocorrect('nevertheless')

Out:
'Your word seems to be correct'

Here now we check for the wrong spelled word 'nevrtless' and it return the 5 most similar words ordered by similarity and probability.

In :
my_autocorrect('nevrtless')

Out:
Word Prob Similarity
2571 nevertheless 0.000225 0.461538
10481 heartless 0.000018 0.454545
13600 nestle 0.000004 0.444444
16146 heartlessness 0.000004 0.428571
12513 subtleness 0.000004 0.416667
{"ft_lang_label":"__label__en","ft_lang_prob":0.72323465,"math_prob":0.74919534,"size":3115,"snap":"2023-40-2023-50","text_gpt3_token_len":826,"char_repetition_ratio":0.12375442,"word_repetition_ratio":0.041763343,"special_character_ratio":0.30593902,"punctuation_ratio":0.16809605,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9771912,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2023-12-03T23:33:16Z\",\"WARC-Record-ID\":\"<urn:uuid:f21d30ad-e73e-4381-bcfa-0f62f27fa1be>\",\"Content-Length\":\"99777\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:517705ba-11f9-4978-a188-49957ac75182>\",\"WARC-Concurrent-To\":\"<urn:uuid:91959dfb-f31b-416a-8459-8ba53405e71e>\",\"WARC-IP-Address\":\"68.66.226.100\",\"WARC-Target-URI\":\"http://ris-ai.com/build-an-autocorrect-in-python\",\"WARC-Payload-Digest\":\"sha1:5WQFFJENZB7KKGR7NJMUFSX57X72M4B5\",\"WARC-Block-Digest\":\"sha1:HYUHJBGVUUUS3YODNTDTYHJV3UFM6U3M\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2023/CC-MAIN-2023-50/CC-MAIN-2023-50_segments_1700679100518.73_warc_CC-MAIN-20231203225036-20231204015036-00487.warc.gz\"}"}