How to deal with spelling errors NLP I have some data where the main column is the description of one product. The main task is to extract the name of some product from this column, where it sometimes is spelled wrong and amended in other words. I have more than a thousand possible product names.

Currently I'm just using regular expressions with a list of product names to find and extract the product names in each row of the dataset, but it's not working well in the cases of misspelled product names. 

Since I already have more than 50,000 rows in which the product was extracted by hand in cases where there is some product from the list in the column. I am wondering if after breaking each description into multiple lines (tokenization) could apply some classification/search method to detect if there is some product and which is in the description.


  Example: Medical product used in cancer treatment "PRODUCT XXXXYYYY"
  where the XXXX and YYYY are the first and second name of the product.


I think most of the description would be unused, because as in the example above, simply being a medical product would not help at all since there are several possibilities, but the presence of the wrongly spelled words that describe a product specific would be helpful. I am open to suggestions on how to deal with these spelling errors.
Other options would be to...

Compare similar text sequences
Compare similar string sequences
Use fuzzy matching

Fuzzy Matching:
library(fuzzyjoin)
# https://stackoverflow.com/questions/26405895/how-can-i-match-fuzzy-match-strings-from-two-datasets

a &lt;- data.frame(name = c('Ace Co', 'Bayes', 'asd', 'Bcy', 'Baes', 'Bays'),
                price = c(10, 13, 2, 1, 15, 1))
b &lt;- data.frame(name = c('Ace Co.', 'Bayes Inc.', 'asdf'),
                qty = c(9, 99, 10))

# Find matches
stringdist_join(a, b, 
                by = &quot;name&quot;,
                mode = &quot;left&quot;,
                ignore_case = FALSE, 
                method = &quot;jw&quot;, 
                max_dist = 99, 
                distance_col = &quot;dist&quot;
) %&gt;%
  group_by(name.x) %&gt;%
  top_n(1, -dist)

This gives you a &quot;distance&quot; between one and another word. So if you know the real product name, you might be able to find the erroneous names via the distance.
# A tibble: 6 x 5
# Groups:   name.x [6]
  name.x price name.y       qty   dist
  &lt;fct&gt;  &lt;dbl&gt; &lt;fct&gt;      &lt;dbl&gt;  &lt;dbl&gt;
1 Ace Co    10 Ace Co.        9 0.0476
2 Bayes     13 Bayes Inc.    99 0.167 
3 asd        2 asdf          10 0.0833
4 Bcy        1 Bayes Inc.    99 0.378 
5 Baes      15 Bayes Inc.    99 0.2   
6 Bays       1 Bayes Inc.    99 0.2  

Alternatively, you could look into similarity in string sequences:
If your product names are not to common from their string sequences and the erroneous names get only part of the name wrong, you could try something like:
library(dplyr)
library(tidytext)
library(fuzzyjoin)
library(tokenizers)

##############################
# Compare text sequences

text1=as.character(&quot;Hi my name is Bixi and I like cycling a lot. It is just great!&quot;)
mytext1=data_frame(text1)

text2=as.character(&quot;Hi my name is Lissi and I'm good in swimming. It is just great!&quot;)
mytext2=data_frame(text2)

ngram1 = unnest_tokens(mytext1, ngram, text1, token = &quot;ngrams&quot;, n = 4)
ngram2 = unnest_tokens(mytext2, ngram, text2, token = &quot;ngrams&quot;, n = 4)

# Find matching sequence(s)
semi_join(ngram1,ngram2)

##############################
# Compare sequences of single letters

ngram3=tokenize_character_shingles(mytext1$text1, n = 10, n_min = 10, strip_non_alphanum = FALSE)
ngram4=tokenize_character_shingles(mytext2$text2, n = 10, n_min = 10, strip_non_alphanum = FALSE)

ngram3=as.data.frame(ngram3)
ngram4=as.data.frame(ngram4)

# Find matching sequences of single letters
semi_join(ngram3,ngram4)

You may have a look at my Github. There are some more related options/solutions.