title: Efficient string matching in Apache Spark
body: <p>Using an OCR tool I extracted texts from screenshots (about 1-5 sentences each). However, when manually verifying the extracted text, I noticed several errors that occur from time to time.</p>
  
  <p>Given the text "Hello there ! I really like Spark ❤️!", I noticed that:</p>
  
  <p>1) Letters like "I", "!", and "l" get replaced by "|".</p>
  
  <p>2) Emojis are not correctly extracted and replaced by other characters or are left out.</p>
  
  <p>3) Blank spaces are removed from time to time.</p>
  
  <p>As a result, I might end up with a string like this: "Hello there 7l | real|y like Spark!"</p>
  
  <p>Since I am trying to match these string against a dataset including the correct text (in thise case "Hello there ! I really like Spark ❤️!"), I am looking for an efficient way how to match the string in Spark.</p>
  
  <p>Can anyone suggest an efficient algorithm for Spark which allows me to compare the extract texts (~100.000) against my dataset (~100 million)?</p>