title: How to fuzzy search JSON in memory instead of MySQL
body: <p><code>project_row</code> table with up to 15k rows per <code>project_id</code>:</p>
  <pre><code>ID | source                       |  target     | project_id
  ----------------------
  0  | hello world                  | hola tierra | 2
  1  | hello                        | hola        | 2
  2  | what                         |             | 2
  3  | you                          | tu          | 2
  4  | nevermind. I will head home. |             | 2
  5  | There are many sentences     |             | 2
     | stored in the source column  |             | 2
     | as well!                     |             | 2
  ...
  </code></pre>
  <p><code>glossary_term</code> table with up to 40k rows per <code>glossary_id</code>:</p>
  <pre><code>ID | source                       |  target               | glossary_id
  ----------------------
  0  | hello world                  | hola tierra           | 2
  1  | hello word                   | hola palabra          | 2
  2  | what?                        | que?                  | 2
  3  | your                         | tu                    | 2
  4  | I will head home.            | Me voy a la casa      | 2
  5  | There are many sentences     | también hay muchas    | 2
     | stored in the source column  | oraciones almacenadas | 2
     | as well.                     | en la columna fuente  | 2
  ...
  </code></pre>
  <p>I have an 'analysis' feature that basically compiles a summary of the project's source columns. I am OK with it taking up to 5 minutes to complete, because the user only runs it around once a week. But, my current implementation will take HOURS.</p>
  <pre><code>SELECT * FROM project_row
  SELECT * FROM glossary_row WHERE glossary_id = 2
  
  $data = array();
  
  foreach($projectRow as $row){ //Up to 15k results
      //We need to find the best fuzzy match for this row's source in the glossary
      //I do this by looping each glossary term and running similar_text
  
      $highestMatchPercent = 0;
  
      $rowSrc = $row[&quot;source&quot;];
  
      foreach($glossaryTerm as $term){ //Up to 40k results
          $termSrc = $term[&quot;source&quot;];
  
          if(abs(strlen($rowSrc) - strlen($termSrc)) &gt; 10) continue; //Added efficiency. If length difference more than 10, don't bother.
  
          similar_text($rowSrc, $termSrc, $matchPercent);
          if($matchPercent &gt; $highestMatchPercent) $highestMatchPercent = $matchPercent;
  
          if($highestMatchPercent == 100) break; //Added efficiency
      }
  
      //Now we have the highest match percent for this row. This is the data that I need for every row.
  
      if($highestMatchPercent == 100){
          $data[&quot;exact&quot;]++;
      } else if($highestMatchPercent &gt; 75){
          $data[&quot;fuzzy&quot;]++;
      }else{
          $data[&quot;nothing&quot;]++;
      }
      
  }
  
  //Now I have the $data variable with a summary of what I need for this project...after many hours of running this process...
  
  </code></pre>
  <p>Of course, this will take ages to complete with 15k rows and 40k glossary terms. This double loop is very taxing. I have tried to figure out ways to limit the MySQL glossary_term results for each row, but I haven't found a good solution there.</p>
  <p>I've been told that you can fuzzy search a massive JSON file loaded in memory MUCH faster than selecting that same data from a database and filtering in PHP. But I have no idea how I would implement something like this. My PHP would somehow need to communicate with the flat DB and query results from it.</p>
  <p>Any ideas?</p>