title: How can i filter fast the huge csv file based on list of ids from another file using bash?
body: <p>I have 2 csv files.
  The first one is just a list of IDs (around 300k rows). Lest call it ids.csv</p>
  <pre><code>id
  1
  3
  7
  ...
  </code></pre>
  <p>The second one is list of objects with id (around 2 milions rows). Lets call it data.csv</p>
  <pre><code>id,date,state
  1,2022-01-01,true
  4,2022-01-03,false
  ...
  </code></pre>
  <p>I would like to build the 3rd csv file with rows from data.csv which having IDs from ids.csv. It is not necessary that all of IDs from ids.csv will be present in data.csv.</p>
  <p>I tried something like this:</p>
  <pre><code>while IFS=, read -r line
  do
      awk -F',' -v id='$line' '$1==id' data.csv &gt;&gt; out.csv
  done &lt; ids.csv
  </code></pre>
  <p>It is working, but executions takes something like forever. I tried to split it for several streams, but the script is working slow by itself so it was not helping mutch.</p>
  <p>Can you suggest me better or more optimal way how to filter that data.csv faster?</p>