On a daily basis, I get a source csv file that has 250k rows and 40 columns. It’s 290MB. I will need to filter it because it has more rows than I need and more columns than I need.

For each row that meets the filtering criteria, we want to update that into the destination system 1 record at a time using its PHP API.

What will be the best approach for everything up until the API call (the reading / filtering / loading) for the fastest performance?

1. Iterating through each row of the file, deciding if it’s a row I want, grabbing only the columns I need, and then passing it to the API?
2. Loading ALL records into a temporary MySQL table using LOAD DATA INFILE. Then querying the table for the rows and fields I want, and iterating through the resultset passing each record to the API?

Is there a better option?

Thanks!