title: Efficiently read CSV file into array of doubles in C
body: <p>Looking for a more efficient method of reading a CSV file into a 2D array of doubles. My current code works okay but takes 2 minutes to read into the array.</p>
  
  <p>The CSV file being read is as follows. The entire file contains 28,325,381 lines and is 1.8GB.</p>
  
  <pre><code>CoordinateX,CoordinateY,CoordinateZ,Pressure,Temperature,VelocityX,VelocityY,VelocityZ,
  0,0,0.0904,33.5797,300,-0.00146382,0.000389435,-0.00147085,
  0,0.0003,0.0904,33.5795,300,0.126682,-0.000382509,0.00330599,
  0,0.0006,0.0904,33.5793,300,0.250278,-0.00151828,0.0100881,
  0,0.0009,0.0904,33.5788,300,0.365407,-0.00287706,0.0184123,
  ...
  </code></pre>
  
  <p>The original code...</p>
  
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;string.h&gt;
  
  int main() {
  
      /* Declare file and character pointers */
      FILE *pFile_r = NULL;
      FILE *pFile_w = NULL;
      char *pCell;
      int n = 0, p = 0, number_properties = 8;
  
      /* Open CSV file */
      char buffer[128];
      pFile_r = fopen("G:\\csv\\coarse (19000 - 39000)\\39500.csv","r");
  
      /* Declare array */
      int array_total_nodes = 28325340; //number of nodes in array
      double **properties_array;
      properties_array = malloc(array_total_nodes * sizeof(double*));
      for (n=0; n&lt;array_total_nodes; n++) {
          properties_array[n] = malloc(number_properties * sizeof(double));
      }
  
      /* Fill array */
      rewind(pFile_r);
      fgets(buffer, 128, pFile_r);
      while (!feof(pFile_r)) {
          pCell = strtok(buffer, ",");
          for (p=0; p&lt;number_properties; p++) {
              properties_array[n][p] = strtod(pCell, &amp;pCell);
              pCell = strtok(NULL, ",");
          }
          fgets(buffer, 128, pFile_r);
          n++;
      }
  
      return 0;
  }
  </code></pre>
  
  <p>This method is something simple I came up with. I'm hoping there is a more clever way that's faster. Your help is appreciated.</p>
  
  <h2>Update</h2>
  
  <p>I have switched for fgets to fscanf as such...</p>
  
  <pre><code>while (!feof(pFile_r)) {
          fscanf(pFile_r, "%lf,%lf,%lf,%lf,%lf,%lf,%lf,%lf,", &amp;properties_array[n][0], &amp;properties_array[n][1], &amp;properties_array[n][2], &amp;properties_array[n][3], 
                                     &amp;properties_array[n][4], &amp;properties_array[n][5], &amp;properties_array[n][6], &amp;properties_array[n][7]);
              n++;
          }
  </code></pre>
  
  <p>Reading in now takes 70 seconds as opposed to 120 (a 40% decrease). I would like to improve upon this further if possible. </p>