title: AJAX - Large response locking up browser upon appending data
body: <p>I am working on an application that fetches many records from a database and renders them to a table. This is done with an AJAX call and appends them to the end of the results that are already there.</p>
  
  <p>The number of records is the variable in this case and could be 10 or 20,000 depending on what the search criteria returns.</p>
  
  <p>Now, first things first. I understand that having 20,000 records on the page isn't smart but this was a requirement to allow all the records be seen, modified, or marked up from this page.</p>
  
  <p>The AJAX call to the PHP page is fine and seems to work pretty quick. The bottle neck however is appending that data to the DOM once its received via AJAX.</p>
  
  <p>In the code below, you will see the main function that creates the table row and returns it back to the AJAX call.  </p>
  
  <pre><code>/*
    Create Table Rows
  */
  function createTableRows($dashboardID, $type){
  
    // Timer
    $start = microtime(true);
  
    // Dashboard
    $dashboardID = $_REQUEST['dashboardID'];
    $objDB = new DB;
    $objData = $objDB
      -&gt; setStoredProc('RenderDashboard')
      -&gt; setParam('dashboardID', $dashboardID)
      -&gt; setParam('limit', $_REQUEST['limit'])
      -&gt; setParam('offset', $_REQUEST['offset'])
      -&gt; setParam('actor', '1234')
      -&gt; execStoredProc()
      -&gt; parseXML();
  
      // Fetch other data
      $markupData = fetchMarkup($dashboardID);
      $exportFields = fetchExportFields($dashboardID);
      $ignore = Array('identifierQID', 'identifierNTID');
  
      // Vars
      $outputArray = Array();
      $recordCount = 0;
      $i = 0;
  
      // Loop over our data
      foreach($objData-&gt;data as $r){
  
        $outputArray[$i++] = '&lt;tr data-qid="'.$r-&gt;identifierQID.'" class="primaryValue ' . searchMarkup($markupData, $r-&gt;identifierQID) . '"&gt;';
  
          // Loop over our fields
          foreach($r as $key =&gt; $value){
  
            // Vars
            $fieldID = str_replace('_', '', $key);
  
            // Don't include our identifier columns
            if(!in_array($fieldID, $ignore)){
              $outputArray[$i++] = '&lt;td data-tableexport-display="always" class="small' . ($exportFields ? (in_array($fieldID, $exportFields) ? ' hidden' : '') : '') . '"&gt;' . formatFieldData($fieldID, $value) . '&lt;/td&gt;';
            }
  
          }
  
          // Notes always come last
          $outputArray[$i++] = '&lt;td data-tableexport-display="always" class="notesTD allowContext hidden"&gt;&lt;/td&gt;';
  
        $outputArray[$i++] = '&lt;/tr&gt;';
        $recordCount++;
  
      }
  
      // Join our rows array and return it
      $end = microtime(true);
      $timer = number_format($end - $start, 2);
      return array(join("",$outputArray), $recordCount, $timer);
  }
  
  // This is what gets passed back to our AJAX call on the UI
  echo createTableRows($dashboardID)[0];
  </code></pre>
  
  <p>Here is the Javascript that processes the response it receives back.</p>
  
  <pre><code>// Given data, create our table rows
  function createRows(data) {
  
      // Update Progress Bar
      $('[name=progressDiv]').show();
  
      // Append the results to the DOM
      /* THIS IS WHAT IS KILLING THE SPEED!!!!*/
      $('[name=resultsTable]').append(data);
  
      // If our total number of records exceeds the threshold, we will be using the progress bar for the status
      if (totalRecords &gt; maxThreshold) {
          $('[name=resultsProgress]').attr('aria-valuenow', currentPage / totalPages * 100)
              .css('width', currentPage / totalPages * 100 + '%')
              .text((currentPage &lt; totalPages ? recordsPerPage * currentPage + ' of ' + totalRecords + ' records loaded' : 'Loaded ' + totalRecords + ' records!'));
      } else {
          // Loaded all records in one shot, update progress bar
          $('[name=resultsProgress]').attr('aria-valuenow', 100)
              .css('width', '100%')
              .text('Loaded ' + totalRecords + ' records!')
              .removeClass('active');
      }
      // Do we have more data to load?
      if (currentPage &lt; totalPages &amp;&amp; totalRecords &gt; maxThreshold) {
        // Allow a little time for the progress to update before locking up
        setTimeout(function(){
          fetchMore();
        }, 100);
  
      }
  
      // After the table has been appended to the DOM, run clean up to enable any additional functionality
      cleanUp();
  }
  </code></pre>
  
  <p><strong>The Issue:</strong></p>
  
  <p>The problem is that the <code>APPEND</code> is locking up the browser and causing it to be unresponsive until the append has been completed. I already have this broken up so it will fetch the data in batches but that isn't the issue, its handling the rows in the response. </p>
  
  <p><strong>The Question:</strong></p>
  
  <p>Is there a way to process the results in batches and append that without it locking up the browser? The response it self is just a bunch of <code>TR</code>'s that are appended to the <code>TBODY</code> of my table.</p>
  
  <p>My last resort is having to page the results. If I can fix this bottle neck, I can convince them to do the paging for larger data sets.</p>
  
  <p>I guess I am looking for a way to either return the results in a better format for appending or break up the response and have it append in batches while another AJAX call is fetching more data to process.</p>
  
  <p>Thoughts?</p>