title: django and dataTables - speed up query and page display
body: <p>I have a page in my app that lists all our client records - about a thousand total. The query is straight forward (two queries, actually) and look like this (one for commercial and one for  residential):</p>
  
  <pre><code>all_comm = System.objects.all().filter(isRessy=False)
  all_ressy = System.objects.all().filter(isRessy=True)
  </code></pre>
  
  <p>In my template, I simply iterate over both queries displaying the information in a table. This code looks like this:</p>
  
  <pre><code>  &lt;table style="width:100%;" class="field_container dt full-border"&gt;
      &lt;thead&gt;
        &lt;tr&gt;
          &lt;th align=left width=200&gt;Owner&lt;/th&gt;
          &lt;th align=center width=100&gt;System ID&lt;/th&gt;
          &lt;th align=left&gt;System Address&lt;/th&gt;
          &lt;th align=center width=200&gt;Options&lt;/th&gt;
        &lt;/tr&gt;
      &lt;/thead&gt;
  
      &lt;tbody&gt;
  
        {% for System in all_ressy %}
        &lt;tr onclick="window.location.href='{% url cpm.systems.views.system_overview System.systemID %}'"&gt;
          &lt;td&gt;{{ System.billingContact.lastName }}, {{ System.billingContact.firstName }}&lt;/td&gt;
          &lt;td align=center&gt;{{ System.pk }}&lt;/td&gt;
          &lt;td&gt;{{ System.systemAddress }}, {{ System.systemCity }}&lt;/td&gt;
          &lt;td align=center&gt;
            &lt;a href="{% url cpm.work_orders.views.create_wo 'R' System.systemID %}?=next{% url cpm.systems.views.system_overview System.systemID %}"&gt;Create Service Call&lt;/a&gt;
          &lt;/td&gt;
        &lt;/tr&gt;
        {% endfor %}
      &lt;/tbody&gt;
    &lt;/table&gt;
  </code></pre>
  
  <p>This code is identical for showing all the commercial records. If you'll notice that I have a class of <code>dt</code> listed in the table. That sets the table to be a <code>dataTable</code> table. As such, the rows get nice highlighting, the columns can be sorted, and there's a search box at the top of the table. All nice stuff.</p>
  
  <p>The problem is that the page, as a whole, is a bit slow to load. It seems that half of the loading time is the raw displaying of data (fetching data then iterating over all the records generating the basic <code>HTML</code> table). The second half of the loading time (or at least a decent chunk of the time) looks to be devoted to converting the regular table into a <code>dataTable</code>. </p>
  
  <p>I'm wondering if there's anything I can do to speed this whole process up. I've tried using pagination on the <code>dataTable</code> but that seems to be useless since all the records are loaded anyway, just hidden across multiple <code>dataTable</code> pages. Real pagination for the whole page isn't really possible given the nature of the app. I feel like the queries aren't going to get any faster so there's got to be optimization or some trick to make this page load faster.</p>
  
  <p>Any thoughts?</p>
  
  <p>Thanks for the help</p>
  
  <p><em><strong>Edit</em></strong> I'm referring to this dataTable plugin: <a href="http://datatables.net/index" rel="nofollow">http://datatables.net/index</a></p>