title: Amazon Redshift queries mysteriously dying
body: <p>Why is my Amazon Redshift query sometimes working, sometimes getting killed, and sometimes running out of memory?</p>
  
  <p>This is a simple query:</p>
  
  <pre><code>dev=# EXPLAIN SELECT row_number, browser_cookie, "timestamp", request_path,
                       status,     outcome,         duration,   referrer
         FROM annotated_apache_logs
        WHERE date = '2015-09-15';
                                           QUERY PLAN                                          
  ------------------------------------------------------------------------------------
   XN Seq Scan on annotated_apache_logs  (cost=0.00..114376.71 rows=9150137 width=207)
     Filter: (date = '2015-09-15'::date)
  </code></pre>
  
  <p>Pulling about 9 million rows:</p>
  
  <pre><code>dev=# SELECT count(*) FROM annotated_apache_logs WHERE date = '2015-09-15';
    count  
  ---------
   9150137
  (1 row)
  </code></pre>
  
  <p>And choking:</p>
  
  <pre><code>dev=# SELECT row_number, browser_cookie, "timestamp", request_path,
               status,     outcome,        duration,    referrer
          FROM annotated_apache_logs
         WHERE date = '2015-09-15';
  out of memory
  </code></pre>
  
  <p>Sometimes the sql says <code>Killed</code>. Sometimes it works. Sometimes I get <code>out of memory</code>. No idea why. The table looks like this (I've removed rows not in the above query):</p>
  
  <pre><code>CREATE TABLE IF NOT EXISTS annotated_apache_logs (
    row_number           double precision,
    browser_cookie       character varying(240),
    timestamp            integer,
    request_path         character varying(2500),
    status               character varying(12),
    outcome              character varying(128),
    duration             integer,
    referrer             character varying(2500)
  )
  DISTKEY (date)
  SORTKEY (browser_cookie);
  </code></pre>
  
  <p>And I've worked very hard to get all of those columns as small as I can to reduce memory usage. What do I look for now? If I read the <code>EXPLAIN</code> output correctly, this might return a couple of gigs of data. Not much data, no joins, nothing fancy. For a "petabyte scale data warehouse", that's trivial, so I'm assuming I'm missing something fundamental here.</p>