title: Unicorn + nginx - upstream prematurely closed connection while reading response header from upstream -504 Gateway Timeout
body: <p>I'm getting 504 Gateway Timeout error message in browser when I am trying to download the records in excel format. As I did the investigation about this, there was a timeout issue when the query execution is going long. Hence, I have changed the unicorn workers timeout value to 240 (even I tested with 1000+) and nginx proxy_*_timeout values from 300 to 600. But, again failed. </p>
  
  <p>My implementation is to export the user details as excel format but I'm calculating everything on the fly only. Four queries should be run for exporting single record. So, I'm trying 8000 records for testing. </p>
  
  <blockquote>
    <p>8000 (records) * 4 (queries) = 32000 queries</p>
  </blockquote>
  
  <p>If you have any other idea to export long run query result in excel, please let me know. thx.</p>
  
  <p>PFB for the configuration and error details.</p>
  
  <p><strong>unicorn.rb</strong></p>
  
  <pre><code>worker_processes 4    
  working_directory "/home/mrblack/myproject" # available in 0.94.0+    
  listen "/home/mrblack/myproject/tmp/.unicorn.sock", :backlog =&gt; 64    
  timeout 240
  
  pid "/home/mrblack/myproject/tmp/unicorn.pid"
  
  stderr_path "/home/mrblack/myproject/log/unicorn.stderr.log"
  stdout_path "/home/mrblack/myproject/log/unicorn.stdout.log"
  
  preload_app true
  GC.respond_to?(:copy_on_write_friendly=) and
    GC.copy_on_write_friendly = true 
  
  check_client_connection false
  
  before_fork do |server, worker|    
    defined?(ActiveRecord::Base) and
      ActiveRecord::Base.connection.disconnect!
  end
  
  after_fork do |server, worker|
  
    defined?(ActiveRecord::Base) and
      ActiveRecord::Base.establish_connection
  end
  </code></pre>
  
  <p><strong>nginx-conf</strong></p>
  
  <pre><code>user              ngnix;
  worker_processes  1;    
  error_log  /var/log/nginx/error.log;    
  pid        /var/run/nginx.pid;    
  timer_resolution  500ms;
  
  worker_rlimit_nofile 10240;
  events {
      use epoll;
      worker_connections 10240;
  }
  
  http {
      add_header X-Frame-Options SAMEORIGIN;
      include       /etc/nginx/mime.types;
      default_type  application/octet-stream;
  
      server_name_in_redirect off;
      server_tokens           off;
  
      log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
  
      access_log  /var/log/nginx/access.log  main;
  
      sendfile       on;
      tcp_nopush     on;
  
      keepalive_timeout  70s;
      tcp_nodelay        off;
  
      client_body_timeout   60s;
      client_header_timeout 60s;
  
      client_header_buffer_size 128;
      client_max_body_size      8m;
  
      open_file_cache           max=1000 inactive=20s;
      open_file_cache_valid     30s;
      open_file_cache_min_uses  2;
      open_file_cache_errors    on;
  
      proxy_cache_path  /var/cache/nginx/  levels=1:2   keys_zone=cache:80m inactive=1d  max_size=2500m;
      proxy_cache_key   "$scheme$request_method$host$request_uri";
      proxy_cache       cache;
  
  #    proxy_temp_path   /var/lib/nginx/proxy;
  
      proxy_cache_valid  200 302  1d;
      proxy_cache_valid  301      1d;
      proxy_cache_valid  any      1m;
  
      #proxy_ignore_headers  "X-Accel-Redirect" "X-Accel-Expires" "Expires" "Cache-Control" "Set-Cookie";
  
      proxy_connect_timeout 600s;
      proxy_send_timeout    600s;
      proxy_read_timeout    600s;
      send_timeout          600s;
  
      proxy_buffer_size           32k;
      proxy_buffers               4 32k;
      proxy_busy_buffers_size     32k;
      proxy_temp_file_write_size  32k;
  
      gzip              on;
      gzip_http_version 1.0;
      gzip_disable      "MSIE [1-6]\.(?!.*SV1)";
  
      gzip_buffers    4 16k;
      gzip_comp_level 2;
      gzip_min_length 0;
  
      gzip_types    text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
      gzip_proxied  expired no-cache no-store private auth;
  
      # Load config files from the /etc/nginx/conf.d directory
      # The default server is in conf.d/default.conf
      include /etc/nginx/conf.d/*.conf;
  
      upstream app_server {
        server unix: /home/mrblack/myproject/tmp/.unicorn.sock;
      }
  
      server {
          listen 80;
          add_header X-Frame-Options "DENY";
          client_max_body_size 4G;
          server_name localhost;
          keepalive_timeout 5;
          root /home/mrblack/myproject/public;
          try_files $uri/index.html $uri.html $uri @app;
          location @app {
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header Host $http_host;
              proxy_redirect off;
              proxy_pass http://app_server;
          }
  
          error_page 500 502 503 504 /500.html;
          location = /500.html {
              root /home/mrblack/myproject/public;
          }
      }
  
  }
  </code></pre>
  
  <p><strong>unicorn-stderr</strong></p>
  
  <pre><code>I, [2014-08-04T15:11:47.966332 #30951]  INFO -- : listening on addr=/home/mrblack/myproject/tmp/.unicorn.sock fd=10
  I, [2014-08-04T15:11:47.992843 #30970]  INFO -- : worker=0 ready
  I, [2014-08-04T15:11:47.998725 #30951]  INFO -- : master process ready
  I, [2014-08-04T15:11:48.002149 #30973]  INFO -- : worker=1 ready
  I, [2014-08-04T15:11:48.013117 #30976]  INFO -- : worker=2 ready
  I, [2014-08-04T15:11:48.015180 #30979]  INFO -- : worker=3 ready
  E, [2014-08-04T15:17:16.262433 #30951] ERROR -- : worker=0 PID:30970 timeout (61s &gt; 60s), killing
  E, [2014-08-04T15:17:16.363226 #30951] ERROR -- : reaped #&lt;Process::Status: pid 30970 SIGKILL (signal 9)&gt; worker=0
  I, [2014-08-04T15:17:16.384091 #31476]  INFO -- : worker=0 ready
  </code></pre>
  
  <p><strong>ngnix-error</strong></p>
  
  <blockquote>
    <p>upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "GET /users/download_excel.xls?sEcho=1&amp;iColumns=7&amp;sColumns=c0%2Cc1%2C%2C%2C%2C%2C&amp;iDisplayStart=0&amp;iDisplayLength=10&amp;mDataProp_0=0&amp;mDataProp_1=1&amp;mDataProp_2=2&amp;mDataProp_3=3&amp;mDataProp_4=4&amp;mDataProp_5=5&amp;mDataProp_6=6&amp;sSearch=&amp;bRegex=false&amp;sSearch_0=&amp;bRegex_0=false&amp;bSearchable_0=true&amp;sSearch_1=&amp;bRegex_1=false&amp;bSearchable_1=true&amp;sSearch_2=&amp;bRegex_2=false&amp;bSearchable_2=true&amp;sSearch_3=&amp;bRegex_3=false&amp;bSearchable_3=true&amp;sSearch_4=&amp;bRegex_4=false&amp;bSearchable_4=true&amp;sSearch_5=&amp;bRegex_5=false&amp;bSearchable_5=true&amp;sSearch_6=&amp;bRegex_6=false&amp;bSearchable_6=true&amp;iSortCol_0=1&amp;sSortDir_0=asc&amp;iSortingCols=1&amp;bSortable_0=true&amp;bSortable_1=true&amp;bSortable_2=true&amp;bSortable_3=false&amp;bSortable_4=false&amp;bSortable_5=true&amp;bSortable_6=false HTTP/1.1", upstream: "/home/mrblack/myproject/tmp/.unicorn.sock:/users/download_excel.xls?sEcho=1&amp;iColumns=7&amp;sColumns=c0%2Cc1%2C%2C%2C%2C%2C&amp;iDisplayStart=0&amp;iDisplayLength=10&amp;mDataProp_0=0&amp;mDataProp_1=1&amp;mDataProp_2=2&amp;mDataProp_3=3&amp;mDataProp_4=4&amp;mDataProp_5=5&amp;mDataProp_6=6&amp;sSearch=&amp;bRegex=false&amp;sSearch_0=&amp;bRegex_0=false&amp;bSearchable_0=true&amp;sSearch_1=&amp;bRegex_1=false&amp;bSearchable_1=true&amp;sSearch_2=&amp;bRegex_2=false&amp;bSearchable_2=true&amp;sSearch_3=&amp;bRegex_3=false&amp;bSearchable_3=true&amp;sSearch_4=&amp;bRegex_4=false&amp;bSearchable_4=true&amp;sSearch_5=&amp;bRegex_5=false&amp;bSearchable_5=true&amp;sSearch_6=&amp;bRegex_6=false&amp;bSearchable_6=true&amp;iSortCol_0=1&amp;sSortDir_0=asc&amp;iSortingCols=1&amp;bSortable_0=true&amp;bSortable_1=true&amp;bSortable_2=true&amp;bSortable_3=false&amp;bSortable_4=false&amp;bSortable_5=true&amp;bSortable_6=false", host: "localhost", referrer: "localhost/users/members"</p>
  </blockquote>