title: Haproxy custom error pages only work sometimes
body: <p>I have Apache operating as a reverse proxy which redirects to a HaProxy server set up which load balances between two backend servers using cookies.  It's not elegant but everything works fine ...</p>
  
  <p>The backend servers run a web application which uses basic authentication (it's all it offers) and if users enter the wrong login details then the backend app times out.  As a result HaProxy shows a 504 error.</p>
  
  <p>To help my users log in properly I've put together custom error pages for the 504 error as well as 503 and 502 and these display <em>some</em> of the time.  Other times a standard error page is shown in the browser which bears no relation to the ones that I've defined and are nothing like the default ones that come with HaProxy (e.g. /etc/haproxy/errors/504.http etc).</p>
  
  <p>The error pages are defined in the global section as well as in each backend section.  They are small ~1800 bytes and are valid html</p>
  
  <p>It appears that the browsers read the HTTP response and some of the time display their own version of the page.  They then also cache the error page so that when the user tries to access the page again the error page is served up again instead of pulling the page and showing the basic authentication login box.  The only way to get the login box to reappear is to close the browser, reopen it and then navigate back to the login url.</p>
  
  <p>I'd forgotten about the Apache reverse proxy sat in front of my HaProxy servers and wondered if my backend server was sending a 503 error which was then getting caught by HaProxy and this was, in turn, getting caught by Apache.  To test this I changed my custom HaProxy error pages so that they return a 200 OK status to Apache.  It worked once, but then defaulted back to the standard message:</p>
  
  <pre><code>Proxy Error
  
  The proxy server received an invalid response from an upstream server.
  The proxy server could not handle the request
  
  Reason: Error reading from remote server
  </code></pre>
  
  <p>Each error file looks something like this:</p>
  
  <pre><code>HTTP/1.0 504 Gateway Time-out
  Cache-Control: no-cache
  Connection: close
  Content-Type: text/html
  
  &lt;html&gt;
  &lt;head&gt;
  &lt;title&gt;Username and/or password error&lt;/title&gt;
  &lt;/head&gt;
  &lt;body&gt;
  &lt;p&gt;Some explanatory text ...&lt;/p&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  
  <p>What is going on here ?  How do I get my custom error page to appear consistently ?  And for then not to be cached in the users browser ?</p>
  
  <p>I've reloaded and restarted haproxy having edited haproxy.cfg numerous times.  Here's my haproxy.cfg file:</p>
  
  <pre><code>global
      log /dev/log    local0
      log /dev/log    local1 notice
      chroot /var/lib/haproxy
      stats socket /run/haproxy/admin.sock mode 660 level admin
      stats timeout 30s
      user haproxy
      group haproxy
      daemon
  
      # Default SSL material locations
      ca-base /etc/ssl/certs
      crt-base /etc/ssl/private
  
      # Default ciphers to use on SSL-enabled listening sockets.
      # For more information, see ciphers(1SSL). This list is from:
      #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
      ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
      ssl-default-bind-options no-sslv3
  
  defaults
      log     global
      mode    http
      option  httplog
      option  dontlognull
      timeout connect 5000
      timeout client  30000
      timeout server  30000
      errorfile 400 /etc/haproxy/errors/400.http
      errorfile 403 /etc/haproxy/errors/403.http
      errorfile 408 /etc/haproxy/errors/408.http
      errorfile 500 /etc/haproxy/errors/500.http
      errorfile 502 /etc/haproxy/errors/502-custom.http
      errorfile 503 /etc/haproxy/errors/503-custom.http
      errorfile 504 /etc/haproxy/errors/504-custom.http
  
  frontend http_front
     bind *:8085
     stats uri /haproxy?stats
  
     acl is_S0 hdr_sub(cookie) S0
     acl is_S1 hdr_sub(cookie) S1
  
     use_backend backend_S0 if is_S0
     use_backend backend_S1 if is_S1
     default_backend backend_S0
  
  backend backend_S0
     server Public0 xxx.xxx.xxx.xxx:8080 check
     errorfile 502 /etc/haproxy/errors/502-custom.http
     errorfile 503 /etc/haproxy/errors/503-custom.http
     errorfile 504 /etc/haproxy/errors/504-custom.http
  
  
  backend backend_S1
     server Public1 yyy.yyy.yyy.yyy:8080 check
     errorfile 502 /etc/haproxy/errors/502-custom.http
     errorfile 503 /etc/haproxy/errors/503-custom.http
     errorfile 504 /etc/haproxy/errors/504-custom.http
  </code></pre>