title: Limit varnish client-wait time, not back-end time
body: <p>I'm looking for a solution to guarantee client response time with varnish, without limiting backend response time.</p>
  
  <p>I have around 100 different resources (http://host/resource.js?id=1 etc.) which on average compute within a second on a back-end. The resources are cached by varnish so each of them can be served to many clients concurrently. The resources are included as synchronous (page blocking) javascript, so the responses should be served fast (e.g. 3 seconds). Because I would like to guarantee client response time, I could not think of a better solution than to configure the back-end timeout on this 3 seconds. An example vcl looks like:</p>
  
  <pre><code>backend mybackend {
    .host = "127.0.0.1";
    .port = "8080";
    .connect_timeout = 100ms;
    .first_byte_timeout = 3s;
    .between_bytes_timeout = 3s;
    .probe = {
       .url = "/resource?id=1";
       .timeout = 3s;
       .window = 4;
       .threshold = 4;
       .interval = 15s;
     }
  }
  
  sub vcl_recv {
    set req.backend = mybackend;
    set req.grace = 5d;
    return (lookup);
  }
  
  sub vcl_fetch {
    set obj.ttl = 2m;
    set obj.grace = 5d;
    return (deliver);  
  }
  </code></pre>
  
  <p>My problem is the following. After I've stopped the back-end for 5 minutes and restart it (while varnish serves stale data within the grace period), many different resources (out of TTL but within grace) are fetched concurrently at the back-end. This hits the database hard and none of the resources is delivered within 3 seconds and nothing gets cached. </p>
  
  <p>How do I avoid this problem? I would like to guarantee a client response time but not limit back-end response time. A failure (dummy javascript) would be acceptable temporarily. Is there some way to spread the requests over time? (Stale data is preferred above errors).</p>
  
  <p>Thanks,
  Ivor</p>