title: throttling http api calls with delay
body: <p>I'm trying to implement some throttles on our REST API. A <a href="http://django-tastypie.readthedocs.org/en/latest/throttling.html" rel="noreferrer">typical approach</a> is after a certain threshold to block the request (with <code>403</code> or <code>429</code> response). However, I've seen <a href="http://apidocs.mailchimp.com/api/faq/#faq6" rel="noreferrer">one api</a> that adds a <strong>delay to the response</strong> instead.</p>
  <blockquote>
  <p>As you make calls to the API, we will be looking at your average calls per second (c/s) over the previous five-minute period. Here's what will happen:</p>
  <ul>
  <li><p>over 3c/s and we add a 2 second delay</p>
  </li>
  <li><p>over 5c/s and we add a 4 second delay</p>
  </li>
  <li><p>over 7c/s and we add a 5 second delay</p>
  </li>
  </ul>
  </blockquote>
  <p>From the client's perspective, I see this being better than getting back an error. The worst that can happen is that you'll slow down.</p>
  <p>I am wondering how this can be achieved <strong>without negatively impacting the app server</strong>. i.e. To add those delays, the server needs to keep the request open, causing it to keep more and more request processors busy, meaning it has less capacity for new requests coming in.</p>
  <p><strong>What's the best way to accomplish this?</strong> (i.e. is this something that can be done on the web server / load balancer so that the application server is not negatively affected? Is there some kind of a throttling layer that can be added for this purpose?)</p>
  <p>We're using Django/Tastypie, but the question is more on the architecture/conceptual level.</p>