Making sure a block of code takes at least 5 seconds to execute I am working on a personal project where I analyse Big Data by using twitter statuses. Therefore I am trying to query the Twitter API as many times as I can.

The API has a limit of 180 queries per 15 minutes, so I figured I can do 1 query every 5 seconds (at the minimum) if I want to not exceed that limit.

In the code below I do this the old-fashioned way with System.currentTimeMillis():

while (true) {
    newStatuses.clear();

    long startTime = System.currentTimeMillis();
    // Querying the API 
    newStatuses.addAll(DbTools.TWITTER_FACTORY.search(_query).getTweets());
    long totalTime = System.currentTimeMillis() - startTime;

    allStatuses.addAll(newStatuses);
    Thread.sleep((totalTime &gt; 5000) ? 0 : 5000 - totalTime);
    System.out.println(System.currentTimeMillis());
}


Pretty much if the total querying time took more than 5 seconds then sleep() for 0 seconds, if it took less than or equal 5 seconds then sleep() for the time that is remaining.

What I basically want is the time period from between hitting the line newStatuses.addAll(DbTools.TWITTER_FACTORY.search(_query).getTweets()); each time to be at least 5 seconds.

Is there any better way to do this or do you have any suggestions of how I can improve my solution?
Rather than waiting for 5 second between calls, consider that Twitter API returns three HTTP headers that explicitly tells you your quota:


  
  X-Rate-Limit-Limit: the rate limit ceiling for that given request
  X-Rate-Limit-Remaining: the number of requests left for the 15 minute window
  X-Rate-Limit-Reset: the remaining window before the rate limit resets in UTC epoch seconds
  


You can just query as much as you like, until the X-Rate-Limit-Remaining reaches zero or until you receive a 429 Too Many Request status code, then sleep for X-Rate-Limit-Reset.

This gives you several advantages:


You're utilising your limit to the fullest
If twitter changes their rate limit (e.g. someone is DDoSing them and they need to stricken it up or they make a significant upgrade that allows them to increase their quota), then your application will automatically adapt.
sleeping can't take into account of the per-user quota when you share access tokens for multiple applications/multiple purposes


Refer to Twitter documentation: https://dev.twitter.com/docs/rate-limiting/1.1