title: Fastest way to crawl Twitter data with Python - Followers' followers
body: <p>I'm doing a research on user social relations in Twitter, in Python.</p>
  
  <p>The problem is that "what is the fastest way to crawl followers of a certain user's followers information"
  I searched a lot of information and am currently using Tweepy:</p>
  
  <pre><code>c = tweepy.Cursor(api.followers_ids, id=centre, count=5000).items()
      while True:
          try:
              followers_ids_list.append(c.next())
          except tweepy.TweepError:
              # hit rate limit, sleep for 15 minutes
              time.sleep(15 * 60 + 15)
              continue
  </code></pre>
  
  <p>and after that I am using the /users/lookup to find the User() object according to those ids gained before.</p>
  
  <p>However, this way is quite slow...I was wondering if there any fastest than what I am doing currently.</p>
  
  <p>Because I want to find the user relations, which means followers in depth 2 is not enough.</p>
  
  <p>Say, I have 100 followers, and those 100 followers have their own 200 followers, then the time needed for grabbing this social network (depth=3) would be:</p>
  
  <p>(1 + 100 + 100*200)/15calls * 15mins / 60mins = 335 hours = about 14 days!</p>
  
  <ul>
  <li><p>1 call: request my follower ids (100ids)</p></li>
  <li><p>100 calls: request 100 followers' followers ids (100*200ids)</p></li>
  <li><p>100*200 calls(at least): request 100*200(followers' followers) users's ids.</p></li>
  </ul>
  
  <p>What I can think about to be alternative is to crawl the twitter.com website without api (but, I figure, this way would make my IP or account banned from Twitter....)</p>