title: Extracting external links from tweets in python
body: <p>I wrote this simple program to extract links from tweets for a certain user. I was able to extract the links that are inside the tweets, but it seems like all I am getting are links that are shortened with t.co as the domain. These links are leading to other tweets.</p>
  
  <p>The problem is that these links sometimes lead to other tweets. How do I get links from tweets and make sure that these links are for an external site, not twitter itself.</p>
  
  <p>I hope my question is clear because this is the best way I can describe it.</p>
  
  <p>Thanks</p>
  
  <pre><code>#!/usr/bin/env python
  # -*- coding: utf-8 -*-
  
  import sys
  import re
  
  #http://www.tweepy.org/
  import tweepy
  
  #Get your Twitter API credentials and enter them here
  consumer_key = ""
  consumer_secret = ""
  access_key = ""
  access_secret = ""
  
  #method to get a user's last  200 tweets
  def get_tweets(username):
  
          #http://tweepy.readthedocs.org/en/v3.1.0/getting_started.html#api
          auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
          auth.set_access_token(access_key, access_secret)
          api = tweepy.API(auth)
  
          #set count to however many tweets you want; twitter only allows 200 at once
          number_of_tweets = 200
  
          #get tweets
          tweets = api.user_timeline(screen_name = username,count = number_of_tweets)
  
          for tweet in tweets:
                  urls = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&amp;+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', tweet.text)
                  for url in urls:
                          print url
  
  
  #if we're running this as a script
  if __name__ == '__main__':
  
      #get tweets for username passed at command line
      if len(sys.argv) == 2:
          get_tweets(sys.argv[1])
      else:
          print "Error: enter one username"
  
      #alternative method: loop through multiple users
          # users = ['user1','user2']
  
          # for user in users:
  #       get_tweets(user)
  </code></pre>
  
  <p>Here is an output sample: (I could not post it because it has shortened links). Editor wouldn't allow me to.</p>