title: Python: urlopen not downloading the entire site
body: <p>Greetings,</p>
  
  <p>I have done:</p>
  
  <pre><code>import urllib
  
  site = urllib.urlopen('http://www.weather.com/weather/today/Temple+TX+76504')
  site_data = site.read()
  site.close()
  </code></pre>
  
  <p>but it doesn't compare to viewing the source when loaded in firefox.</p>
  
  <p>I suspected the user agent and did this:</p>
  
  <pre><code>class AppURLopener(urllib.FancyURLopener):
      version = "Mozilla/5.0 (X11; U; Linux i686; zh-CN; rv:1.9.2.8) Gecko/20100722 Ubuntu/10.04 (lucid) Firefox/3.6.8"
  
  urllib._urlopener = AppURLopener()
  </code></pre>
  
  <p>and downloaded it, but it still doesn't download the whole website.</p>
  
  <p>Can someone please help me do user agent switching, if that is the likely culprit?</p>
  
  <p>Thanks,
  Narnie</p>