title: Scraping from multiple url's using beatifulsoup
body: <p>I have my code working. Now I would like to do a little bit of modification to get the date from multiple URLs, But URLs have only one word difference. </p>
  
  <p>Here is my code, I am fetching from only one URL.</p>
  
  <pre><code>from string import punctuation, whitespace
  import urllib2
  import datetime
  import re
  from bs4 import BeautifulSoup as Soup
  import csv
  today = datetime.date.today()
  html = urllib2.urlopen("http://www.99acres.com/property-in-velachery-chennai-south-ffid").read()
  
  soup = Soup(html)
  print "INSERT INTO `property` (`date`,`Url`,`Rooms`,`place`,`PId`,`Phonenumber1`,`Phonenumber2`,`Phonenumber3`,`Typeofperson`,` Nameofperson`,`typeofproperty`,`Sq.Ft`,`PerSq.Ft`,`AdDate`,`AdYear`)"
  print 'VALUES'
  re_digit = re.compile('(\d+)')
  properties = soup.findAll('a', title=re.compile('Bedroom'))
  
  for eachproperty in soup.findAll('div', {'class':'sT'}):
    a      = eachproperty.find('a', title=re.compile('Bedroom'))
    pdate  = eachproperty.find('i', {'class':'pdate'})
    pdates = re.sub('(\s{2,})', ' ', pdate.text)
    div    = eachproperty.find('div', {'class': 'sT_disc grey'})
    try:
      project = div.find('span').find('b').text.strip()
    except:
      project = 'NULL'        
    area = re.findall(re_digit, div.find('i', {'class': 'blk'}).text.strip())
    print ' ('
    print today,","+ (a['href'] if a else '`NULL`')+",", (a.string if a else 'NULL, NULL')+ "," +",".join(re.findall("'([a-zA-Z0-9,\s]*)'", (a['onclick'] if a else 'NULL, NULL, NULL, NULL, NULL, NULL')))+","+ ", ".join([project] + area),","+pdates+""
    print ' ), '
  </code></pre>
  
  <p>Here are the URLs that I would like to fetch from, at the same time</p>
  
  <pre><code>http://www.99acres.com/property-in-velachery-chennai-south-ffid
  http://www.99acres.com/property-in-thoraipakkam-chennai-south-ffid
  http://www.99acres.com/property-in-madipakkam-chennai-south-ffid
  </code></pre>
  
  <p>So you can see that there is only one word that differs in every URL.</p>
  
  <p>I am trying to create an array like the following</p>
  
  <pre><code>for locality in areas (http://www.99acres.com/property-in-velachery-chennai-south-ffid
  , http://www.99acres.com/property-in-thoraipakkam-chennai-south-ffid,    http://www.99acres.com/property-in-madipakkam-chennai-south-ffid):
  link = "str(locality)"
  html = urllib2.urlopen(link)
  soup = Soup(html)
  </code></pre>
  
  <p>This quite does not seem to work and I actually would like to just pass that ONE WORD to the URL like this</p>
  
  <pre><code>for locality in areas(madipakkam, thoraipakkam, velachery):
      link = “http://www.99acres.com/property-in-+ str(locality)+-chennai-south-ffid"
      html= urllib2.urlopen(link)
      soup = BeautifulSoup(html)
  </code></pre>
  
  <p>Hope I made it clear</p>