title: Python + BeautifulSoup New York Times Web Article Scraping
body: <p>I am trying to extract the content of any NYTimes article and put it into a string to count certain words. All article contents are found in HTML 'p' tags. I am able to get paragraphs one by one (commented in the code) but I can't iterate over the variable paragraphs because I keep getting the following error:</p>
  <pre><code> ---------------------------------------------------------------------------
  TypeError                                 Traceback (most recent call last)
  &lt;ipython-input-52-ccc2f7cf5763&gt; in &lt;module&gt;()
       16 
       17 for i in paragraphs:
  ---&gt; 18     article = article + paragraphs[i].get_text()
       19 
       20 print(article)
  
  TypeError: list indices must be integers, not Tag
  </code></pre>
  <p>Here is the code:</p>
  <pre><code>import requests
  from bs4 import BeautifulSoup
  session = requests.Session()
  url = &quot;http://www.nytimes.com/2015/01/02/world/europe/turkey-police-thwart-attack-on-prime-ministers-office.html&quot;
  req = session.get(url)
  soup = BeautifulSoup(req.text)
  paragraphs = soup.find_all('p', class_='story-body-text story-content')
  
    #article = paragraphs[0].get_text()
    #article = article + paragraphs[1].get_text()
    #article = article + paragraphs[2].get_text()
    #article = article + paragraphs[3].get_text()
    #article = article + paragraphs[4].get_text()
    #article = article + paragraphs[5].get_text()
    #article = article + paragraphs[6].get_text()
  
  for i in paragraphs:
      article = article + paragraphs[i].get_text()
  
  print(article)
  </code></pre>