title: Scraping part of a Wikipedia Infobox
body: <p>I'm using Python 2.7, <code>requests</code> &amp; BeautifulSoup to scrape approximately 50 Wikipedia pages.  I've created a column in my dataframe that has partial URL's that relate to the name of each song (these have been verified previously and I'm getting response code 200 when testing against all of them).  </p>
  
  <p>My code loops through and appends these individual URL's to the main Wikipedia URL.  I've been able to get the heading of the page or other data, but what I really want is the Length of the song only (don't need everything else).  The song length is contained within an infobox (example here:  <a href="http://en.wikipedia.org/wiki/No_One_Knows" rel="nofollow">http://en.wikipedia.org/wiki/No_One_Knows</a>)  </p>
  
  <p>My code either drags through everything on the page or nothing at all.  I think that the main problem is the bit where I have underlined below (i.e. mt = ...) - I put different html tags in here but I either get nothing back or most of the page.  </p>
  
  <pre><code>xyz = df.lengthlink  
  #column in a dataframe containing partial strings to append to the main Wikipedia url
  
  def songlength():
      url = ('http://en.wikipedia.org/wiki/' + xyz)
      resp = requests.get(url)
      page = resp.content
      take = BeautifulSoup(page)
      mt = take.find_all(____________)
      sign = mt
      return xyz, sign
  
  for xyz in df.lengthlink:
      print songlength()
  </code></pre>
  
  <p>Edited to Add:
  Using Martijn's suggestion below worked for the single url (i.e. No_One_Knows) but not for my multiple links.  It threw up this random error.</p>
  
  <pre><code>InvalidSchema                             Traceback (most recent call last)
  &lt;ipython-input-166-b5a10522aa27&gt; in &lt;module&gt;()
        2 xyz = df.lengthlink 
        3 url = 'http://en.wikipedia.org/wiki/' + xyz
  ----&gt; 4 resp = requests.get(url, params={'action': 'raw'})
        5 page = resp.text
        6 
  
  C:\Python27\lib\site-packages\requests\api.pyc in get(url, **kwargs)
       63 
       64     kwargs.setdefault('allow_redirects', True)
  ---&gt; 65     return request('get', url, **kwargs)
       66 
       67 
  
  C:\Python27\lib\site-packages\requests\api.pyc in request(method, url,    **kwargs)
       47 
       48     session = sessions.Session()
  ---&gt; 49     response = session.request(method=method, url=url, **kwargs)
       50     # By explicitly closing the session, we avoid leaving sockets open which
       51     # can trigger a ResourceWarning in some cases, and look like a memory leak
  
  C:\Python27\lib\site-packages\requests\sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
      459         }
      460         send_kwargs.update(settings)
  --&gt; 461         resp = self.send(prep, **send_kwargs)
      462 
      463         return resp
  
  C:\Python27\lib\site-packages\requests\sessions.pyc in send(self, request, **kwargs)
      565 
      566         # Get the appropriate adapter to use
  --&gt; 567         adapter = self.get_adapter(url=request.url)
      568 
      569         # Start time (approximately) of the request
  
  C:\Python27\lib\site-packages\requests\sessions.pyc in get_adapter(self, url)
      644 
      645         # Nothing matches :-/
  --&gt; 646         raise InvalidSchema("No connection adapters were found for '%s'" % url)
      647 
      648     def close(self):
  
  InvalidSchema: No connection adapters were found for '1     http://en.wikipedia.org/wiki/Locked_Out_of_Heaven
   2     http://en.wikipedia.org/wiki/No_One_Knows
   3     http://en.wikipedia.org/wiki/Given_to_Fly
   4     http://en.wikipedia.org/wiki/Nothing_as_It_Seems  
  
  Name: lengthlink, Length: 50, dtype: object'
  </code></pre>