title: How to scrape the web table with multiple pages using R or Python
body: <p>I want to scrape a web to gather the data for studying data mining. This web data contains a big table with 43 pages. And it also hide some stocks at the most right hand side of the expand menu. </p>
  
  <p><img src="https://i.stack.imgur.com/bt9MB.jpg" alt="enter image description here"></p>
  
  <p>The web page is below.</p>
  
  <p><a href="http://data.10jqka.com.cn/market/longhu/yyb/" rel="nofollow noreferrer">http://data.10jqka.com.cn/market/longhu/yyb/</a></p>
  
  <pre><code>import bs4
  import requests
  
  
  url = r"http://data.10jqka.com.cn/market/longhu/yyb/"
  
  response = requests.get(url)
  if response.status_code == 200:
      content = response.content
  
  soup = bs4.BeautifulSoup(content)
  table_results = soup.findAll("table", {"class": "m_table"})
  for item in table_results:
      company_name = item.findAll("td", {"class": "tl"})[0].text.strip()
      detail = item.findAll("td", {"class": "tc"})[0].text.strip()
      c_rise = item.findAll("td", {"class": "c_rise"})[0].text.strip()
      c_fall = item.findAll("td", {"class": "c_fall"})[0].text.strip()
      cur = item.findAll("td", {"class": "cur"})[0].text.strip()
      lhb_stocklist = item.findAll("div", {"class": "lhb_stocklist"})[0].text.strip()
      print company_name, detail, c_rise, c_fall, lhb_stocklist
  </code></pre>