title: Web scraping when scrolling down is needed
body: <p>I want to scrape, e.g., the title of the first 200 questions under the web page <a href="https://www.quora.com/topic/Stack-Overflow-4/all_questions" rel="nofollow noreferrer">https://www.quora.com/topic/Stack-Overflow-4/all_questions</a>. And I tried the following code:</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  
  url = "https://www.quora.com/topic/Stack-Overflow-4/all_questions"
  print("url")
  print(url)
  r = requests.get(url) # HTTP request
  print("r")
  print(r)
  html_doc = r.text # Extracts the html
  print("html_doc")
  print(html_doc)
  soup = BeautifulSoup(html_doc, 'lxml') # Create a BeautifulSoup object
  print("soup")
  print(soup)
  </code></pre>
  
  <p>It gave me a text <a href="https://pastebin.com/9dSPzAyX" rel="nofollow noreferrer">https://pastebin.com/9dSPzAyX</a>. If we search <code>href='/</code>, we can see that the html does contain title of some questions. However, the problem is that the number is not enough; actually on the web page, a user needs to <strong>manually scroll down to trigger extra load</strong>.</p>
  
  <p>Does anyone know how I could mimic "scrolling down" by the program to load more content of the page?</p>