title: BeautifulSoup and Gadget Selector for scraping a table
body: <p>I am desperately trying to scrape this table: <a href="https://futures.huobi.com/en-us/linear_swap/info/realtime_fee/" rel="nofollow noreferrer">https://futures.huobi.com/en-us/linear_swap/info/realtime_fee/</a>, but unfortunately by running the following code:</p>
  <pre><code>from bs4 import BeautifulSoup
  import requests
  url = &quot;https://futures.huobi.com/en-us/linear_swap/info/realtime_fee/&quot;
  res = requests.get(url)
  soup = BeautifulSoup(res.text)
  soup
  </code></pre>
  <p>I don't see the table in soup.
  I believe it's because the data is not static, and is fetched with Javascript.</p>
  <p>What's a general solution for scraping this kind of tables?</p>