title: Fixing error 408 on first POST request when scraping data
body: <p>I'm trying to scrape website with BS4. This is the website that I have:</p>
  <p><a href="https://www.wsl.ch/de/ueber-die-wsl/news/alle-news.html" rel="nofollow noreferrer">https://www.wsl.ch/de/ueber-die-wsl/news/alle-news.html</a></p>
  <p>I want to scrape all urls of the news articles that are on this page. If I just put url in request lib, I will not get URLs of the website. But If i go to inspect page -&gt; network, there is one post request that returns HTML that has all urls ( href-s ).
  I have to use post request In order to get all URLs on the website, but the problem is that Im always getting error 408.</p>
  <pre><code>url = 'https://www.wsl.ch/de/ueber-die-wsl/news/alle-news.filter.html?tx_wslfilter_filter%5Baction%5D=ajax&amp;tx_wslfilter_filter%5Bcontroller%5D=Filter&amp;cHash=88a50dfb12c7c7e03ce68f244dbfda20'
  
  headers = {
              'Accept-Encoding': 'gzip, deflate, br',
              'Accept-Language': 'en-GB,en-US;q=0.9,en;q=0.8',
              'Connection': 'keep-alive',
              'Content-Length': '757',
              'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
              'Host': 'www.wsl.ch',
              'Origin': 'https://www.wsl.ch',
              'Referer': 'https://www.wsl.ch/de/ueber-die-wsl/news/alle-news.html',
              'Sec-Fetch-Dest': 'empty',
              'Sec-Fetch-Mode': 'cors',
              'Sec-Fetch-Site': 'same-origin',
              'Server-Timing': 'miss, db;dur=63, app;dur=55.2'}
  
  
  response = requests.post(url, headers = headers)
  print(response)
  soup = BeautifulSoup(response.content, 'html.parser')
  print(soup)
  </code></pre>
  <p>I have tried with and without headers but its the same.
  What should I do?</p>