title: How to scrape stock data incorporating pagination next tag using python bs4?
body: <p>The code can not get the next page,  it only repeats in an infinite loop.  I am using the example from <a href="https://oxylabs.io/blog/pagination-in-web-scraping" rel="nofollow noreferrer">oxylabs</a></p>
  <p>Could you tell me what I'm doing wrong?  Thank you.</p>
  <pre><code>import requests
  from bs4 import BeautifulSoup as bs
  from urllib.parse import urljoin
  
  url = 'https://hnx.vn/en-gb/cophieu-etfs/chung-khoan-ny.html'
  
  while True:
      response = requests.get(url)
      soup = bs(response.content, &quot;lxml&quot;)
  
      symbols = soup.find_all('td', class_='STOCK_CODE' )
      for s in symbols:
          symbol = s.find('a').text
          print(symbol)
  
      next_page = soup.select_one('span', id = 'next')
      if next_page:
          next_url = next_page.get('href')
          url = urljoin(url, next_url)
      else:
          break
  
      print(url)
  </code></pre>