title: scrape only new link added after scrape the site
body: <p>I have a code that scrape all links, titles and sizes of products with certain keywords. After the first scrape is done i want the script check again and again if new item are added. I try while True: but it seems doesnt work because gives me the same data multiple time. The script is this:</p>
  
  <pre><code>import requests
  import csv
  from bs4 import BeautifulSoup
  import time
  
  headers = {"user-agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 
  10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 
  Safari/537.36"}
  keywords = ["nike", "air"]
  base_url = "https://www.julian-fashion.com"
  
  while True:
      for page in range(0,11):
          url = "https://www.julian-fashion.com/en-US/men/shoes/sneakerscurrPage={}".format(page)
          r = requests.get(url)
          soup = BeautifulSoup(r.content,"html.parser")
          all_links = soup.find_all("li", attrs={"class":"product in-stock"})
          for link in all_links:
              for s in keywords:
                  if s not in link.a["href"]:
                      found = False
                      break
                  else:
                      product = link.a["href"]
                      found = True
                      if found:
                          print("Product found.")
                          print(base_url+link.a["href"])
                          print(link.img["title"])
                          print(link.div.div.ul.text)
  </code></pre>