I am attempting to scrape websites and I sometimes get this error and it is concerning as I randomly get this error but after i retry i do not get the error.

```
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='www.somewebsite.com', port=443): Read timed out. (read timeout=None)

```

My code looks like the following

```
from bs4 import BeautifulSoup
from random_user_agent.user_agent import UserAgent
from random_user_agent.params import SoftwareName, OperatingSystem
import requests

software_names = [SoftwareName.CHROME.value]
operating_systems = [OperatingSystem.WINDOWS.value, OperatingSystem.LINUX.value]
user_agent_rotator = UserAgent(software_names=software_names, operating_systems=operating_systems, limit=100)
pages_to_scrape = ['https://www.somewebsite1.com/page', 'https://www.somewebsite2.com/page242']

for page in pages_to_scrape:
  time.sleep(2)
  page = requests.get(page, headers={'User-Agent':user_agent_rotator.get_random_user_agent()})
  soup = BeautifulSoup(page.content, "html.parser")
  # scrape info 

```

As you can see from my code I even use Time to sleep my script for a couple of seconds before requesting another page. I also use a random user\_agent. I am not sure if i can do anything else to make sure I never get the Read Time out error.

I also came across [this](https://github.com/swar/nba_api/issues/94) but it seems they are suggesting to add additional values to the headers but I am not sure if that is a generic solution because that may have to be specific from website to website. I also read on another [SO Post](https://stackoverflow.com/questions/52717282/requests-httpconnectionpool-read-timeout-never-recovers) that we should base64 the request and retry. It went over my head as I have no idea how to do that and there was not a example provided by the person.

Any advice by those who have experience in scraping would highly be appreciated.