title: TypeError: 'Request' object is not iterable error while Parsing from HTML
body: <p>I wrote script to parse the information from one website using BeautifulSoup, but I have problems with it. </p>
  
  <p>As seen from the code, in the commented section for websiteReq it gives me error of </p>
  
  <p><strong>raise HTTPError(req.full_url, code, msg, hdrs, fp)
  HTTPError: Forbidden</strong></p>
  
  <p>I have searched on internet and usage of Request was recommended. I tried it as seen in the code, but I get error of <strong>TypeError: 'Request' object is not iterable</strong> . </p>
  
  <p>I would like to know where I'm doing wrong. Thank you. Have a nice day. </p>
  
  <p><em>Code:</em></p>
  
  <pre><code>   # -*- coding: utf-8 -*-
  """
  Created on Tue Jul 23 22:39:05 2019
  
  @author: Nijat
  """
  
  try:
      # For Python 3.0 and later
      from urllib.request import Request, urlopen
  except ImportError:
      # Fall back to Python 2's urllib2
      from urllib2 import urlopen
  
  from bs4 import BeautifulSoup
  import csv 
  from datetime import datetime
  import requests
  
  webData = []
  
  #websiteReq = ['https://boss.az/vacancies'] #FORBIDDEN ERROR WITH THIS 
  
  websiteReq = Request('https://boss.az/vacancies', headers={'User-Agent': 'Mozilla/5.0'}) #Request not iterable
  
  #for pg in websiteReq: #ERROR OCCURS HERE
  page = urlopen(websiteReq)
  soupping = BeautifulSoup(page, 'html.parser').encode("utf-8")
  
  takingTitle = soupping.find_all('h3', attrs={'class':'results-i-title'})
  takingCompany = soupping.find_all('a', attrs={'class':'results-i-company'})
  takingDescription = soupping.find_all('div', attrs={'class':'results-i-summary'})
  
  nameofVac = takingTitle.text.strip()
  nameofComp = takingCompany.text.strip()
  nameofDescript = takingDescription.text.strip()
  
  webData.append((nameofVac, nameofComp, nameofDescript))
  
  with open('Vacancies.csv','a') as csvfile:
      writingtocsv = csv.writer(csvfile)
      for nameofVac, nameofComp in webData:
          writingtocsv.writerow([nameofVac, nameofComp, nameofDescript,datetime.now()])
  </code></pre>
  
  <p>Updated Code from answer:</p>
  
  <p>So, the result gives me all the parsed information from first page which was about 30 vacancies. If you would like to add other pages, just do += </p>
  
  <pre><code>from bs4 import BeautifulSoup as bs
  import requests
  import pandas as pd
  
  for t in range(28):
      url = "https://boss.az/vacancies?action=index&amp;controller=vacancies&amp;only_path=true&amp;page={}&amp;type=vacancies".format(t)
      r = requests.get(url)
      soup = bs(r.content, 'html.parser')
  
      titles = [i.text for i in soup.select('.results-i-title')]
      #print(titles)
      companies = [i.text for i in soup.select('.results-i-company')]
      #print(companies)
      summaries = [i.text for i in soup.select('.results-i-summary')]
  
  df = pd.DataFrame(list(zip(titles, companies, summaries)), columns = ['Title', 'Company', 'Summary'])
  df.to_csv(r'Data.csv', sep=',', encoding='utf-8-sig',index = False )
  </code></pre>
  
  <p>However, it doesn't read all the pages now. Any clue why?</p>