title: Permission Denied Error While Scraping Website
body: <p>Scraping the following website, <a href="http://www.starcitygames.com/buylist/" rel="nofollow noreferrer">http://www.starcitygames.com/buylist/</a>, but I keep getting the following error, and I do not know what is causing it.  When I first wrote the program it was working just fine without any errors, scraping the data I needed and everything but now I am getting this error and I have no idea why, tried changing the Splash URL and the user agent but that did not work, still gave me same error:</p>
  
  <pre><code>2019-07-23 12:37:28 [scrapy.core.engine] INFO: Spider opened
  2019-07-23 12:37:28 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
  2019-07-23 12:37:28 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
  2019-07-23 12:37:28 [scrapy.extensions.throttle] INFO: slot: www.starcitygames.com | conc: 1 | delay:15000 ms (+0) | latency:  148 ms | size:     0 bytes
  2019-07-23 12:37:28 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (307) to &lt;GET https://www.starcitygames.com/login&gt; from &lt;GET http://www.starcitygames.com/buylist/&gt;
  2019-07-23 12:37:43 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying &lt;GET https://www.starcitygames.com/login&gt; (failed 1 times): An error occurred while connecting: 13: Permission denied.
  2019-07-23 12:38:04 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying &lt;GET https://www.starcitygames.com/login&gt; (failed 2 times): An error occurred while connecting: 13: Permission denied.
  2019-07-23 12:38:24 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying &lt;GET https://www.starcitygames.com/login&gt; (failed 3 times): An error occurred while connecting: 13: Permission denied.
  2019-07-23 12:38:24 [scrapy.core.scraper] ERROR: Error downloading &lt;GET https://www.starcitygames.com/login&gt;
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/scrapy/core/downloader/middleware.py", line 43, in process_request
      defer.returnValue((yield download_func(request=request,spider=spider)))
  twisted.internet.error.ConnectError: An error occurred while connecting: 13: Permission denied.
  2019-07-23 12:38:24 [scrapy.core.engine] INFO: Closing spider (finished)
  </code></pre>
  
  <p>LoginSpider.py</p>
  
  <pre><code># Import needed functions and call needed python files
  import scrapy
  import json
  from scrapy.spiders import Spider
  from scrapy_splash import SplashRequest
  from ..items import DataItem
  
  # Spider class
  class LoginSpider(scrapy.Spider):
      # Name of spider
      name = "LoginSpider"
  
      #URL where dated is located
      start_urls = ["http://www.starcitygames.com/buylist/"]
  
      # Login function
      def parse(self, response):
          # Login using email and password than proceed to after_login function
          return scrapy.FormRequest.from_response(
          response,
          formcss='#existing_users form',
          formdata={'ex_usr_email': 'example@email.com', 'ex_usr_pass': 'password'},
          callback=self.after_login
          )
  
  
      # Function to barse buylist website
      def after_login(self, response):
          # Loop through website and get all the ID numbers for each category of card and plug into the end of the below
          # URL then go to parse data function
          for category_id in response.xpath('//select[@id="bl-category-options"]/option/@value').getall():
              yield scrapy.Request(
                      url="http://www.starcitygames.com/buylist/search?search-type=category&amp;id={category_id}".format(category_id=category_id),
                      callback=self.parse_data,
                      )
      # Function to parse JSON dasta
      def parse_data(self, response):
          # Declare variables
          jsonreponse = json.loads(response.body_as_unicode())
          # Call DataItem class from items.py
          items = DataItem()
  
          # Scrape category name
          items['Category'] = jsonreponse['search']
          # Loop where other data is located
          for result in jsonreponse['results']:
              # Inside this loop, run through loop until all data is scraped
              for index in range(len(result)):
                  # Scrape the rest of needed data
                  items['Card_Name'] = result[index]['name']
                  items['Condition'] = result[index]['condition']
                  items['Rarity'] = result[index]['rarity']
                  items['Foil'] = result[index]['foil']
                  items['Language'] = result[index]['language']
                  items['Buy_Price'] = result[index]['price']
                  # Return all data
                  yield items
  </code></pre>
  
  <p>settings.py</p>
  
  <pre><code># Name of project
  BOT_NAME = 'LoginSpider'
  
  # Module where spider is
  SPIDER_MODULES = ['LoginSpider.spiders']
  # Mode where to create new spiders
  NEWSPIDER_MODULE = 'LoginSpider.spiders'
  
  # Obey robots.txt rules set by website, disable to not be detected as web scraper
  ROBOTSTXT_OBEY = False
  
  # The path of the csv file that contains the proxies/user agnets paired with URLs
  #PROXY_CSV_FILE = "url.csv"
  USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36'
  # The downloader middleware is a framework of hooks into Scrapy's request/response processing.
  # It's a light, low-level system for globally altering Scrapy's requests and responses.
  DOWNLOADER_MIDDLEWARES = {
          # This middleware enables working with sites that require cookies, such as those that use sessions.
          # It keeps track of cookies sent by web servers, and send them back on subsequent requests (from that spider), just like web browsers do.
          'scrapy_splash.SplashCookiesMiddleware': 723,
  
          'scrapy_splash.SplashMiddleware': 725,
          # This middleware allows compressed (gzip, deflate) traffic to be sent/received from web sites.
          # This middleware also supports decoding brotli-compressed responses, provided brotlipy is installed.
          'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
  }
  
  # URL that splash server is running on, must be activated to use splash
  SPLASH_URL = 'http://199.89.192.98:8050'
  # The class used to detect and filter duplicate requests
  DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter'
  # This middleware provides low-level cache to all HTTP requests and responses. It has to be combined with a cache storage backend as well as a cache policy.
  HTTPCACHE_STORAGE = 'scrapy_splash.SplashAwareFSCacheStorage'
  
  # The maximum number of concurrent (ie. simultaneous) requests that will be performed by the Scrapy downloader (default: 16)
  
  CONCURRENT_ITEMS = 1
  CONCURRENT_REQUESTS = 1
  
  # Configure a delay for requests for the same website (default: 0)
  # See https://doc.scrapy.org/en/latest/topics/settings.html#download-delay
  # See also autothrottle settings and docs
  
  # If enabled, Scrapy will wait a random amount of time (between 0.5 * DOWNLOAD_DELAY and 1.5 * DOWNLOAD_DELAY) while fetching requests from the same website.
  RANDOMIZE_DOWNLOAD_DELAY = True
  # Delay between scraping webpages
  DOWNLOAD_DELAY = 10
  # The download delay setting will honor only one of:
  # Number of concurrent requests made to one URL(enabled)
  CONCURRENT_REQUESTS_PER_DOMAIN = 1
  # Number of concurrent requests made to one IP(disabled)
  #CONCURRENT_REQUESTS_PER_IP = 1
  
  # Disable cookies (enabled by default)
  # Whether to enable the cookies middleware. If disabled, no cookies will be sent to web servers.
  COOKIES_ENABLED = True
  #REDIRECT_ENABLED = False
  # Disable Telnet Console (enabled by default)
  # A boolean which specifies if the telnet console will be enabled (provided its extension is also enabled)
  #TELNETCONSOLE_ENABLED = False
  
  # Override the default request headers:
  DEFAULT_REQUEST_HEADERS = {
          'Referer': 'http://www.starcitygames.com/buylist/'
  }
  </code></pre>