title: Scrapy Parsing Unwanted URLs
body: <p>I'm scraping apartments.com with Scrapy. I'm getting a few unwanted results. Specifically, I'm getting results from nearby cities and from the Spanish-language version of apartments.com. On the bottom of each apartment listing, there is a &quot;nearby apartments&quot; feature, which may be why Scrapy is getting results outside of Boston. I tried to prevent the /es/ listings with my deny rule but it doesn't appear to be working.</p>
  <pre><code>import scrapy
  from scrapy.linkextractors import LinkExtractor
  from scrapy.spiders import CrawlSpider, Rule
  from apt.items import AptItem
  from urllib.parse import urljoin
  from scrapy.selector import Selector
  
  
  class AptSpider(CrawlSpider):
      name = &quot;apt&quot;
      allowed_domains = [&quot;apartments.com&quot;]
      start_urls = [&quot;https://www.apartments.com/boston-ma/&quot;]
  
      rules = (Rule(LinkExtractor(allow=r'[1-9]+/*'), callback='parse_urls', follow=False),
      Rule(LinkExtractor(deny=(r'/es/.*', ))),)
  
      def parse_item(self, response):    
          //parse item code
  
      def parse_urls(self, response):
          apts = response.xpath(&quot;//*[contains(@class, 'property-link')]/@href&quot;).extract()
          for a in apts:
              url = urljoin(response.url, a)
              yield scrapy.Request(url, callback=self.parse_item)
  </code></pre>
  <p>How can I exclude the Spanish results as well as the results outside of Boston?</p>