title: How can I find a URL in a span that in a div?
body: <p>I am trying to find a URL thats in a span, which is in a div. </p>
  
  <p>In this case, its the link with the class "company_url" that I'm after.</p>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>&lt;div class="links standard"&gt;
      &lt;span class="link"&gt;
          &lt;a href="https://twitter.com/abacus" class="twitter_url icon_link fontello-twitter" rel="nofollow" target="_blank"&gt;&lt;/a&gt;
      &lt;/span&gt;
      &lt;span class="link"&gt;
          &lt;a href="http://www.facebook.com/abacuslabs" class="facebook_url icon_link fontello-facebook" rel="nofollow" target="_blank"&gt;&lt;/a&gt;
      &lt;/span&gt;
      &lt;span class="link"&gt;
          &lt;a href="https://www.linkedin.com/company/abacus-labs" class="linkedin_url icon_link fontello-linkedin" rel="nofollow" target="_blank"&gt;&lt;/a&gt;
      &lt;/span&gt;
      &lt;span class="link"&gt;
          &lt;a href="http://blog.abacus.com/" class="blog_url icon_link fontello-rss" rel="nofollow" target="_blank"&gt;&lt;/a&gt;
      &lt;/span&gt;
      &lt;span class="link"&gt;
          &lt;a href="http://abacus.com" class="company_url" rel="nofollow" target="_blank"&gt;abacus.com&lt;/a&gt;
      &lt;/span&gt;
  &lt;/div&gt;</code></pre>
  </div>
  </div>
  </p>
  
  <p>I've tested my xpaths for finding the div in the page and the link in the div. So I'm pretty confident that they are correct (I used <a href="http://www.freeformatter.com/xpath-tester.html#ad-output" rel="nofollow">http://www.freeformatter.com/xpath-tester.html#ad-output</a>).</p>
  
  <p>But when I run the code, nothing gets scraped. What am I doing wrong?</p>
  
  <pre><code>from scrapy import Spider
  from scrapy.selector import Selector
  import datetime
  from saas.items import StartupItem
  
  
  class StackSpider(Spider):
  name = "abacus"
  allowed_domains = ["angel.co"]
  start_urls = [
      "https://angel.co/abacus",
  ]
  
  def parse(self, response):
      questions = Selector(response).xpath('//div[contains(@class, "links standard")]')
  
      for question in questions:
          item = StartupItem()
          item['startupurl'] = question.xpath('/span[@class="link"]/a[@class="company_url"]/@href').extract()[0]
          item['source'] = 'angel.co'
          item['datetime'] = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
          yield item
  </code></pre>