title: In Scrapy, how to use nested item loaders without further 'add_xpath' calls
body: <p>I'm trying to refactor a Scrapy spider making use of <a href="https://doc.scrapy.org/en/latest/topics/loaders.html" rel="nofollow noreferrer">Item Loaders</a>. I've gotten to a part where I'm populating several fields from data extracted from a single XPath expression, and for brevity I'd like to use <a href="https://doc.scrapy.org/en/latest/topics/loaders.html#nested-loaders" rel="nofollow noreferrer">Nested Loaders</a>.</p>
  
  <p>Here is the spider so far:</p>
  
  <pre><code>from scrapy.spiders import SitemapSpider
  from apkmirror_scraper.items import ApkmirrorScraperItem, ApkmirrorItemLoader
  
  class ApkmirrorSitemapSpider(SitemapSpider):
      name = 'apkmirror-spider'
      sitemap_urls = ['http://www.apkmirror.com/sitemap_index.xml']
      sitemap_rules = [(r'.*-android-apk-download/$', 'parse')]
  
      def parse(self, response):
          loader = ApkmirrorItemLoader(item=ApkmirrorScraperItem(), response=response)
  
          loader.add_value('url', response.url)
          loader.add_xpath(field_name='title', xpath='//h1[@title]/text()')
          loader.add_xpath(field_name='developer', xpath='//h3[@title]/a/text()')
          loader.add_xpath(field_name='app', xpath='//*[contains(@data-channel-name, "App Updates")]/@data-channel-name')
  
          apk_details_loader = loader.nested_xpath('//*[@title="APK details"]/following-sibling::*[@class="appspec-value"]')
  
          apk_details_loader.add_xpath(field_name="version_name", xpath=".//text()")
          apk_details_loader.add_xpath(field_name="version_code", xpath=".//text()")
  
          return loader.load_item()
  </code></pre>
  
  <p>where <code>items.py</code> is</p>
  
  <pre><code>import re
  
  import scrapy
  import scrapy.loader
  
  from scrapy.loader.processors import Compose, MapCompose, TakeFirst
  
  class ApkmirrorScraperItem(scrapy.Item):
      url = scrapy.Field()
      title = scrapy.Field()
      developer = scrapy.Field()
      app = scrapy.Field()
      version_name = scrapy.Field()
      version_code = scrapy.Field()
      architectures = scrapy.Field()
      package = scrapy.Field()
      apk_file_size = scrapy.Field()
      android_min_version = scrapy.Field()
      android_target_version = scrapy.Field()
      supported_dpis = scrapy.Field()
      md5_signature = scrapy.Field()
      time_uploaded = scrapy.Field()
      time_scraped = scrapy.Field()
      download_link = scrapy.Field()
  
  
  def parse_app(data_channel_name):
      '''Parse the name of the app from the "data-channel-name" attribute of the button named "Follow [app_name] Updates".'''
      pattern = re.compile(r'(?P&lt;app&gt;.+) App Updates')
      return pattern.search(data_channel_name).groupdict().get("app")
  
  def get_version_line(apk_details):
      '''Get the line containing the version from the 'APK details' section.'''
      return next(line for line in apk_details if line.startswith("Version:"))
  
  def get_architectures_line(apk_details):
      '''Get the line containing the supported architectures (e.g. "arm", "x64") from the 'APK details' section, if present.'''
      return apk_details[1] if not apk_details[1].startswith("Package:") else None       # The line does not contain any keywords and may not be present, in which case None is returned
  
  def get_package_line(apk_details):
      return next(line for line in apk_details if line.startswith("Package:"))                # The 'package line' is always present and starts with "Package:"
  
  def parse_version_line(version_line):
      '''Parse the 'versionName' and 'versionCode' from the relevant line in 'APK details'.'''
      PATTERN = r"^Version: (?P&lt;version_name&gt;.+) \((?P&lt;version_code&gt;\d+)\)\s*$"       # Note that the pattern includes the end-of-line character ($). This is necessary because some package names (e.g. Google Play) themselves contain brackets.
      return re.match(PATTERN, version_line).groupdict()
  
  
  class ApkmirrorItemLoader(scrapy.loader.ItemLoader):
  
      url_out = TakeFirst()
  
      title_in = MapCompose(unicode.strip)
      title_out = TakeFirst()
  
      developer_in = MapCompose(unicode.strip)
      developer_out = TakeFirst()
  
      app_in = MapCompose(parse_app)
      app_out = TakeFirst()
  
      version_name_in = Compose(get_version_line, parse_version_line, lambda d: d.get("version_name"))
      version_name_out = TakeFirst()
  
      version_code_in = Compose(get_version_line, parse_version_line, lambda d: d.get("version_code"))
      version_code_out = TakeFirst()
  </code></pre>
  
  <p>If I run the spider, for example, using the command</p>
  
  <pre><code>scrapy parse --spider=apkmirror-spider http://www.apkmirror.com/apk/google-inc/sheets/sheets-1-7-152-06-release/google-sheets-1-7-152-06-30-android-apk-download/
  </code></pre>
  
  <p>It extracts the fields as expected:</p>
  
  <pre><code># Scraped Items  ------------------------------------------------------------
  [{'app': u'Google Sheets',
    'developer': u'Google Inc.',
    'title': u'Google Sheets 1.7.152.06.30 (arm) (nodpi)',
    'url': 'http://www.apkmirror.com/apk/google-inc/sheets/sheets-1-7-152-06-release/google-sheets-1-7-152-06-30-android-apk-download/',
    'version_code': u'71520630',
    'version_name': u'1.7.152.06.30'}]
  </code></pre>
  
  <p>I would like to further refactor the <code>apk_details_loader</code>, however. Notice how the two lines with <code>apk_details_load.add_xpath</code> have the same <code>xpath</code> argument, <code>".//text()"</code>. To avoid this code repetition, I would like to instead define <code>apk_details_loader</code> as</p>
  
  <pre><code>apk_details_loader = loader.nested_xpath('//*[@title="APK details"]/following-sibling::*[@class="appspec-value"]//text()')
  </code></pre>
  
  <p>including the <code>//text()</code> at the end, and instead of <code>add_xpath</code> do something like just <code>add</code>, which instead of further refining the XPath expression and then calling the input processors on them, simply calls the input processors directly. Is something like this possible?</p>