title: Scrapy: Same Xpath for different fields
body: <p>I am trying to scrape different category products from <a href="http://www.galaxus.ch" rel="nofollow noreferrer">www.galaxus.ch</a> using scrapy. To render the HTML I used Splash and Lua script. To read the excel file I used pandas. Till now my script is working fine. Here is my code_</p>
  <h1>read_excel</h1>
  <pre><code>import pandas as pd
  
  def read_xlsx():
      df = pd.read_excel('externe_festplatte.xlsx')
      return df['Gtin'].dropna().astype('int64').tolist()
  </code></pre>
  <h1>spider</h1>
  <pre><code>import scrapy
  from scrapy_splash import SplashRequest
  from galaxus.spiders.read_files import read_xlsx
  
  
  base_url = &quot;https://www.galaxus.ch/search?q={}&quot;
  
  
  class GtinSpider(scrapy.Spider):
      name = 'gtin'
      allowed_domains = ['www.galaxus.ch']
  
      script = '''
          function main(splash, args)
              splash:set_user_agent(&quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.212 Safari/537.36&quot;)
              splash.private_mode_enabled = false
              assert(splash:go(args.url))
              assert(splash:wait(5))
              
              item_select = assert(splash:select(&quot;div.panelLayout_mainContainer__11Jh_&quot;))
              item_select:mouse_click()
              assert(splash:wait(5))
              
              see_more = assert(splash:select(&quot;[data-test='showMoreButton-specifications'] span&quot;))
              see_more:mouse_click()
              assert(splash:wait(5))
              
              
              splash:set_viewport_full()
              return splash:html()
          end
      '''
  
      def start_requests(self):
          for value in read_xlsx():
              yield SplashRequest(
              url=base_url.format(value),
              callback=self.parse, endpoint='execute', args={
              'lua_source': self.script
          }
          )
  
      def parse(self, response):
          yield{
          'Titel': response.xpath(&quot;.//span[@class='jqo5ci-1 goteOY']/text()&quot;).get(),
          'Untertitel': response.xpath(&quot;.//span[@class='jqo5ci-2 beeFWi']/text()&quot;).get(),
          'Beschreibung': response.xpath(&quot;//div[@class='sc-1op7ol6-0 hYPLAr']/span/text()&quot;).get(),
          'Kategorie': response.xpath(&quot;(.//div[@class='breadcrumbView_withIcon__3mWwP']/a)[4]/text()&quot;).get(),
          'Produktetyp': response.xpath(&quot;.//span[@class='yip624-0 dpAcNY']/text()&quot;).get(),
          'Hersteller': response.xpath(&quot;.//h1[@class='jqo5ci-0 czhxQj']/strong/text()&quot;).get()
         }
  </code></pre>
  <p>The problem is that if I also want to scrape the Spezifikationen/ Specification field from the same page, it is different for every product category but having the same Xpath <code>//td[@class='sc-18g78bs-4 sxRfA']</code>.
  Example:</p>
  <ol>
  <li><p>SSD
  <a href="https://www.galaxus.ch/de/s1/product/freecom-mobile-drive-classic-750gb-075tb-externe-festplatte-12097425?supplier=3204073" rel="nofollow noreferrer">https://www.galaxus.ch/de/s1/product/freecom-mobile-drive-classic-750gb-075tb-externe-festplatte-12097425?supplier=3204073</a></p>
  </li>
  <li><p>RAM
  <a href="https://www.galaxus.ch/de/s1/product/csx-ddr3-1333-mhz-sodimm-memory-4-gb-1-x-4gb-ddr3-1333-so-dimm-ram-12446698?supplier=3204073" rel="nofollow noreferrer">https://www.galaxus.ch/de/s1/product/csx-ddr3-1333-mhz-sodimm-memory-4-gb-1-x-4gb-ddr3-1333-so-dimm-ram-12446698?supplier=3204073</a></p>
  </li>
  </ol>
  <p>For these two products categories, in the Spezifikationen field, they are having the same Xpath with different field names. For SSD it is &quot;Formfaktor&quot; and for RAM it is &quot;Arbeitsspeichertyp&quot; but the Xpath is the same for both. How to solve this issue? I also want to export the result to the same excel file.</p>
  <p>*I hope I am able to make my point clear. I'm a new StackOverflow user. I'm trying to accustomed to it. Expecting your suggestions and guidance.</p>