title: Python, Scrapy, Selenium - Fill out a form
body: <p>I have a page where I click on the text "Neu hier? Jetzt registrieren". Then a text box and a button will appear. How can I eliminate the error?</p>
  
  <p>He does not find either the text box or the button. </p>
  
  <p>HTML-Part:</p>
  
  <pre><code>&lt;form method="post" action="./index.aspx" onsubmit="javascript:return WebForm_OnSubmit();" id="aspnetForm"&gt;
  &lt;input type="text" class="email w_200" name="Email" value="" autocomplete="on" tabindex="1" /&gt;
  &lt;input type="button" class="submit bu" style="width: 197px" tabindex="4" value="Registrieren"/&gt;
  &lt;/form&gt;
  </code></pre>
  
  <p>Result from the code below:</p>
  
  <pre><code>start!ready!you are here!
  </code></pre>
  
  <p>The source:</p>
  
  <pre><code>from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  import scrapy
  from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
  from scrapy.http import Request
  #https://github.com/mozilla/geckodriver/releases
  class northshoreSpider(scrapy.Spider):
      name = 'xxx'
      allowed_domains = ['mypage.de']
      start_urls = ['http://mypage.de']
  
      def __init__(self, category='', **kwargs):
          print('start!')
          options = Options()
          options.add_argument('--no-sandbox')
          options.add_argument('--disable-dev-shm-usage')
          options.add_argument('--headless')
          options.add_argument('--disable-gpu')
          super().__init__(**kwargs)
  
          self.driver = webdriver.Chrome(executable_path='/home/chef/Desktop/chromedriver', chrome_options=options)
  
      def parse(self,response):
          self.driver.get('http://mypage.de')
          try:
              next = self.driver.find_elements_by_xpath("//*[contains(text(), 'Neu hier? Jetzt registrieren')]")[0]
              url = 'http://mypage.de'
              yield Request(url,callback=self.parse2)
              next.click()
          except Exception as e:
              print('error2!' +  str(e))
          self.driver.close()
          print('ready!')
  
      def parse2(self,response):
          name="Email"
          print('you are here!')
          formdata = {'name': 'mailfrom@gmx.de'}
          yield FormRequest.from_response(response,
                                  formnumber=0,
                                  formdata=formdata,
                                  clickdata={'value': 'Registrieren'},
                                  callback=self.parse3)
  
      def parse3(self,response):
          print("doneTestRegistration")
  </code></pre>