How to get the xpath for jobpage?
Question: I have tried too many possibilities to get xpath for click the "Search Jobs
Now" and "Search " button for to get job list page. but its not find exactly
what i am expected.

Please let me know how to find click the "Search Jobs Now" and "Search" button
and get the joblist page.

Note:

I checked the Web-element **'Search jobs now'** is located under **frame id=
ptifrmtgtframe and name =TargetContent**. Once we switch to this frame then we
will be able to click on first page button and second page button like
"**Search** "

Platform : scrapy + selenium remote control + python

Here is spider code:

    
    
    class WellsfargocomSpider(Spider):
        name = 'wellsfargo'
        allowed_domains = ['www.wellsfargo.com']
        start_urls = ['https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=']
    
    
        #driver = webdriver.Remote('http://127.0.0.1:4444/wd/hub', desired_capabilities=webdriver.DesiredCapabilities.HTMLUNIT)
            # Create a new instance of the Firefox webdriver
        driver = webdriver.Firefox()
            # Create implicitly wait for 30
        #driver.implicitly_wait(0.5)
    
    
        def parse(self,response):
    
            selector = Selector(response)
            #self.driver.get('https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=')
            driver = self.driver
    
            #driver = webdriver.Firefox()
            self.driver.get('https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=');
            self.driver.switchTo().frame(self.driver.find_element_by_tag_name('TargetContent'))
            Thread.sleep(10000)
            clk = self.driver.find_element_by_xpath("//input[@id='HRS_CE_WELCM_WK_HRS_CE_WELCM_BTN']")
            clk.click()
    
            clk1 = self.driver.find_element_by_xpath("//input[@name='SEARCHACTIONS#SEARCH']")
            clk1.click()
    
            self.driver.switchTo().defaultContent()
    
            #inputElement = self.driver.find_element_by_css_selector("input.PSPUSHBUTTON")
            #inputElement.submit()
            #inputElement1 = self.driver.find_element_by_xpath("//input[@name='SEARCHACTIONS#SEARCH']")
            #inputElement1.click()
    
    
    
    
            #while True:
                #next = self.driver.find_element_by_xpath(".//*[@id='HRS_APPL_WRK_HRS_LST_NEXT']")
    
                #try:
            links = []
            for link in selector.css('span.PSEDITBOX_DISPONLY').re('.*>(\d+)<.*'):
                    #intjid = selector.css('span.PSEDITBOX_DISPONLY').re('.*>(\d+)<.*')
                        abc = 'https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?Page=HRS_APP_JBPST&FOCUS=Applicant&SiteId=1&JobOpeningId='+link+'&PostingSeq=1'
                            #print abc
                        yield Request(abc,callback=self.parse_iframe, headers={"X-Requested-With": "XMLHttpRequest"}, dont_filter=True)
    
                    #next.click()
                #except:
                    #break
    
            #self.driver.close()
    
        def parse_iframe(self,response):
            selector = Selector(response)
            url = selector.xpath('//*[@id="ptifrmtgtframe"]/@src').extract()[0]
            yield Request(url,callback=self.parse_listing_page, headers={"X-Requested-With": "XMLHttpRequest"}, dont_filter=True)
    

Here is output:

    
    
    C:\Users\xxxx\Downloads\wellsfargocom>scrapy crawl wellsfargo
    2014-11-28 10:40:07+0530 [scrapy] INFO: Scrapy 0.24.4 started (bot: wellsfargoco
    m)
    2014-11-28 10:40:07+0530 [scrapy] INFO: Optional features available: ssl, http11
    
    2014-11-28 10:40:07+0530 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE'
    : 'wellsfargocom.spiders', 'SPIDER_MODULES': ['wellsfargocom.spiders'], 'BOT_NAM
    E': 'wellsfargocom'}
    2014-11-28 10:40:07+0530 [scrapy] INFO: Enabled extensions: LogStats, TelnetCons
    ole, CloseSpider, WebService, CoreStats, SpiderState
    2014-11-28 10:40:07+0530 [scrapy] INFO: Enabled downloader middlewares: HttpAuth
    Middleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, Def
    aultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, Redirec
    tMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
    2014-11-28 10:40:07+0530 [scrapy] INFO: Enabled spider middlewares: HttpErrorMid
    dleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddlew
    are
    2014-11-28 10:40:07+0530 [scrapy] INFO: Enabled item pipelines:
    2014-11-28 10:40:07+0530 [wellsfargo] INFO: Spider opened
    2014-11-28 10:40:07+0530 [wellsfargo] INFO: Crawled 0 pages (at 0 pages/min), sc
    raped 0 items (at 0 items/min)
    2014-11-28 10:40:07+0530 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6
    023
    2014-11-28 10:40:07+0530 [scrapy] DEBUG: Web service listening on 127.0.0.1:6080
    
    2014-11-28 10:40:09+0530 [wellsfargo] DEBUG: Redirecting (302) to <GET https://e
    mployment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GB
    L?FOCUS=&> from <GET https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRM
    S/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=>
    2014-11-28 10:40:10+0530 [wellsfargo] DEBUG: Redirecting (302) to <GET https://e
    mployment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GB
    L?FOCUS=> from <GET https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS
    /c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=&>
    2014-11-28 10:40:10+0530 [wellsfargo] DEBUG: Crawled (200) <GET https://employme
    nt.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS
    => (referer: None)
    2014-11-28 10:40:20+0530 [wellsfargo] ERROR: Spider error processing <GET https:
    //employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB
    .GBL?FOCUS=>
            Traceback (most recent call last):
              File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 82
    4, in runUntilCurrent
                call.func(*call.args, **call.kw)
              File "C:\Python27\lib\site-packages\twisted\internet\task.py", line 63
    8, in _tick
                taskObj._oneWorkUnit()
              File "C:\Python27\lib\site-packages\twisted\internet\task.py", line 48
    4, in _oneWorkUnit
                result = next(self._iterator)
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\uti
    ls\defer.py", line 57, in <genexpr>
                work = (callable(elem, *args, **named) for elem in iterable)
            --- <exception caught here> ---
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\uti
    ls\defer.py", line 96, in iter_errback
                yield next(it)
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\con
    trib\spidermiddleware\offsite.py", line 26, in process_spider_output
                for x in result:
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\con
    trib\spidermiddleware\referer.py", line 22, in <genexpr>
                return (_set_referer(r) for r in result or ())
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\con
    trib\spidermiddleware\urllength.py", line 33, in <genexpr>
                return (r for r in result or () if _filter(r))
              File "C:\Python27\lib\site-packages\scrapy-0.24.4-py2.7.egg\scrapy\con
    trib\spidermiddleware\depth.py", line 50, in <genexpr>
                return (r for r in result or () if _filter(r))
              File "C:\Users\sureshp\Downloads\wellsfargocom\wellsfargocom\spiders\w
    ellsfargo.py", line 48, in parse
                self.driver.switchTo().frame(self.driver.find_element_by_tag_name('T
    argetContent'))
            exceptions.AttributeError: 'WebDriver' object has no attribute 'switchTo
    '
    
    2014-11-28 10:40:20+0530 [wellsfargo] INFO: Closing spider (finished)
    2014-11-28 10:40:20+0530 [wellsfargo] INFO: Dumping Scrapy stats:
            {'downloader/request_bytes': 1880,
             'downloader/request_count': 3,
             'downloader/request_method_count/GET': 3,
             'downloader/response_bytes': 7190,
             'downloader/response_count': 3,
             'downloader/response_status_count/200': 1,
             'downloader/response_status_count/302': 2,
             'finish_reason': 'finished',
             'finish_time': datetime.datetime(2014, 11, 28, 5, 10, 20, 84000),
             'log_count/DEBUG': 5,
             'log_count/ERROR': 1,
             'log_count/INFO': 7,
             'response_received_count': 1,
             'scheduler/dequeued': 3,
             'scheduler/dequeued/memory': 3,
             'scheduler/enqueued': 3,
             'scheduler/enqueued/memory': 3,
             'spider_exceptions/AttributeError': 1,
             'start_time': datetime.datetime(2014, 11, 28, 5, 10, 7, 448000)}
    2014-11-28 10:40:20+0530 [wellsfargo] INFO: Spider closed (finished)
    


Answer: It is called [`switch_to`](http://selenium-
python.readthedocs.org/api.html#selenium.webdriver.remote.webdriver.WebDriver.switch_to),
not `switchTo`. Here is a working example:

    
    
    from selenium import webdriver
    
    driver = webdriver.Firefox()
    driver.get('https://employment.wellsfargo.com/psp/PSEA/APPLICANT_NW/HRMS/c/HRS_HRAM.HRS_APP_SCHJOB.GBL?FOCUS=')
    
    # find the frame and switch to it
    frame = driver.find_element_by_id('ptifrmtgtframe')
    driver.switch_to.frame(frame)
    
    # find the button and click it
    button = driver.find_element_by_id('HRS_CE_WELCM_WK_HRS_CE_WELCM_BTN')
    button.click()
    

