title: Scrapy FormRequest parameter not working but showing all result instead
body: <p>I am scraping this webpage <a href="https://researchgrant.gov.sg/eservices/advanced-search/?keyword=&amp;source=sharepoint&amp;type=project&amp;status=open&amp;_pp_projectstatus=&amp;_pp_hiname=&amp;_pp_piname=&amp;_pp_source=sharepoint&amp;_pp_details=#project" rel="nofollow noreferrer">https://researchgrant.gov.sg/eservices/advanced-search/?keyword=&amp;source=sharepoint&amp;type=project&amp;status=open&amp;_pp_projectstatus=&amp;_pp_hiname=&amp;_pp_piname=&amp;_pp_source=sharepoint&amp;_pp_details=#project</a> using Scrapy FormRequest. My code is as below. The parameter <code>_pp_hiname</code> with <code>ab</code> and <code>_pp_piname</code> with <code>pua</code> should only return 1 result in <code>response.text</code> but instead it returns all the result in HTML code. Parameter apparently not working but I couldn't see any wrong with it. </p>
  
  <pre><code>def start_requests(self):
      params = {
          'keyword': '',
          'source': 'sharepoint',
          'type': 'project',
          'status': 'open',
          'page': '1',
          '_pp_projectstatus': '',
          '_pp_hiname': 'ab',
          '_pp_piname': 'pua',
          '_pp_source': '',
          '_pp_details': '',
          'name':'advancesearchawardedprojectsp'
      }
      yield scrapy.FormRequest('https://researchgrant.gov.sg/eservices/mvcgrid',callback=self.parse_item,method='POST',formdata=params,headers = {'X-Requested-With':'XMLHttpRequest'})
  
  def parse_item(self,response):
      print(response.text)
  </code></pre>
  
  <p>Should be only 1 entry:
  <a href="https://i.stack.imgur.com/sv88a.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sv88a.png" alt="actual result"></a></p>
  
  <p>But apparently it is showing all entry:
  <a href="https://i.stack.imgur.com/RcSYy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RcSYy.png" alt="shows all result"></a></p>
  
  <p><strong>Latest update:</strong></p>
  
  <pre><code>class ToScrapeCSSSpiderSG(scrapy.Spider):
  name = "toscrapesg-css"
  # start_urls = [
      # 'https://researchgrant.gov.sg/eservices/mvcgrid',
  # ] 
  params = {
      'name':'advancesearchawardedprojectsp'
      }  
  args = {
          'keyword': '',
          'source': 'sharepoint',
          'type': 'project',
          'status': 'open',
          'page': 1,
          '_pp_projectstatus': '',
          '_pp_hiname': 'ab',
          '_pp_piname': '',
          '_pp_source': '',
          '_pp_details': '',
          'name':'advancesearchawardedprojectsp'
      }
  def start_requests(self):
      args = urllib.parse.urlencode(self.args)
      url = 'https://researchgrant.gov.sg/eservices/mvcgrid?' + args
      yield scrapy.FormRequest(url,callback=self.parse_item,method='POST',formdata=self.params,headers = {'X-Requested-With':'XMLHttpRequest'})
  
  
  def parse_item(self,response):
      args = urllib.parse.urlencode(self.args)
      url = 'https://researchgrant.gov.sg/eservices/mvcgrid?' + args
      for quote in response.xpath('//div[contains(@style,"overflow-x:auto")]'):
          for row in quote.xpath('./table[contains(@class,"table-striped")]/tbody/tr'):
              link=row.xpath('td[1]/a/@href').extract_first()
              yield scrapy.FormRequest(link,callback = self.parse_product,method='GET')
  
  
      onclick = response.xpath('//a[@aria-label="Next page"]/@onclick').get()
      if onclick:
          self.args['page'] += 1
          args = urllib.parse.urlencode(self.args)
          url = 'https://researchgrant.gov.sg/eservices/mvcgrid?' + args
          yield scrapy.FormRequest(url, callback=self.parse_item, method='POST', formdata=self.params, headers = {'X-Requested-With': 'XMLHttpRequest'})
  
  def parse_product(self,response):
          text = response.xpath('//span[contains(@id,"ctl00_ctl47_g_b43c0a74_fae0_498f_b75e_c103772db011_ctl00_lblProjectTitle")]/text()').extract()
          # text = info.xpath('./text()').extract()
          print(text)
  </code></pre>
  
  <p><strong>Console message:</strong>
  <a href="https://i.stack.imgur.com/ecpL1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ecpL1.png" alt="enter image description here"></a></p>