title: How to scrape the data using requests module only in python
body: <p>I am actually trying to parse a website using the requests module, and extract some text out of it. </p>
  
  <p>Url : <a href="https://www.icsi.in/student/Members/MemberSearch.aspx" rel="nofollow noreferrer">https://www.icsi.in/student/Members/MemberSearch.aspx</a></p>
  
  <p>after hitting the url in the Cp Number text field input : 16803</p>
  
  <p>hit search, </p>
  
  <p>on the bottom you can see some data, I want that data, let's say a name.</p>
  
  <p>I am successfully able to get the data using selenium, but can't able to get it using requests module.</p>
  
  <p>I have tried the requests module giving parameters, sessions, cookies etc. 
  but nothing worked.</p>
  
  <pre><code>url = "https://www.icsi.in/student/Members/MemberSearch.aspx"
  ss = {'dnn$ctr410$MemberSearch$txtCpNumber':'16803',
        '__EVENTTARGET':'dnn$ctr410$MemberSearch$btnSearch',
        '__VIEWSTATEGENERATOR':'6A295697',
        'dnn$ctlHeader$dnnSearch$Search':'SiteRadioButton'}
  
  session = requests.Session()
  cookies = session.cookies.get_dict()
  
  for cookie in cookies:
      session.cookies.set(cookie['name'], cookie['value'])
  
  response = requests.post(url, data=ss)
  print(response)
  
  HTMLTree = html.fromstring(response.content)
  
  name = HTMLTree.xpath('//div[@class="name_head"]//text()')
  
  print(name)
  </code></pre>
  
  <p>I expect the output of the name of the person.</p>
  
  <p>Anyone out there please help me.</p>