title: Login with __RequestVerificationToken cookie before scraping data?
body: <p>I want to parse a data from a page but this data is showing only registered users. So, I have to login first then parse it. There is no problem with the parsing side but I have problem on login side. Where am I wrong with?</p>
  
  <p>Here is my only login code:</p>
  
  <blockquote>
    <p>I have changed private domain name to domain.com</p>
  </blockquote>
  
  <pre><code>import requests
  
  with requests.Session() as c:
      url = 'https://domain.com/giris?returnUrl=https://domain.com/'
      USERNAME = 'xxxxxxx@gmail.com'
      PASSWORD = '1111111'
      c.get(url)
      __RequestVerificationToken = c.cookies['__RequestVerificationToken']
      login_data = dict(__RequestVerificationToken=__RequestVerificationToken, UserName=USERNAME, Password=PASSWORD, ReturnUrl='https://domain.com/', RememberMe='false')
      c.post(url, data=login_data, headers={"Referer": "https://domain.com/giris?returnUrl=https%3A%2F%2Fdomain.com%2F"})
      page = c.get('https://domain.com/mesaj/')
      print page.content
  </code></pre>
  
  <p>If login is succeeded I need to see <code>https://domain.com/mesaj/</code> page but it redirects login page again because of unsuccessful login.</p>
  
  <p>Also, here is the Header of login page that is captures from Google Chrome:</p>
  
  <p><strong><em>General</em></strong></p>
  
  <pre><code>Request URL:https://domain.com/giris
  Request Method:POST
  Status Code:302 Found
  Remote Address:176.53.43.2:443
  Response Headers
  view source
  </code></pre>
  
  <p><strong><em>Response Headers</em></strong></p>
  
  <pre><code>Cache-Control:no-cache
  Content-Length:140
  Content-Type:text/html; charset=utf-8
  Date:Tue, 24 Jan 2017 16:51:30 GMT
  Expires:-1
  Location:https://domain.com/
  Pragma:no-cache
  Set-Cookie:a=vjFFqBh+ZZMKr71K2XrBCr5SutOMAOpWjv1RAS5hRYMrR2RojaTV/wgIP8HiUOjdMU7x28DpfxRsCnfSvLeLHPvGTBKjwF0O5W99julK7w23vdctrnE5FDBlhXSSB9nCQm+DB3vNgGjxEr+DNRMrWNwMZWbSQID+klPDtUnReAJQA/GfLdoo2izsD0HP6tir; path=/; HttpOnly
  Strict-Transport-Security:max-age=31536000; includeSubDomains; preload
  X-Content-Type-Options:nosniff
  X-Frame-Options:DENY
  X-XSS-Protection:1; mode=block
  Request Headers
  view source
  </code></pre>
  
  <p><strong><em>Request Headers</em></strong></p>
  
  <pre><code>Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
  Accept-Encoding:gzip, deflate, br
  Accept-Language:tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
  Cache-Control:max-age=0
  Connection:keep-alive
  Content-Length:251
  Content-Type:application/x-www-form-urlencoded
  Cookie:iq=c3bae8d44a624f03a431c8df6741af36; __gfp_64b=0icQApqT.NFM3ZR0rZLAyXumlDI4n2eStqQJ74n4H...U7; __gads=ID=bb39b708ca7de25d:T=1484676131:S=ALNI_MYxcRsRoDIaQmsY859bPz_jriFRDA; ASP.NET_SessionId=efsub3xz101y542pyjhugvi2; cookies_info_viewed=yes; notheme=1; __adm_int_sc=1; __adm_int=1; __RequestVerificationToken=xzayOzscFJ3a_m4C5jF8eaKrW7F_Yen7umGMm_nZxDPKmO5rUKacPc4yHK63wVqQwd2S_H2mLiKt_ROW2pCG1B5ZTEtytYF-GU0khK2BlnM1; _gat=1; _ga=GA1.2.633914475.1484676130; __asc=14f63ba5159d14e81061abc20c9; __auc=328f9ce5159ad97e58cfeb70218
  Host:domain.com
  Origin:https://domain.com
  Referer:https://domain.com/giris?returnUrl=https%3A%2F%2Fdomain.com%2F
  Upgrade-Insecure-Requests:1
  User-Agent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
  Form Data
  view source
  view URL encoded
  </code></pre>
  
  <p><strong><em>Form Data</em></strong></p>
  
  <pre><code>__RequestVerificationToken:F2ZCmIge7rSV4A4Xoelf3aweaDQ9vNHew16Bfb6GDSlpeFQeQ_cfmV6UrFhNRWRBqvGPXzXrxVLAIXgbKI-08Q0fD3Vfttezq5hTkMFYTwo1
  ReturnUrl:https://domain.com/
  UserName:xxxxxxx@gmail.com
  Password:1111111
  RememberMe:false
  </code></pre>