title: Structuring an XHR request properly with Requests [python]
body: <p>I'm trying to do some web scraping of a site that generates it's data via Javascript.  I've done enough reading on here to know by now that the way to scrape these is to:</p>
  
  <ol>
  <li>Watch the network tab in Firebug for what happens when you make the request</li>
  <li>Isolate the XHR requests and recreate them in a script.</li>
  </ol>
  
  <p>So, when I do 1, a POST request is sent to the link visible in this screenshot:
  <a href="https://i.stack.imgur.com/7unC4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7unC4.png" alt="enter image description here"></a>
  and you can also see the response it gets.  Looks great, right?  </p>
  
  <p>But when I try and recreate that request &amp; response, with the payload that I see under the Post tab in Firebug, in Python like so:</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  
  payload = {"Max":999,"RectCoord":"89,-179,-89,179","Source":"","SortField":"NEWID()","OfficeName":"","FirstName"
  :"","LastName":"da","CityName":"","ZipCode":"","Category":"S","SecLanguageReq":"","OfficeCode":""}
  
  r = requests.post('http://search.cnyrealtor.com/MyAjaxService.asmx/MemberSearch', data=payload)
  
  print(r.content)
  </code></pre>
  
  <p>I get a page that displays an error message:
  <code>Request format is unrecognized for URL unexpectedly ending in \'/MemberSearch\'</code></p>
  
  <p>So, my question is - why am I getting that response when the response in Firebug works fine?  Am I missing something in my <code>requests.post(url)</code> line in the Python script?</p>