Python XHR Request Timing Out
Question: Trying to wrap my head around using requests to get Javscript loaded content
without spawning an actual browser to render it. I'm looking at using the
requests lib to get the tables but I keep getting a 504 with my test code and
I'm not 100% why.

So I'm looking at getting horse racing data from: sports.betway.com/#/horse-
racing/uk-and-ireland/haydock

I watched the network traffic and found the source of the traffic. It's a call
to /emoapi/emos with an eventIds number.

I tried this:

    
    
    import requests
    
    url = 'https://sports.betway.com/emoapi/emos'
    params = {
        'eventIds': '807789',
        'lang': 'en'
    }
    headers = {'Accept': '*/*',
               'Accept-Encoding': 'gzip, deflate',
               'Accept-Language': 'en-US,en;q=0.8',
               'Connection': 'keep-alive',
               'Content-Length': '271',
               'Content-Type': 'application/json',
               'Host': 'sports.betway.com',
               'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36'}
    
       #Note: I do also set the origin and ref link in the header  but I can't post that many links in a question.
    
    
    
    response = requests.post(url, params=params, headers=headers)
    
    print response
    fixtures = response.json()
    print fixtures
    

I can't see what else I'm missing from the request. But the print response
comes back as a

This is an example of the full payload on the browser header which requests a
whole bunch of Ids rather than just the one I'm trying:

    
    
    {"eventIds":[807789,808612,808597,807790,808613,808598,807791,808611,808599,807792,808614,808600,807793,808615,808601,807794,808616,808602,807795,808617,807781,808591,807782,808589,807783,808590,807785,808592,807784,808593,807786,808594,807788,808595,807787],"lang":"en"}
    

And it's a POST to that URL so I'm not sure why it's timing out.

Can anyone shed any light on where I'm going wrong here? Is it something
painfully obvious?


Answer: The payload should be included in request body rather than url params. The
payload in this case is a json raw string.

    
    
    import requests
    url = 'https://sports.betway.com/emoapi/emos'
    data = '{"eventIds": [807789]}'
    response = requests.post(url, data=data )
    print response.text
    

