Python3 requests library to submit form that disallows post request
Question: I am trying to get the police district from a given location at the [Philly
Police webpage](https://www.phillypolice.com/districts/). I too many locations
to do this by hand, so I am trying to automate the process using Python's
requests library. The webpage's form that holds the location value is as
follows:

    
    
    <form id="search-form" method="post" action="districts/searchAddress">
    <fieldset>
        <div class="clearfix">
            <label for="search-address-box"><span>Enter Your Street Address</span></label>
            <div class="input">
                <input tabindex="1" class="district-street-address-input" id="search-address-box" name="name" type="text" value="">
            </div>
        </div>
        <div class="actions" style="float: left;">
            <button tabindex="3" type="submit" class="btn btn-success">Search</button>
        </div>
        <a id="use-location" href="https://www.phillypolice.com/districts/index.html?_ID=7&_ClassName=DistrictsHomePage#" style="float: left; margin: 7px 0 0 12px;"><i class="icon-location-arrow"></i>Use Current Location</a>
        <div id="current-location-display" style="display: none;"><p>Where I am right now.</p></div>
    </fieldset>
    </form>
    

However when I try to post or put to the webpage using the following:

    
    
    r = requests.post('http://www.phillypolice.com/districts',data={'search-address-box':'425 E. Roosevelt Blvd'})
    

I get error 405, POST is not allowed. I then turned off Javascript and tried
to find the district on the webpage, and when I hit submit I received the same
405 error message. Therefore the form is definitely not submitted and the
district is found using JavaScript.

Is there a way to simulate 'clicking' the submit button to trigger the
JavaScript using the requests library?


Answer: The data is retrieved after first querying google maps to the the coordinates
where the final request is a get like the following:

[![enter image description
here](http://i.stack.imgur.com/gBwqd.png)](http://i.stack.imgur.com/gBwqd.png)

You can setup a free account with the [bing maps
api](https://www.bingmapsportal.com/) and get the coords you need to make the
get request:

    
    
    import requests
    
    key = "my_key"
    coord_params = {"output": "json",
                    "key": key}
    
    # This provides the coordinates.
    coords_url = "https://dev.virtualearth.net/REST/v1/Locations"
    
    # Template to pass each address to in your actual loop.
    template = "{add},US"
    url = "https://api.phillypolice.com/jsonservice/Map/searchAddress.json"
    with requests.Session() as s:
        # Add the query param passing in each zipcode
        coord_params["query"] = template.format(add="425 E. Roosevelt Blvd")
        js = s.get(coords_url, params=coord_params).json()
        # Parse latitude and longitude from the returned json.
        # Call str to make make it into `(lat, lon)`
        latitude_longitude = str((js[u'resourceSets'][0][u'resources'][0]["point"][u'coordinates']))
        data = s.get(url, params={"latlng": latitude_longitude})
    
        print(data.json())
    

If we run it minus my key:

    
    
    In [2]: import requests
       ...: 
       ...: key = "my_key..."
       ...: 
       ...: coord_params = {"output": "json",
       ...:                 "key": key}
       ...: coords_url = "https://dev.virtualearth.net/REST/v1/Locations"
       ...: template = "{add},US"
       ...: url = "https://api.phillypolice.com/jsonservice/Map/searchAddress.json"
       ...: with requests.Session() as s:
       ...:     coord_params["query"] = template.format(add="425 E. Roosevelt Blvd")
       ...: 
       ...:     js = s.get(coords_url, params=coord_params).json()
       ...:     latitude_longitude = str(js[u'resourceSets'][0][u'resources'][0]["po
       ...: int"][u'coordinates'])
       ...:     print(latitude_longitude)
       ...:     data = s.get(url, params={"latlng": latitude_longitude})
       ...:     print(data.json())
       ...:     
    [40.02735900878906, -75.1153564453125]
    {'response': ['35', '2', 'Marques Newsome', 'PPD.35_PSA2@phila.gov ', '267-357-1436']}
    

You can see it matches the response you see if you look at the request in your
browser.

