Python-Requests, extract url parameters from a string
Question: I am using this awesome library called [`requests`](http://docs.python-
requests.org/en/latest) to maintain python 2 & 3 compatibility and simplify my
application requests management.

I have a case where I need to parse a url and replace one of it's parameter.
E.g:

    
    
    http://example.com?param1=a&token=TOKEN_TO_REPLACE&param2=c
    

And I want to get this:

    
    
    http://example.com?param1=a&token=NEW_TOKEN&param2=c
    

With the `urllib` I can achieve it this way:

    
    
    from urllib.parse import urlparse
    from urllib.parse import parse_qs
    from urllib.parse import urlencode
    
    url = 'http://example.com?param1=a&token=TOKEN_TO_REPLACE&param2=c'
    
    o = urlparse(url)
    query = parse_qs(o.query)
    if query.get('token'):
        query['token'] = ['NEW_TOKEN', ]
        new_query = urlencode(query, doseq=True)
        url.split('?')[0] + '?' + new_query
    
    >>> http://example.com?param2=c&param1=a&token=NEW_TOKEN
    

How can you achieve the same using the `requests` library?


Answer: You cannot use `requests` for this; the library **builds** such URLs if passed
a Python structure for the parameters, but does not offer any tools to parse
them. That's not a goal of the project.

Stick to the `urllib.parse` method to parse out the parameters. Once you have
a dictionary or list of key-value tuples, just pass that to `requests` to
build the URL again:

    
    
    try:
        # Python 3
        from urllib.parse import urlparse, parse_qs
    except ImportError:
        # Python 2
        from urlparse import urlparse, parse_qs
    
    o = urlparse(url)
    query = parse_qs(o.query)
    # extract the URL without query parameters
    url = o._replace(query=None).geturl()
    
    if 'token' in query:
        query['token'] = 'NEW_TOKEN'
    
    requests.get(url, params=query)
    

You can get both the `urlparse` and `parse_qs` functions in both Python 2 and
3, all you need to do is adjust the import location if you get an exception.

Demo on Python 3 (without the import exception guard) to demonstrate the URL
having been built:

    
    
    >>> from urllib.parse import urlparse, parse_qs
    >>> url = "http://httpbin.org/get?token=TOKEN_TO_REPLACE&param2=c"
    >>> o = urlparse(url)
    >>> query = parse_qs(o.query)
    >>> url = o._replace(query=None).geturl()
    >>> if 'token' in query:
    ...     query['token'] = 'NEW_TOKEN'
    ... 
    >>> response = requests.get(url, params=query)
    >>> print(response.text)
    {
      "args": {
        "param2": "c", 
        "token": "NEW_TOKEN"
      }, 
      "headers": {
        "Accept": "*/*", 
        "Accept-Encoding": "gzip, deflate", 
        "Host": "httpbin.org", 
        "User-Agent": "python-requests/2.5.1 CPython/3.4.2 Darwin/14.1.0"
      }, 
      "origin": "188.29.165.245", 
      "url": "http://httpbin.org/get?token=NEW_TOKEN&param2=c"
    }
    

