import requests

# get request
r = requests.get('https://api.github.com/events')
r.url

# post request
r = requests.post('http://httpbin.org/post', data={'name':'john'})
r.status_code

# put request
r = requests.put('http://httpbin.org/put',data={'name':'john doe'})
r.status_code

# delete request
r = requests.delete('http://httpbin.org/delete')
r.status_code

# header request
r = requests.head('http://httpbin.org/get')
r.content

"""
Explanation: Python - Requests Library
Requests is an elegant and simple HTTP library for Python built for human beings.
Installation
Using pipenv
sh
pipenv install requests
Using Source Code
```sh
$ git clone git://github.com/requests/requests.git
$ curl -OL https://github.com/requests/requests/tarball/master
$ cd requests
$ pip install .
```
Make a request
End of explanation
"""


# passing params
payload={'name':'john','age':12}
r = requests.get('http://httpbin.org/get',params=payload)
r.url

"""
Explanation: Passing Parameters in Urls
End of explanation
"""


r = requests.get('http://en.wikipedia.org/wiki/Monty_Python')

# access the headers the server sent back to us
r.headers

# headers we sent to the server
r.request.headers

"""
Explanation: Whenever a call is made to requests.get() first a Request object is constructed and a response object is generated once Requests gets a response back from the server. The Response object contains all of the information returned by the server and also contains the Request object you created originally.
End of explanation
"""


s = requests.Session()

req = requests.request('POST',url,data=data,headers=headers)
prepped = req.prepare()

# do something with prepped body
prepped.body = 'No I want exactly this as the body'

# do something with prepped.headers
del prepped.headers['Content-Type']

resp = s.send(prepped,stream=stream,verify=verify,proxies=proxies,cert=cert,timeout=timeout)

print(resp.status_code)

"""
Explanation: Prepared Requests
Whenever we receive a Response object from an API call or a Session call, the request attribute is actually the PreparedRequest that was used. In some cases you may wish to do some extra work to the body or headers before sending a request.
End of explanation
"""


s = requests.Session()

req = requests.request('GET',url,data=data,headers=headers)
prepped = s.prepare_request(req)

# do something with prepped body
prepped.body = 'No I want exactly this as the body'

# do something with prepped.headers
del prepped.headers['Keep-Deatd'] = 'parrot'

resp = s.send(prepped,stream=stream,verify=verify,proxies=proxies,cert=cert,timeout=timeout)

print(resp.status_code)

"""
Explanation: However the above code will lose some of the advantages of having a Requests Session object. In particular, Session-level state such as cookies will not get applied to your request. To get a PreparedRequest with that state applied, replace the call to Request.prepare() with a call to Session.prepare_request() like this:
End of explanation
"""


s = requests.Session()

req = requests.request('POST',url,data=data,headers=headers)
prepped = s.prepare_request(req)

# do something with prepped body
prepped.body = 'No I want exactly this as the body'

# Merger environment settings into session
settings = s.merge_environment_settings(prepped.url,None,None,None,None)

resp = s.send(prepped, **settings)

print(resp.status_code)

"""
Explanation: When you are using the prepared request flow, keep in mind that it does not take into account the environment. This can cause problems if you are using environment variables to change the behavior of requests. For example Self-signed SSL certificates specified in REQUESTS_CA_BUNDLE will not taken into account. You can get around this behavior by explicitly merging the environment settings into the session:
End of explanation
"""


# get request
r = requests.get('https://api.github.com/events')
print(r.text)

# get the encoding
r.encoding

"""
Explanation: Response Content
Text Response Content
End of explanation
"""


# get request
r = requests.get('https://api.github.com/events')
r.json()

"""
Explanation: Binary Response Content
You can also access the response body as bytes for non text-requests. The gzip and deflate transfer-encodings are automatically decoded for you. For example to create an image from binary data returned by a request you can use the following code:
```python
from PIL import Image
from io import BytesIO
i = Image.open(BytesIO.(r.content))
```
JSON Response Content
End of explanation
"""


# get request
r = requests.get('https://api.github.com/events',stream=True)
r.raw.read(10)

"""
Explanation: In case the JSON decoding fails, r.json() raises an exception. For example, if the response gets a 204(No Content) or the response contains invalid JSON, attempting r.json() raises
python
ValueError: No JSON object could be decoded
It should also be noted that the success of the call to r.json() does not indicate the success of the response. Some servers may return a JSON object in a failed response. Such JSON will be decoded and returned. To check that a request is successful use r.raise_for_status() or check r.status_code is what you expect.
Raw response content
End of explanation
"""


# get request
url = 'https://api.github.com/events'
headers = {'user-agent':'app-request'}
r = requests.get(url,headers)
r.headers

"""
Explanation: In general however you should use a pattern like this to save what is being streamed to file.
python
with open(filename,'wb') as fd:
    for chunk in r.iter_content(chunk_size=128):
        fd.write(chunk)
Using Response.iter_content will handle a lot of what you would otherwise have to handle when using Response.raw directly. When streaming a download, the above is the preferred and recommended way to retrieve the content. Note that chunk_size can be freely adjusted to a number that may better fit your use cases.
Custom Headers
If you'd like to add HTTP headers to a request, simply pass in a dict to the headers parameter.
For example we did n't specify the user-agent in the previous example
End of explanation
"""


# using form-encoded data with a dictionary
payload = {'name':'john','age':12}
r = requests.post('https://httpbin.org/post',data=payload)
print(r.text)

"""
Explanation: Note that the custom headers are given less precedence than more specific sources of information.
* Authorization headers set with headers= will be overridden if credentials are specified in .netrc which will be overridden by the auth= parameter.
* Authorization headers will be removed if you get redirected off-host.
* Proxy-Authorization headers will be overridden by proxy credentials provided in the URL
* Content-Length headers will be overridden when we can determine the length of the content.
Advanced POST requests
Typically, you want to send some form-encoded data much like an HTML form. To do this, simply pass a dictionary to the data argument. You dictionary of data will automatically be form-encoded when the request is made.
End of explanation
"""


# using form-encoded data with a tuple
payload = (('name','john'),('age',12))
r = requests.post('https://httpbin.org/post',data=payload)
print(r.text)

# using json
import jsonpayload = {'name':'john','age':12}
r = requests.post('https://httpbin.org/post',data=payload)
print(r.text)
payload = {'name':'john','age':12}
r = requests.post('https://httpbin.org/post',data=json.dumps(payload))
print(r.text)

payload = {'name':'john','age':12}
r = requests.post('https://httpbin.org/post',json=payload)
print(r.text)

"""
Explanation: You can also pass a list of tuples to the data argument. This is particularly useful when the form has multiple elements that use the same key:
End of explanation
"""


url = 'http://httpbin.org/post'
files = {'file':open('test.xls','rb')}

r = requests.post(url,files=files)
r.text

# set filename, content-type and headers explicitly
url = 'http://httpbin.org/post'
files = {'file':('report.xls',open(),'application/vnd.ms-excel',{'Expires':'0'})}
r = requests.post(url,files=files)

"""
Explanation: Post a Multipart - Encoded File
End of explanation
"""


url = 'http://httpbin.org/post'
multiple_files = [
    ('images',('foo.png',open('foo.png','rb'),'image/png')),
    ('images',('bar.png',open('foo.png','rb'),'image/png'))
]
r = requests.post(url,file=multiple_files)
r.text

"""
Explanation: In the event you are posting a very large file as multipart/form-data request you may want to stream the request. By default requests does not support this but there is a separate package which does -requests-toolbelt.
Post Multiple Multipart-Encoded Files
You can send multiple files in one request. For example, suppose you want to upload image files to an HTML form with a multiple file field 'images':
End of explanation
"""


r = requests.get('http://httpbin.org/get')
r.status_code

# built in status code objects
r.status_code == requests.codes.ok

# bad request, raise exception
bad_r = requests.get('http://httpbin.org/status/404')
bad_r.status_code

bad_r.raise_for_status()

r.raise_for_status()

"""
Explanation: Response Status Codes
We can check the response status code:
End of explanation
"""


# get the headers
r.headers

# access headers
r.headers['Content-Type']

# access headers using get
r.headers.get('content-type')

"""
Explanation: Response Headers
End of explanation
"""


url = 'http://example.com/some/cookie/setting/url'
r = requests.get(url)
r.cookies['example-cookie-name']

# send cookies
url = 'http://httpbin.org/cookies'
cookies = dict(cookies_are='working')
r = requests.get(url,cookies=cookies)
r.text

"""
Explanation: Cookies
End of explanation
"""


jar = requests.cookies.RequestsCookieJar()
jar.set('tasty-cookie','yum',domain='httpbin.org',path='/cookies')
url = 'http://httpbin.org/cookies'
r = requests.get(url,cookies=jar)
r.text

r.cookies

"""
Explanation: Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths.
End of explanation
"""


r = requests.get('http://github.com')
r.url

r.status_code

r.history

"""
Explanation: Redirection and History
By default Requests will perform location redirection for all verbs except HEAD.
We can use the history property of the Response object to track redirection.
The Response.history list contains the Response objects that were created in order to complete the request. The list is sorted from the oldest to the most recent response.
End of explanation
"""


r = requests.get('http://github.com',allow_redirects=False)
r.status_code

r.history

"""
Explanation: If you are using GET,OPTIONS,POST,PUT,PATCH or DELETE you can disable redirection handling with the allow_redirects parameter.
End of explanation
"""


requests.get('http://github.com',timeout=0.001)

"""
Explanation: Timeouts
Most requests to externals servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more.
The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine call on the socket. It is a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.
Once your client has connected to the server and sent the HTTP request, the read timeout is the number of the seconds the client will wait for the server to send a response. Specifically its the number of seconds that the client will wait between bytes sent from a server. In 99.9% of cases, this is the time before the server sends the first byte).
End of explanation
"""


# single value timeout
# good for both 'connect' and 'read' timeouts
r = requests.get('https://github.com',timeout=5)

# separate value for 'connect' and 'readout' timeouts
r = requests.get('https://github.com',timeout=(3.05,27))

"""
Explanation: Note: timeout is not a time limit on the entire response download, rather an exception is raised when server has not issued a response for timeout seconds.If no timeout is specified explicitly requests do not timeout.
End of explanation
"""


r = requests.get('https://github.com',timeout=None)

"""
Explanation: If the remote server is very slow, you can tell Requests to wait forever for a response, by passing None as a timeout value and then retrieving a cup of coffee.
End of explanation
"""


# call back function - print url function
def print_url(r, *args, **kwargs):
    print(r.url)
    
requests.get('http://httpbin.org',hooks={'response':print_url})

# second callback function
def record_hook(r,*args,**kwargs):
    r.hook_called = True
    return r

# add multiple hooks to a single request
requests.get('http://httpbin.org',hooks={'response':[print_url,record_hook]})

r = requests.get('http://httpbin.org',hooks={'response':[print_url,record_hook]})
r.hook_called

"""
Explanation: Errors and Exception
In the event of a network problem, Requests will raise a ConnectionError exception
Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code.
If a request times out, a Timeout exception is raises.
If a request exceeds the configured the number of maximum redirections a TooManyRedirects exception is raised.
All exceptions that Requests explicitly raises inherit from
requests.exceptions.RequestException
Event Hooks
Requests has a hook system that you can use to manipulate portions of the request process or signal event handling.
Available Hooks:
* response - response generated from a request
You can assign a hook function on a per-request basis by passing a {hook_name:callback_function} dictionary to the hooks request parameter.
End of explanation
"""


# proxies = {'http':'http://10.10.1.10:3128','https':'http://10.10.1.10:1080'}
# requests.get('http://example.org',proxies = proxies)

"""
Explanation: Proxies
If you need to use a proxy, you can configure individual requests with the proxies argument to any request method:
End of explanation
"""


import json
import requests

r = requests.get('http://httpbin.org/stream/5',stream=True)

for line in r.iter_lines():
    # filter out keep-alive new lines
    if line:
        decoded_line = line.decode('utf-8')
        print(json.loads(decoded_line))

"""
Explanation: To use HTTP Basic Auth with your proxy use the http://user:password@host/syntax
To give a proxy for a specific scheme and host use the scheme://hostname form for the key. This will match for any request to the given scheme and hostname.
proxies={'http://10.20.1.18':'http://10.20.1.292:5342'}
Chunk Encoded Requests
Requests also supports Chunked transfer encoding for outgoing and incoming requests. To send a chunk-encoded request, simply provide a generator (or any iterator without a length) for your body.
python
def gen():
    yield 'hi'
    yield 'there'
requests.post('http://some.url/chunked',data=gen())
For chunked encoded repsonses its best to iterate over the data using Response.iter_content(). In an ideal situation you'll have set stream=True on the request, in which case you can iterate chunk-by-chunk by calling iter_content with a chunk_size parameter of None. If you want to set a maximum size of the chunk, you can set a chunk_size parameter to any integer.
Streaming Requests
With Response.iter_lines() you can easily iterate over streaming APIs such as Twitter Streaming API. Simply set to Stream to True and iterate over the response with iter_lines:
End of explanation
"""


r = requests.get('http://httpbin.org/stream/5', stream=True)

if r.encoding is None:
    r.encoding = 'utf-8'
    
for line in r.iter_lines(decode_unicode=True):
    if line:
        print(json.loads(line))

"""
Explanation: When using decode_unicode=True with Response.iter_lines() or Response.iter_content(), provide a fallback encoding in the event the server does n't provide one:
End of explanation
"""


s = requests.Session()
s.get('http://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('http://httpbin.org/cookies')

print(r.text)

"""
Explanation: Session Objects
The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance and will use urllib3's connection pooling. So if you are making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase.
End of explanation
"""


s = requests.Session()
s.auth = ('user','pass')
s.headers.update({'x-test':'true'})

# both x-test and x-test are send
s.get('http://httpbin.org/headers', headers={'x-test-2':'true'})

"""
Explanation: Sessions can also be used to provide default data to the request methods. This is done by providing data to the properties on a Session object:
End of explanation
"""


s = requests.Session()

r = s.get('http://httpbin.org/cookies',cookies={'from-my':'browser'})
print(r.text)

r = s.get('http://httpbin.org/cookies')
print(r.text)

"""
Explanation: Any dictionaries you pass to a request method will be merged with the session-level values that are set. The method level parameters override the session parameters.
Note however that method level parameteres will not be persisted across requests even if using a session. The following example will only send the cooking with the first request but not the second:
End of explanation
"""


