Python Requests Library

• Getting started with Requests
• Python requests: GET
• Reading responses
• Using Python request headers
• Python requests: POST

The Requests module is one widely used to send HTTP requests. It’s a third-party alternative to the standard urllib, urllib2, and urllib3 as they can be confusing and often need to be used together. Requests in Python greatly simplifies sending HTTP requests to their destination.

This article gives you an overview of everything you need about Requests.

For a detailed explanation, see our blog post.

Getting started with Requests

Installing Requests is simple as it can be done through a terminal.

$ pip install requests

Finally, before beginning to use Requests in any project, the library needs to be imported:

import requests

Python requests: GET

To send a GET request, invoke requests.get() in Python and add a destination URL, as follows:

import requests
requests.get('http://httpbin.org/')

GET requests can be sent with specific parameters if required:

payload = {'key1': 'value1', 'key2': 'value2'}
requests.get('http://httpbin.org/', params=payload)

Reading responses

response = requests.get('http://httpbin.org/')
print(response.status_code)

# OUTPUT: <Response [200]>

To read the content of the response, we need to access the text part by using response.text:

print(response.text)

Responses can also be decoded to the JSON format:

response = requests.get('http://api.github.com')
print(response.json())

Using Python request headers

Response headers are another important part of the request:

print(response.headers)

You can also send custom Python request headers. To check whether our request header has been sent successfully, we will need to make the call response.request.headers:

import requests

headers = {'user-agent': 'my-agent/1.0.1'}
response = requests.get('http://httpbin.org/', headers=headers)
print(response.request.headers)

Python requests: POST

Sending a POST request is almost as simple as sending a GET:

response = requests.post('https://httpbin.org/post', data = {'key':'value'})

The requests library accepts arguments from dictionary objects which can be utilized to send more advanced data:

payload = {'key1': 'value1', 'key2': 'value2'}
response = requests.post('https://httpbin.org/post', data = payload)

Requests have an added feature that automatically converts the POST request data into JSON.

import requests

payload = {'key1': 'value1', 'key2': 'value2'}
response = requests.post('https://httpbin.org/post', json = payload)
print(response.json())

Alternatively, the json library might be used to convert dictionaries into JSON objects:

import json
import requests

payload = {
    'key1': 'value1',
    'key2': 'value2'}
jsonData = json.dumps(payload)
response = requests.post('https://httpbin.org/post', data = jsonData)
print(response.json())

If you wish to learn more about Requests, see our blog post.