Requests Module in Python

Introduction to requests module in python

In client-server architecture, we use HTTP very frequently to communicate between the client and the server. Typically, a client makes a request and the server returns some response. The request could be a request to get/ insert/ update some data. Similarly, the response can either return some data or some error. In some cases, the client undergoes authentication to ensure the request is valid. In this article, we shall see how to implement these HTTP requests and responses in Python using the requests module.

. We need to import the following module to get started:

Why learn Python Requests Module?

Learning the requests module will enable you to start making HTTP requests within a matter of writing a few lines of code. You do not need to worry about how to create the HTTP client, how to create the request object, how to deserialize the response object. A lot of the low level details are abstracted out to you using sub modules, classes and simple methods. As a web developer who builds apps using client-server architecture requests module is a handy tool to have in your toolbox

Making a Request

Let us see how an HTTP request is made in Python. We will use an open weather API that gives the current weather details at a particular latitude and longitude

Output

All Methods in the Requests Module

Requests module supports the following methods:

GET Request

HTTP GET method is used to retrieve data from server. GET methods can be invoked using the get() method of requests module.

Syntax

• url*: url of the API endpoint you want to send the request to

• params*: A dictionary used to send query params to the API endpoint

• args*: args can be used to specify some client level configs, such as Headers, authorization details, timeouts, etc

Let us say, you are building a cab booking app Rider. The very first requirement that you need to implement is fetching the users lat-long. Let us assume the following API endpoint exists to do the same:

We can convert the above cURL request to a Python requests module request. Assuming that the response is json, we can use the json() method on the response object provided by the Python requests module itself. This will help us parse the json

Output

POST Request

HTTP POST method is used to submit data to a server. A POST request can be executed by first creating a request object and using post() method of requests module

Syntax

• url*: url of the API endpoint you want to send the request to

• data*: This is an optional parameter that you can use in case you need to send a file object or a dictionary to the server

• json*: This is the json request payload to the server

Let us take our cab booking app Rider example again. Now that we have the lat long of a user, let us say we have the following POST API to request for a cab ride given the lat long details

Output

Response object

Requests module has a response object that is used for the following:

1. While fetching response from server
2. Knowing the HTTP status code of the response
3. If the response status is unsuccessful, getting the deserialized error message received from the server

All Response Methods

When you invoke a method of the request module, such as get() or post() the module returns a response object. The following summarizes the important methods of the response object

Authentication

Authentication means validating a client if they can access an API. This serves several purposes:

1. Limiting requests to an API
2. Filtering malicious requests

To implement authentication, we need to import the following module present in requests module:

The HTTPBasicAuth module is present in requests.auth and has classes and methods to detail with HTTP Authentication

An Authentication object needs to be created and passed to the get() body method using the 'Auth' key. We are trying to authenticate a user who is trying to login using a username and password. If the username and password entered by the client do not match the credentials present in the server's database, the server will throw a 403 forbidden error, else a 200 OK response

Syntax

Let us say we are not working on our cab booking app Rider's login page. Login API naturally, needs to be authenticated, the user's credentials will be passed to the server and the server will match them with the stored credentials in its database. We can check for authentication status using the status_code method

SSL Certificate Verification

Many websites are not on HTTPS which means the client needs to verify an SSL certificate for the client to access the resource. A website needs an SSL certificate to maintain user data security, verify ownership of the website, prevent attackers from creating a fake version of the site, and gain user trust

For this purpose the 'verify' key of the get method can be used

If our Rider APIs were hosted on an HTTPS secure server, clients would need to verify their identity using a certificate

The Session Object

HTTP is a stateless protocol. This means that there is no persistence of a request executed by a client, hence every request is independent of the other. However, there are few states a server-side application might need to store in a client-server architecture. For example:

1. Login state of a client: Once a user has logged in the login state shall be preserved for some time
2. Cookies generated by the server: Cookies are used by servers to identify requests coming from the same web browser, for faster rendering of pages
3. Request Id for a request, for caching purposes where the server does not want to call a database but instead server the data from caches whenever a client hits a request to a server, the server-generated
4. Advertising ID of a user to understand the browsing preferences of the user

For these use cases, we can use a Session object

For example, in our Rider app, once a user logs in, we would like to store the login state, so that when any user switches to some another app and comes back to Rider, they are not logged out

Syntax

All variables in the requests module