Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages
wb_sunny

Python json Module

Json Module

Before we dive into the Python JSON module, let’s understand what JSON is. The JSON (JavaScript Object Notation) is a standardized format that allows exchanging of data across the internet.

Since this has become the standard for any information exchange through the internet, it makes sense for any Python application to send and receive data using this format.

Python’s built-in json module is the interface that converts Python objects into JSON objects.

In this tutorial, let’s take a look at some of the most commonly used methods in the json module.


Format of a JSON object

Before going into the module details, let’s understand what a JSON object consists of.

This is actually very similar to a Python dictionary, where you have a set of {Key: value} pairs. The only small difference is that a JSON object has an opening and closing curly parenthesis.

Given below is a simple example of a JSON object

{
    "name": "John",
    "age": 42,
    "married": True,
    "qualifications": ["High School Diploma", "Bachelors"]
}

The JSON object can consist of various attributes, including strings, integers or even lists.

Now that we know what a JSON object is made of, let’s look at the Python json module’s methods.


Import the Python json module

Python already has the json module ready with it, so there is no need to install using pip.

To import this module, simply type

import json

json.dumps() – Construct a JSON object

We can encode a Python object into a JSON object using the json.dumps() method.

You can think of dumps() as serializing the Python object into a Python JSON object and returning a string. This is needed if you wish to transfer data across the internet.

The encoded data is mentioned in the below table, for different Python objects.

PythonJSON
dictobject
list, tuplearray
strstring
int, float, int– & float-derived Enumsnumber
Truetrue
Falsefalse
Nonenull

This takes any Python object that can be serialized as an argument and returns a string.

Format:

json_object = json.dumps(serializable_object)

Here, serializable_object is a Python object, such as a list, string, etc, that can be serializable. It cannot be a function/lambda, etc.

import json

python_object = ['Hello', 'from', 'AskPython', 42]

json_object = json.dumps(python_object)

print(type(json_object), json_object)

Output

<class 'str'> ["Hello", "from", "AskPython", 42]

This method will raise a TypeError if the object is not serializable.

>>> import json
>>> a = lambda x : x * 2
>>> a(2)
4
>>> json.dumps(a)
Traceback (most recent call last):
    raise TypeError(f'Object of type {o.__class__.__name__}
TypeError: Object of type function is not JSON serializable

Sorting Keys of a Dict

If we’re passing a Python dictionary into json.dumps(), we can specify another parameter sort_keys, which will make the Python json object have sorted keys.

import json

dict_obj = {1:"one", 20: "twenty", 5:"five"}

json_obj = json.dumps(dict_obj, sort_keys = True)

print(json_obj)

Output

{"1": "one", "5": "five", "20": "twenty"}

Our output indeed has sorted keys.

NOTE: the numbers are converted into strings since it is encoded into JSON. It will be properly deserialized back into integers on using appropriate methods.

Pretty Printing Python JSON objects

We can use the indent parameter of json.dumps() to specify the indentation level. Usually, indent = 4 will make the output look really good.

import json

dict_obj = {1:"one", 20: "twenty", 5:"five"}

json_obj = json.dumps(dict_obj, sort_keys = True, indent = 4)

print(json_obj)

Output

{
    "1": "one",
    "5": "five",
    "20": "twenty"
}

json.dump() – Dump into a file

We can also dump an object into a file, if you wish to use it later, using another method json.dump().

Format:

json.dump(data, file_object)

The json.dump() method takes in data and writes it to a file object.

So you can open a new file and write to that file object using json.dump()

import json

python_object = ['Hello', 'from', 'AskPython', 42]

with open("sample.json", "w") as wf:
    json.dump(python_object, wf)

Output

user@AskPython $ cat sample.json
["Hello", "from", "AskPython", 42]

As you can see, the Python object was indeed dumped to the file.

Now, let’s take that JSON object we showed in the first example and store it into a file.

import json

json_object = {
    "name": "John",
    "age": 42,
    "married": True,
    "qualifications": ["High School Diploma", "Bachelors"]
}

with open("sample.json", "w") as wf:
    json.dump(json_object, wf)

Output

user@AskPython $ cat sample.json
{"name": "John", "age": 42, "married": true, "qualifications": ["High School Diploma", "Bachelors"]}

Deserialize JSON objects

Similar to encoding a Python object into a JSON object, we can also do the reverse, by converting a JSON object into a Python object. This is called deserialization.

We can do this using the methods json.loads() and json.load(), similar to json.dumps() and json.dump().

json.loads()

This converts a json object encoded using json.dumps() back into a Python object.

import json

python_object = ['Hello', 'from', 'AskPython', 42]

encoded_object = json.dumps(python_object)

decoded_object = json.loads(encoded_object)

print(type(decoded_object), decoded_object)

Output

<class 'list'> ['Hello', 'from', 'AskPython', 42]

We have successfully obtained our old list object back!

json.load() – Deserialize from a file

This performs the reverse operation of json.dump(), by converting the json object back from a file, into a Python object.

Let’s take our sample.json file, and get back the data using this method.

import json

with open("sample.json", "r") as rf:
    decoded_data = json.load(rf)

print(decoded_data)

Output

{'name': 'John', 'age': 42, 'married': True, 'qualifications': ['High School Diploma', 'Bachelors']}

Indeed, we have again got back our old JSON object, that we stored in the file!

Now that we’ve covered the most commonly used methods of this module, let’s go to the next step: Creating our own JSON Encoder!


Create our own JSON Encoder

The json module uses an encoder called json.JSONEncoder, which uses the rules in the table given above to encode Python objects.

However, it does not encode all Python objects and depending on the problem we face, we may need to write our own JSON Encoder to encode those objects in a special way.

To do that, we must write our custom Encoder Class. Let’s call it MyEncoder. This must extend the json.JSONEncoder class, to add to its existing features.

For this demonstration, we’ll take numpy arrays and convert them to Python JSON objects. Now the json module by default cannot handle numpy arrays so if you try to convert a numpy array without our extended class, you’ll get a TypeError:

TypeError: Object of type ndarray is not JSON serializable

Let’s write this class to serialize and encode a numpy array into json objects as well, by converting it into a Python list, in our default() handler method.

import json
import numpy as np

class MyEncoder(json.JSONEncoder):
    # Handles the default behavior of
    # the encoder when it parses an object 'obj'
    def default(self, obj):
        # If the object is a numpy array
        if isinstance(obj, np.ndarray):
            # Convert to Python List
            return obj.tolist()
        else:
            # Let the base class Encoder handle the object
            return json.JSONEncoder.default(self, obj)


# Numpy array of floats
a = np.arange(1, 10, 0.5)
print(type(a), a)

# Pass our encoder to json.dumps()
b = json.dumps(a, cls=MyEncoder)
print(b)

We finally encode it, by passing the class name to the cls parameter of json.dumps().

So, the encoding call will be:

json_object = json.dumps(python_object, cls=MyEncoder)

Output

<class 'numpy.ndarray'> [1.  1.5 2.  2.5 3.  3.5 4.  4.5 5.  5.5 6.  6.5 7.  7.5 8.  8.5 9.  9.5]
[1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5]

Indeed, our custom encoder can now convert numpy arrays into JSON objects! We have now completed our first complex encoder.

You can extend this functionality to write different encoders for your specific use case!


Conclusion

In this article, we learned how we can use Python’s json module to do various operations involving JSON objects.


References