The pickle Module in Python

We often come across a situation where we would need to store or transfer objects. The pickle module in Python is one such library that serves the purpose of storing Python objects as a serialized byte sequence into files for retrieval at a later time. Let us look into what it exactly does in this article.

1. Python Pickle Module Examples

Let’s look into some examples of using the pickle module in Python.

1.1) ‘pickling’ into a file

Since a file consists of bytes of information, we can transform a Python object into a file through the pickle module. This is called pickling. Let us look at how we could do this through an example.

To do this operation of serializing an object into a file, we use the pickle.dump() method.

Format: pickle.dump(data_to_store, filename, protocol_type)

  • data_to_store -> The object to be serialized (pickled)
  • filename -> The name of the file where the data is stored
  • protocol_type -> The type of the protocol used (is set to 4 by default in Python 3.8)

Here is an example to illustrate the same.

import pickle

data = ['one', 2, [3, 4, 5]]

with open('data.dat', 'wb') as f:
    pickle.dump(data, f)

1.2) ‘Unpickling’ from a file

This is just the opposite of pickling, wherein the object is retrieved from the file. The file, which contains the serialized information of the object as a byte sequence, is now deserialized into the Python object itself, and we can get the original information back.

To perform this operation, we use the pickle.load() library function.

Format: new_object = pickle.load(filename)

  • new_object -> The object into which the method stores the information into
  • filename -> The file containing the serialized information
import pickle

objdump = None

with open('data.dat', rb') as f:
    # Stores the now deserialized information into objdump
    objdump = pickle.load(f)

2. Exception Handling with pickle Module

The Pickle module defines some Exceptions, which are useful for programmers or developers to handle different scenarios and debug them appropriately.

The module mentions that the following can be pickled:

  • None, True, False
  • integers, floating points, complex numbers
  • strings, bytes, byte arrays
  • tuples, lists, sets, and dictionaries containing only picklable objects
  • Named functions defined at the top level of a module
  • Classes and built-in functions defined at the top level of a module

Any other object is not picklable, and is called unpicklable.

There are 3 primary exceptions that the module defines, namely:

Exception NameWhen is this exception raised?
pickle.PickleErrorThis is just the base class for the other exceptions. This inherits Exception
pickle.PicklingErrorRaised when an unpicklable object is encountered.
pickle.UnpicklingErrorRaised during unpickling of an object, if there is any problem (such as data corruption, access violation, etc)

Here is an example of using exception handling to handle pickle.PicklingError, when trying to pickle an unpicklable object.

import pickle

# A lambda is unpicklable
data = ['one', 2, [3, 4, 5], lambda l: 1]

with open('data2.dat', 'wb') as f:
    try:
        pickle.dump(data, f)
    except pickle.PicklingError:
        print('Error while reading from object. Object is not picklable')

Output

Error while reading from object. Object is not picklable

Here is an example of using exception handling to handle pickle.UnpicklingError, when trying to unpickle a non serialized file.

import pickle

with open('data1.dat', 'wb') as f:
    f.write('This is NOT a pickled file. Trying to unpickle this will cause an exception')

objdump = None
with open('data1.dat', 'rb') as f:
    try:
        objdump = pickle.load(f)
    except pickle.UnpicklingError:
        print('Cannot write into object')

Output

Cannot write into object

3. Problems faced in pickling and unpickling

  • As the module states in the documentation, it provides us a stern warning regarding pickling and unpickling of objects files. Do not use this module for unpickling unless you absolutely trust the source, since any kind of malicious code can be injected into an object file.
  • Also, there may be problems faced due to lack of compatibility between Python language versions, since the data structures may differ from version to version, and hence Python 3.0 may not be able to unpickle a pickled file from Python 3.8.
  • There is also no cross-language compatibility, which may serve to be an annoyance for non-Python data transfers. The information is only Python-specific.

4. Conclusion

Here, we learned more about the pickle module, which can be used to serialize/deserialize Python objects to/from files. It is a quick and easy way to transfer and store Python objects, which helps programmers to store data easily and quickly for data transfer.


5. References