We often come across a situation where we would need to store or transfer objects. The pickle
module in Python is one such library that serves the purpose of storing Python objects as a serialized byte sequence into files for retrieval at a later time. Let us look into what it exactly does in this article.
1. Python Pickle Module Examples
Let’s look into some examples of using the pickle module in Python.
1.1) ‘pickling’ into a file
Since a file consists of bytes of information, we can transform a Python object into a file through the pickle
module. This is called pickling. Let us look at how we could do this through an example.
To do this operation of serializing an object into a file, we use the pickle.dump()
method.
Format: pickle.dump(data_to_store, filename, protocol_type)
data_to_store
-> The object to be serialized (pickled)filename
-> The name of the file where the data is storedprotocol_type
-> The type of the protocol used (is set to 4 by default in Python 3.8)
Here is an example to illustrate the same.
import pickle
data = ['one', 2, [3, 4, 5]]
with open('data.dat', 'wb') as f:
pickle.dump(data, f)
1.2) ‘Unpickling’ from a file
This is just the opposite of pickling, wherein the object is retrieved from the file. The file, which contains the serialized information of the object as a byte sequence, is now deserialized into the Python object itself, and we can get the original information back.
To perform this operation, we use the pickle.load()
library function.
Format: new_object = pickle.load(filename)
new_object
-> The object into which the method stores the information intofilename
-> The file containing the serialized information
import pickle
objdump = None
with open('data.dat', rb') as f:
# Stores the now deserialized information into objdump
objdump = pickle.load(f)
2. Exception Handling with pickle Module
The Pickle module defines some Exceptions, which are useful for programmers or developers to handle different scenarios and debug them appropriately.
The module mentions that the following can be pickled:
None
,True
,False
- integers, floating points, complex numbers
- strings, bytes, byte arrays
- tuples, lists, sets, and dictionaries containing only picklable objects
- Named functions defined at the top level of a module
- Classes and built-in functions defined at the top level of a module
Any other object is not picklable, and is called unpicklable.
There are 3 primary exceptions that the module defines, namely:
Exception Name | When is this exception raised? |
pickle.PickleError | This is just the base class for the other exceptions. This inherits Exception |
pickle.PicklingError | Raised when an unpicklable object is encountered. |
pickle.UnpicklingError | Raised during unpickling of an object, if there is any problem (such as data corruption, access violation, etc) |
Here is an example of using exception handling to handle pickle.PicklingError
, when trying to pickle an unpicklable object.
import pickle
# A lambda is unpicklable
data = ['one', 2, [3, 4, 5], lambda l: 1]
with open('data2.dat', 'wb') as f:
try:
pickle.dump(data, f)
except pickle.PicklingError:
print('Error while reading from object. Object is not picklable')
Output
Error while reading from object. Object is not picklable
Here is an example of using exception handling to handle pickle.UnpicklingError
, when trying to unpickle a non serialized file.
import pickle
with open('data1.dat', 'wb') as f:
f.write('This is NOT a pickled file. Trying to unpickle this will cause an exception')
objdump = None
with open('data1.dat', 'rb') as f:
try:
objdump = pickle.load(f)
except pickle.UnpicklingError:
print('Cannot write into object')
Output
Cannot write into object
3. Problems faced in pickling and unpickling
- As the module states in the documentation, it provides us a stern warning regarding pickling and unpickling of objects files. Do not use this module for unpickling unless you absolutely trust the source, since any kind of malicious code can be injected into an object file.
- Also, there may be problems faced due to lack of compatibility between Python language versions, since the data structures may differ from version to version, and hence
Python 3.0
may not be able to unpickle a pickled file fromPython 3.8
.
- There is also no cross-language compatibility, which may serve to be an annoyance for non-Python data transfers. The information is only Python-specific.
4. Conclusion
Here, we learned more about the pickle
module, which can be used to serialize/deserialize Python objects to/from files. It is a quick and easy way to transfer and store Python objects, which helps programmers to store data easily and quickly for data transfer.
5. References
- JournalDev article on pickle: https://www.journaldev.com/15638/python-pickle-example
- Pickle module Documentation: https://docs.python.org/3/library/pickle.html