Hello readers! In this tutorial, we’ll be dealing with how we can effectively save data in Python.
How to Save Data in Python?
When we’re working on Python applications, we’ll be dealing with Python objects directly, since everything is an object in Python. Let’s look at some ways by which we can store them easily!
1. Using Pickle to store Python Objects
If we want to keep things simple, we can use the pickle module, which is a part of the standard library to save data in Python.
We can “pickle” Python objects to a pickle file, which we can use to save/load data.
So if you have a custom object which you might need to store / retrieve, you can use this format:
import pickle
class MyClass():
def __init__(self, param):
self.param = param
def save_object(obj):
try:
with open("data.pickle", "wb") as f:
pickle.dump(obj, f, protocol=pickle.HIGHEST_PROTOCOL)
except Exception as ex:
print("Error during pickling object (Possibly unsupported):", ex)
obj = MyClass(10)
save_object(obj)
If you run this script, you’ll notice a file called data.pickle, which contains the saved data.
To load the same object back again, we could use pickle.load() using similar logic.
import pickle
class MyClass():
def __init__(self, param):
self.param = param
def load_object(filename):
try:
with open(filename, "rb") as f:
return pickle.load(f)
except Exception as ex:
print("Error during unpickling object (Possibly unsupported):", ex)
obj = load_object("data.pickle")
print(obj.param)
print(isinstance(obj, MyClass))
Output
10
True
We’ve just retrieved our old data successfully!
2. Using Sqlite3 to save data in Python persistently
If you want to use a persistent database to save data in Python, you can use the sqlite3 library which provides you APIs for using Sqlite databases.
Again, this is a part of the standard library, so there’s no need to pip install anything!
However, since this is a Relational Database, you can’t directly dump Python objects like in pickle.
You’d have to serialize and de-serialize them to their appropriate Database types.
To look at some examples, you can refer to this article on using sqlite in Python.
3. Using SqliteDict as a persistent cache
If you find using sqlite3 too tedious, there’s a much better solution! You can use sqlitedict for storing persistent data, and this internally uses an sqlite3 database to handle the storage.
You have to install this package using pip:
pip install sqlitedict
The only thing you need to keep in mind is that you need to use key:value mappings to store / retrieve data, just like a dictionary!
Here’s a very simple example using the MyClass instance.
from sqlitedict import SqliteDict
class MyClass():
def __init__(self, param):
self.param = param
def save(key, value, cache_file="cache.sqlite3"):
try:
with SqliteDict(cache_file) as mydict:
mydict[key] = value # Using dict[key] to store
mydict.commit() # Need to commit() to actually flush the data
except Exception as ex:
print("Error during storing data (Possibly unsupported):", ex)
def load(key, cache_file="cache.sqlite3"):
try:
with SqliteDict(cache_file) as mydict:
value = mydict[key] # No need to use commit(), since we are only loading data!
return value
except Exception as ex:
print("Error during loading data:", ex)
obj1 = MyClass(10)
save("MyClass_key", obj1)
obj2 = load("MyClass_key")
print(obj1.param, obj2.param)
print(isinstance(obj1, MyClass), isinstance(obj2, MyClass))
Output
10 10
True True
Indeed, we’ve just loaded our Python object successfully! If you notice, sqlitedict will create a database cache.sqlite3 automatically, if it doesn’t exist, and then use it to store / load data.
Conclusion
In this article, we looked at how we can use Python to store data in different ways.