Hello readers! In this tutorial, we’ll be dealing with how we can effectively save data in Python.
How to Save Data in Python?
When we’re working on Python applications, we’ll be dealing with Python objects directly, since everything is an object in Python. Let’s look at some ways by which we can store them easily!
1. Using Pickle to store Python Objects
If we want to keep things simple, we can use the pickle module, which is a part of the standard library to save data in Python.
We can “pickle” Python objects to a pickle file, which we can use to save/load data.
So if you have a custom object which you might need to store / retrieve, you can use this format:
import pickle class MyClass(): def __init__(self, param): self.param = param def save_object(obj): try: with open("data.pickle", "wb") as f: pickle.dump(obj, f, protocol=pickle.HIGHEST_PROTOCOL) except Exception as ex: print("Error during pickling object (Possibly unsupported):", ex) obj = MyClass(10) save_object(obj)
If you run this script, you’ll notice a file called
data.pickle, which contains the saved data.
To load the same object back again, we could use
pickle.load() using similar logic.
import pickle class MyClass(): def __init__(self, param): self.param = param def load_object(filename): try: with open(filename, "rb") as f: return pickle.load(f) except Exception as ex: print("Error during unpickling object (Possibly unsupported):", ex) obj = load_object("data.pickle") print(obj.param) print(isinstance(obj, MyClass))
We’ve just retrieved our old data successfully!
2. Using Sqlite3 to save data in Python persistently
If you want to use a persistent database to save data in Python, you can use the
sqlite3 library which provides you APIs for using Sqlite databases.
Again, this is a part of the standard library, so there’s no need to pip install anything!
However, since this is a Relational Database, you can’t directly dump Python objects like in
You’d have to serialize and de-serialize them to their appropriate Database types.
To look at some examples, you can refer to this article on using sqlite in Python.
3. Using SqliteDict as a persistent cache
If you find using
sqlite3 too tedious, there’s a much better solution! You can use
sqlitedict for storing persistent data, and this internally uses an
sqlite3 database to handle the storage.
You have to install this package using pip:
pip install sqlitedict
The only thing you need to keep in mind is that you need to use
key:value mappings to store / retrieve data, just like a dictionary!
Here’s a very simple example using the
from sqlitedict import SqliteDict class MyClass(): def __init__(self, param): self.param = param def save(key, value, cache_file="cache.sqlite3"): try: with SqliteDict(cache_file) as mydict: mydict[key] = value # Using dict[key] to store mydict.commit() # Need to commit() to actually flush the data except Exception as ex: print("Error during storing data (Possibly unsupported):", ex) def load(key, cache_file="cache.sqlite3"): try: with SqliteDict(cache_file) as mydict: value = mydict[key] # No need to use commit(), since we are only loading data! return value except Exception as ex: print("Error during loading data:", ex) obj1 = MyClass(10) save("MyClass_key", obj1) obj2 = load("MyClass_key") print(obj1.param, obj2.param) print(isinstance(obj1, MyClass), isinstance(obj2, MyClass))
10 10 True True
Indeed, we’ve just loaded our Python object successfully! If you notice,
sqlitedict will create a database
cache.sqlite3 automatically, if it doesn’t exist, and then use it to store / load data.
In this article, we looked at how we can use Python to store data in different ways.