The Python with
statement is a very useful. This has been there since Python 2.5, and it’s been an ever-present feature used by almost every Python application now!
What’s so useful about this statement, so much that everyone uses it?
The most useful thing (the only thing in fact!) it does is that it opens and frees up resources.
Basically, it handles opening and closing any resources that you may need to use for a specific part of the program, and automatically closes it afterwards.
Let’s look at this statement in a bit more detail, using some examples now.
Why do we Need Context Managers?
Consider the scenario where you do file handling. In other languages, like C, we must manually open and close the file like this:
# Open the file
file_obj = open('input.txt', 'r')
# File operations come here
...
# Manually close the file
file_obj.close()
The with
statement now abstracts this automatically for you, so that don’t need to close the file manually every-time!
The with
statement has a context (block), under which it acts on. This of this as the scope of the statement.
When the program comes out of this context, with
automatically closes your file!
Due to this, with
is often referred to as a Context Manager.
So the same file handling procedures can be used like this, along with the with
statement:
with open('input.txt', 'r') as file_obj:
...
Notice how this is very intuitive. The Python with
statement will always close the file at the end, even if the program terminated abnormally even within the context/block.
This safety feature makes it the accepted (and recommended) choice for all Python programmers!
Using the Python with statement
Now, while there are a lot of classes which have implemented a facility for using with
, we’re interested in looking at how it works, so that we can write one ourselves!
- First, the
with
statement stores an object reference in a context object.
A context object is an object that contains extra information about its state, such as the module/scope, etc. This is useful since we can save or restore the state of this object.
So, there is some meaning to keeping a reference to the object!
Now, let’s move on. Once the context object is created, it calls the __enter__
dunder method on the object.
- The
__enter__
statement is the one that actually does the work of opening resources for the object, such as a file/socket. Normally, we can implement it to save the context object state, if needed.
Now, remember the as
keyword? This actually returns the context object. Since we need the returned object by open(), we use the as
keyword to get the context object.
Using as
is optional, especially if you have a reference to the original context object somewhere else.
After this, we enter the nested block of statements.
Once the nested block has ended, OR, in case there is an Exception within this, the program always executes the __exit__
method on the context object!
This is the safety first feature we talked about earlier. So no matter what happens, we will always use __exit__
to free up resources and exit the context.
Finally, if feasible, __exit__
can be implemented so as to restore the context object state, so that it goes back to whatever state it belonged to.
Okay, that was quite a long explanation. To make it clearer, let’s look through an example of creating our own context manager for a Class.
Creating our own Context Managers for our Class
Consider the below class, for which we will have our own context manager for file handling.
class MyFileHandler():
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
# Originally, context object is None
self.context_object = None
# The context manager executes this first
# Save the object state
def __enter__(self):
print("Entered the context!")
self.context_object = open(self.filename, self.mode)
return self.context_object
# The context manager finally executes this before exiting
# Information about any Exceptions encountered will go to
# the arguments (type, value, traceback)
def __exit__(self, type, value, traceback):
print("Exiting the context....")
print(f"Type: {type}, Value: {value}, Traceback: {traceback}")
# Close the file
self.context_manager.close()
# Finally, restore the context object to it's old state (None)
self.context_object = None
# We're simply reading the file using our context manager
with MyFileHandler('input.txt', 'r') as file_handle:
for line in file_handle:
print(line)
Observe the class methods closely. We have the __init__
method for our handler, which sets the initial state of the Context objects and relevant variables.
Now, the __enter__
dunder method saves the object state and opens the file. Now, we are inside the block.
After the block executes, the context manager executes __exit__
finally, where the original state of the context object is restored, and the file is closed.
Okay, now let’s check our Output now. This should work!
Output
Entered the context!
Hello from AskPython
This is the second line
This is the last line!
Exiting the context....
Type: None, Value: None, Traceback: None
Alright, seems that we got no errors! We’ve just implemented our own context managers for our Custom Class.
Now, there is another approach of creating a Context Manager, which uses generators.
However, this is a bit hacky and is generally not recommended, unless you know exactly what you’re doing since you must handle exceptions yourself.
But, for the purpose of completeness, you can look at using this approach here. I’d recommend you to read this once you’re familiar with the Class-based approach.
Conclusion
In this article, we learned about using Context Managers in Python, using the with
statement.
References
- A wonderful article by preshing on Python context managers