Python IO Module: The Complete Practical Reference

By /
Python Io Module

Hello everyone! In today’s article, we’ll be looking at using the Python IO Module.

This module is quite useful when you want to perform file-related I/O operations (eg. file reading/writing)

While you can use the normal read() and write() methods to read/write to a file, this module gives us a lot more flexibility regarding these operations.

To understand more about this module, let’s take some examples.

Python IO Module

This module is a part of the standard library, so there’s no need to install it separately using pip.

To import the io module, we can do the following:

import io

In the io module there are 2 common classes which are very useful for us:

  • BytesIO -> I/O operations on byte data
  • StringIO -> I/O operations on string data

We can access these classes using io.BytesIO and io.StringIO.

Let’s take a look at them one by one.

Python BytesIO Class

Here, we can keep our data in the form of bytes (b''). When we use io.BytesIO, the data is held in an in-memory buffer.

We can get an instance to the byte stream using the constructor:

import io
bytes_stream = io.BytesIO(b'Hello from Journaldev\x0AHow are you?')

Notice that we are passing a byte string (prefixed using b).

Right now, bytes_stream is simply a handle to the byte stream.

To actually print the data inside the buffer, we need to use bytes_stream.getvalue().

import io
bytes_stream = io.BytesIO(b'Hello from Journaldev\x0AHow are you?')
print(bytes_stream.getvalue())

Here, getvalue() takes the value of the byte string from the handle.

Since the byte string \x0A is the ASCII representation of the newline (‘\n’), we get the following output:

Output

b'Hello from Journaldev\nHow are you?'

Now, it is always a good practice to close our buffer handle whenever we have done our work.

This is also to make sure that we free whatever memory we’ve allocated for the buffer.

To close the buffer, use:

bytes_stream.close()

Now let’s look at the StringIO class.

Python StringIO Class

Similar to io.BytesIO, the io.StringIO class can read string related data from a StringIO buffer.

import io

string_stream = io.StringIO("Hello from Journaldev\nHow are you?")

We can read from the string buffer using string_stream.read() and write using string_stream.write(). This is very similar to reading / writing from a file!

We can print the contents using getvalue().

import io

string_stream = io.StringIO("Hello from Journaldev\nHow are you?")

# Print old content of buffer
print(f'Initially, buffer: {string_stream.getvalue()}')

# Write to the StringIO buffer
string_stream.write('This will overwrite the old content of the buffer if the length of this string exceeds the old content')

print(f'Finally, buffer: {string_stream.getvalue()}')

# Close the buffer
string_stream.close()

Output

Initially, buffer: Hello from Journaldev
How are you?
Finally, buffer: This will overwrite the old content of the buffer if the length of this string exceeds the old content

Since we are writing to the same buffer, the new contents will obviously overwrite the old one!

Reading from a StringIO buffer

Similar to writing, we can also read from a StringIO buffer using buffer.read().

import io

input = io.StringIO('This goes into the read buffer.')
print(input.read())

Output

This goes into the read buffer.

As you can see, the contents are now inside the read buffer, which is printed using buffer.read().

Reading a file using io

We can using the io.open() method to directly read from a file also, similar to reading from a file object.

Here, this module gives us the option of buffered vs non-buffered reading.

For example, the following will use a buffered read to read a file, by setting buffering = SIZE. If SIZE = 0, this will imply no buffering!

Suppose sample.txt has the following contents:

Hello from JournalDev!
How are you?
This is the last line.

import io

# Read from a text file in binary format using io.open()
# We read / write using a buffer size of 5 bytes
file = io.open("sample.txt", "rb", buffering = 5)

print(file.read())

# Close the file
file.close()

Output

b'Hello from JournalDev!\nHow are you?\nThis is the last line.\n'

As you can see, the file has been read successfully! Here, io will read the file using a buffer size of approximately 5 bytes.

Using io.open() vs os.open()

The io.open() function is a much preferred way to perform I/O operations as it is made as a high-level Pythonic interface.

On the contrary, the os.open() will perform a system call to the open() function. This will return a file descriptor, which cannot be used like the io handle object.

Since io.open() is a wrapper function to os.open(), it is generally good practice to use such wrapper functions, since they automatically handle many errors for you.

Conclusion

In this article, we learned about using the Python IO module, and it’s two main classes – io.BytesIO and io.StringIO for reading and writing byte and string data onto a buffer.

References