Python Logging Module

Python Logging module is used to implement a flexible event-driven logging system, which serves as a convenient way of storing log events or messages for an application.

Python Logging Module – Loggers

The Logger object is the object of this module which we can manipulate to do all our required logging.

To instantiate a Logger object, we must always specify:

log_object = logging.getLogger(name)

Multiple calls to getLogger(name) with the same name always give a reference to the same object.

Now that we have our logger object, we can use multiple functions on it.

Writing messages to the console

Whenever there need to be events reported, we emit the contents of the logger objects so that the main running program gets notified of the status changes.

To do this, we have demarcation for the level of severity of a message to be emitted, called LEVEL.

This is used to write to the corresponding log file or to the console. The default level is WARNING, which means that only events of this level and above will be tracked (i.e, WARNING, ERROR, and CRITICAL will be tracked by default)

This allows the programmer to have control over how these status messages can be displayed, based on the severity level chosen.

Format: logging.info(message) will display the message onto the console/file.

The following example illustrates this method

import logging
# This message will be printed to the console
logging.warning('Warning message')

# Will not be printed to the console
logging.info('Works as expected')

Output

WARNING:root:Warning message

Logging into a file

We use logging.basicConfig() to create a logging file handler.

Format: logging.basicConfig(filename, level)

import logging

# Create the logfile
logging.basicConfig(filename='sample.log', level=logging.DEBUG)

logging.debug('Debug message')
logging.info('info message')
logging.warning('Warning message')

Output

root@Ubuntu # cat sample.log
DEBUG:root:Debug message
INFO:root:info message
WARNING:root:Warning message

NOTE: The call to basicConfig() must come before any call to debug(), info(), etc.

There is another parameter filemode, for the basicConfig() function, which specifies the mode of the logfile.

The below example makes sample.log have write-only mode, which means any messages written to it will overwrite the previous content of the file.

logging.basicConfig(filename='sample.log', filemode='w', level=logging.DEBUG)

Logging from multiple modules

Since the log-file object and the handlers provide the same context in multiple modules, we can use them directly in other modules.

An example is shown below

# main.py
import logging
import sample_module

def main():
    logging.basicConfig(filename='application.log', level=logging.INFO)
    logging.info('main module started')
    sample_module.sample_function()
    logging.info('completed')

main()

# sample_module.py
import logging

def sample_function():
    logging.info('Sample module doing work')

Here, the same logging object can be shared by multiple modules, thus making it a good fit for modularised code.

Format of the messages

By default, the output message has a message header containing the name of the node and the level of the message. To change the format of the displayed messages, a suitable format must be specified.

import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logging.debug('sample message')

Output

DEBUG:sample message

Display Date and Time in the message

Add the %(asctime)s format specifier to signify the time in the message.

import logging
logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')
logging.warning('Sample message')

Output

12/26/2019 12:50:38 PM Sample message

Logger object names

The log message, by default, has the first part containing the level and the name of the logger object used. (For example: DEBUG:ROOT:sample message)

Usually, if the name argument is not specified, it defaults to ROOT, the name of the root node.

Otherwise, it is good practice to use the __name__ variable, because it is the name of the module in the Python package namespace.

import logging

logger = logging.getLogger(__name__)

Modifying the level of the message

Logger objects provide us a way of modifying the threshold level at which messages are displayed. The setLevel(level) method is used to set the level of the logger object.

Format: logger.setLevel(level)

import logging

logger = logging.getLogger(__name__)

# Set the log level as DEBUG
logger.setLevel(logging.DEBUG)

# The DEBUG level message is displayed
logger.debug('Debug message')

Output

No handlers could be found for logger "__main__"

This is not what we expected. Why is the message not being displayed, and what is a handler?

Logging Handler

A logging Handler is a component that does the work of writing to the log/console. The logger object invokes the logging handler to display the contents of the message.

A Handler is never instantiated directly, as with the case of Loggers. There are different types of Handlers, each of which has its own method for instantiation.

Types of Handlers

There are various Handlers in the logging module, but we primarily concern ourselves with the 3 most used handlers, namely:

• StreamHandler
• FileHandler
• NullHandler

StreamHandler

A StreamHandler is used to send the logging output to streams such as stdout, stderr, or any file-like object which supports write() and flush() methods, like pipes, FIFOs, among others.

We can use StreamHandler() to initialize a StreamHandler object which can display messages on the console from our Logger object.

The previous code snippet can now be completed with calls to StreamHandler() and handler.setLevel().

import logging

# Instantiate the logger object
logger = logging.getLogger(name='hi')

# Set the level of the logger
logger.setLevel(logging.DEBUG)

# Initialise the handler object for writing
handler = logging.StreamHandler()

# The handler also needs to have a level
handler.setLevel(logging.DEBUG)

# Add the handler to the logger object
logger.addHandler(handler)

# Now, the message is ready to be printed by the handler
logger.debug('sample message')

Output

sample message

FileHandler

For logging into a file, we can use the FileHandler object. It is also similar to the StreamHandler object, but a file descriptor is referenced here so that logging happens to the file.

The above snippet can be modified when instantiating the log handler. By changing the type to a FileHandler, the message can be logged to a file.

handler_name = logging.FileHandler(filename='sample.log', mode='a')

NullHandler

This handler essentially does not write to anything (Equivalent to piping output to /dev/null), and hence, is considered as a no-op handler, useful for Library developers.

Conclusion

We learnt how to use the logging module API to log messages onto the console and to a file based on their severity level. We also learnt how to use the format specifiers to specify how the messages are being displayed, and the usage of Logging Handlers to control and modify the level of the logged messages.

References

Python Official Documentation for the Logging module: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial