Python Logging module is used to implement a flexible event-driven logging system, which serves as a convenient way of storing log events or messages for an application.
Python Logging Module – Loggers
The Logger object is the object of this module which we can manipulate to do all our required logging.
To instantiate a Logger object, we must always specify:
log_object = logging.getLogger(name)
Multiple calls to
getLogger(name) with the same name always give a reference to the same object.
Now that we have our logger object, we can use multiple functions on it.
Writing messages to the console
Whenever there need to be events reported, we emit the contents of the logger objects so that the main running program gets notified of the status changes.
To do this, we have demarcation for the level of severity of a message to be emitted, called
|LEVEL||When it is used|
|DEBUG||Detailed information for debugging purposes|
|INFO||Confirmation that things are working normally|
|WARNING||An indication that something unexpected happened|
|ERROR||A more serious problem, when the software is not able to perform some function|
|CRITICAL||A serious error, having the maximum severity|
This is used to write to the corresponding log file or to the console. The default level is
WARNING, which means that only events of this level and above will be tracked (i.e,
CRITICAL will be tracked by default)
This allows the programmer to have control over how these status messages can be displayed, based on the severity level chosen.
logging.info(message) will display the message onto the console/file.
The following example illustrates this method
import logging # This message will be printed to the console logging.warning('Warning message') # Will not be printed to the console logging.info('Works as expected')
Logging into a file
logging.basicConfig() to create a logging file handler.
import logging # Create the logfile logging.basicConfig(filename='sample.log', level=logging.DEBUG) logging.debug('Debug message') logging.info('info message') logging.warning('Warning message')
root@Ubuntu # cat sample.log DEBUG:root:Debug message INFO:root:info message WARNING:root:Warning message
NOTE: The call to
basicConfig() must come before any call to
There is another parameter
filemode, for the
basicConfig() function, which specifies the mode of the logfile.
The below example makes
sample.log have write-only mode, which means any messages written to it will overwrite the previous content of the file.
logging.basicConfig(filename='sample.log', filemode='w', level=logging.DEBUG)
Logging from multiple modules
Since the log-file object and the handlers provide the same context in multiple modules, we can use them directly in other modules.
An example is shown below
# main.py import logging import sample_module def main(): logging.basicConfig(filename='application.log', level=logging.INFO) logging.info('main module started') sample_module.sample_function() logging.info('completed') main()
# sample_module.py import logging def sample_function(): logging.info('Sample module doing work')
Here, the same logging object can be shared by multiple modules, thus making it a good fit for modularised code.
Format of the messages
By default, the output message has a message header containing the name of the node and the level of the message. To change the format of the displayed messages, a suitable format must be specified.
import logging logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG) logging.debug('sample message')
Display Date and Time in the message
%(asctime)s format specifier to signify the time in the message.
import logging logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p') logging.warning('Sample message')
12/26/2019 12:50:38 PM Sample message
Logger object names
The log message, by default, has the first part containing the level and the name of the logger object used. (For example:
Usually, if the
name argument is not specified, it defaults to
ROOT, the name of the root node.
Otherwise, it is good practice to use the
__name__ variable, because it is the name of the module in the Python package namespace.
import logging logger = logging.getLogger(__name__)
Modifying the level of the message
Logger objects provide us a way of modifying the threshold level at which messages are displayed. The
setLevel(level) method is used to set the level of the logger object.
import logging logger = logging.getLogger(__name__) # Set the log level as DEBUG logger.setLevel(logging.DEBUG) # The DEBUG level message is displayed logger.debug('Debug message')
No handlers could be found for logger "__main__"
This is not what we expected. Why is the message not being displayed, and what is a handler?
A logging Handler is a component that does the work of writing to the log/console. The logger object invokes the logging handler to display the contents of the message.
A Handler is never instantiated directly, as with the case of Loggers. There are different types of Handlers, each of which has its own method for instantiation.
Types of Handlers
There are various Handlers in the logging module, but we primarily concern ourselves with the 3 most used handlers, namely:
A StreamHandler is used to send the logging output to streams such as
stderr, or any file-like object which supports
flush() methods, like pipes, FIFOs, among others.
We can use
StreamHandler() to initialize a StreamHandler object which can display messages on the console from our Logger object.
The previous code snippet can now be completed with calls to
import logging # Instantiate the logger object logger = logging.getLogger(name='hi') # Set the level of the logger logger.setLevel(logging.DEBUG) # Initialise the handler object for writing handler = logging.StreamHandler() # The handler also needs to have a level handler.setLevel(logging.DEBUG) # Add the handler to the logger object logger.addHandler(handler) # Now, the message is ready to be printed by the handler logger.debug('sample message')
For logging into a file, we can use the FileHandler object. It is also similar to the StreamHandler object, but a file descriptor is referenced here so that logging happens to the file.
The above snippet can be modified when instantiating the log handler. By changing the type to a FileHandler, the message can be logged to a file.
handler_name = logging.FileHandler(filename='sample.log', mode='a')
This handler essentially does not write to anything (Equivalent to piping output to
/dev/null), and hence, is considered as a no-op handler, useful for Library developers.
We learnt how to use the logging module API to log messages onto the console and to a file based on their severity level. We also learnt how to use the format specifiers to specify how the messages are being displayed, and the usage of Logging Handlers to control and modify the level of the logged messages.
Python Official Documentation for the Logging module: https://docs.python.org/3/howto/logging.html#logging-basic-tutorial