Python Logging Basics

The standard library with each Python installation contains a logging module that provides a flexible framework for logging messages. Most Python applications and libraries use this module.

There are four types of classes used when logging.

• Loggers — exposes the interface that your application can use to log
• Handlers — send the log messages to the appropriate destination
• Filters — determine which messages is send to what output
• Formatters — how is a message formatted in the final output

Loggers

To create a logger in your application or library, you can call logging.getLogger("app"). The argument "app" is the name that you choose for this logger. You can use the name of a logger to form a hierarchy by using dots in the name like 'app.module.function'. A Filter can use this hierarchy to only view messages from a specific function or module. You are free to choose whatever name or hierarchy.

If you don’t specify the name of a logger, logging.getLogger() returns a logger, which is the root logger of the hierarchy.

Logging Levels

As with all logging frameworks, you can log a message on an individual level to vary the importance. The logging module in Python defines six levels.

• CRITICAL — — logger.critical("critical message")
• ERROR — logger.error("error message")
• WARNING — logger.warn("warn message")
• INFO — logger.info("info message")
• DEBUG — logger.debug("debug message")
• NOTSET

Default, the logging level is set to NotSet on the logging module. Using NOTSET causes the processing of all messages. If you set the log level to INFO, it will include INFO, WARNING, ERROR, and CRITICAL messages. NOTSET and DEBUG messages will not be included.

Handlers

If you run the previous example, the log message won’t show. Although the logging level is set to DEBUG and we are logging on the INFO level. This is because we didn’t specify a handler.

There are several handlers available in the logging module, such as StreamHandler, FileHandler, NullHandler, RotatingFileHandle, SocketHandler, DatagramHandler.

We will use two of them, the StreamHandler for logging to the console and FileHandler for logging to a file. See the Python documentation for more details about the other handlers.

StreamHandler

For logging to the console, we can use the StreamHandler. We add the handler to the logger using the addHandler method.

If you run the previous program, the log message "Starting application" will show on the console.

FileHandler

The same way we added the StreamHandler, you can add the FileHandler to enable logging to a file. The FileHandler accepts four arguments.

FileHandler

The first argument specified the name of the file that should be used to store the log messages. The second argument specifies the file mode. If not specified, mode 'a' is used. The encoding argument is used to open the file with the specified encoding. If the delay is true, the file is opened until the first message must be written to the file. Otherwise, is it directly opened.

If you open the file logging.txt you will see a line stating the message “Starting application”. Normally when logging, you want to see extra information on each logging line such as the date and time.

This is where we can use Formatters.

Formatters

Formatters are responsible for converting the internal LogRecors to a string that can be interpreted by a human or an external system. You can use a format string with the default Formatter. If you don’t supply a format string it just uses the log message as seen in the previous example.

If you want to add a timestamp to each output message, you have to create a Formatter instance and set it on a handler.

In the example below, I create a new formatter on line six and set that formatter on the StreamingHandler. This means that the logging module will use this Formatter to format the message on the console. The FileHandler still has the default Formatter.

The application when started shows the following message in the console.2020-06-14 17:49:34,614 - INFO - Starting application

The second argument on the Formatter can be used to specify a specific format for formatting the date and time.

The last part of the logging framework are Filters.

Filters

You can use Filters to implement more sophisticated filtering than is provided using the levels. Filters can be attached to handlers and loggers. Filters that are attached to a handler are used before an event is emitted by the handlers. Filters that are attached to loggers are used whenever an event is logged before sending it to handlers.

The default filter has a single argument name, this allows you to set the name of the logger to apply this filter. When using a named hierarchy you can apply more granular filtering. For example, a filter initialized with ‘A.B’ will allow events logged by loggers ‘A.B’, ‘A.B.C’, ‘A.B.C.D’, ‘A.B.D’ etc. but not ‘A.BB’, ‘B.A.B’ etc. If initialized with the empty string, all events are passed.

Adding a Formatter and Handler

Because adding a Formatter and Handler is so common, the logging module provides the helper function basicConfig. This function performs basic configuration by creating a StreamHandler with default Formatter and adding it to the root logger.

Conclusion

This article shows the basics of logging in Python using the standard logging module. The benefit of having a standard logging library is that most Python modules use it. So your application log can include your own messages integrated with messages from third-party modules.

Thank you for reading.