Python Decorators Demystified

My colleague Naomi worked on a complex Python project a few weeks ago. As the deadline loomed, she struggled with a stubborn bug that seemed to defy all attempts to squash it. Her frustration was palpable, and as a fellow programmer, I couldn’t help but empathize.

One afternoon, I decided to drop by her desk to see if I could assist. As we talked, it became clear that she was having trouble identifying the bug’s root cause due to the code’s complicated nature.

I couldn’t help but think this was the perfect opportunity to introduce her to one of my favorite Python tools — decorators.

I showed Naomi how to create a simple logging decorator and add it to one of the standard Python functions she used internally.

This would enable her to better understand the program’s execution flow without modifying its core logic.

Initially skeptical, she was soon on board with some explanation and a quick demonstration.

The results were astounding. With the help of the logging decorator, Naomi was able to diagnose the issue and squash the bug quickly.

Not only did it save her hours of debugging, but she also added several other decorators that left her with cleaner and more maintainable code.

In this Medium article, I’ll demystify Python decorators and show how they can simplify your code and life as a programmer.

I’ll share the tips and tricks that helped Naomi overcome her debugging nightmare and harness the power of decorators to improve her code.

So please grab a cup of coffee, sit back, and let’s dive into the wonderful world of Python decorators!

The Python examples from this article can be found in this GitHub repository.

What is a Python decorator?

Python decorators allow you to wrap a function or method with additional functionality, effectively “decorating” the original function with new capabilities.

This is achieved by employing a higher-order function that takes the target function as an argument and returns a new function with the extended behavior.

In simpler terms, a decorator is a function that receives another function as its input, enhances or alters the input function’s behavior, and then returns the modified function without explicitly changing the original function’s implementation.

By using decorators, you can keep your code modular, clean, and maintainable while reducing the need for repetition when implementing shared functionality across multiple functions.

Before we can explore the usage of decorators, it’s essential to understand Python functions as first-class objects that can be passed around and used like any other type of argument.

Functions as first-class objects in Python

When we say that functions are first-class objects, they can be treated like any other object, such as a string, integer, float, or list. This includes being used as arguments in the same way as different types of objects.

To illustrate this point, consider the following code.

def say_message_to_naomi(message_func): 
    message_func("Naomi") 
 
def say_hello(name): 
    print(f"Hello, {name}") 
 
def say_goodbye(name): 
    print(f"Goodbye, {name}") 
 
say_message_to_naomi(say_hello) 
say_message_to_naomi(say_goodbye)

The first function say_message_to_naomi, accepts a single function argument called message_func. It then calls the function given as an argument with the string "Naomi".

We then define two additional functions: say_hello and say_goodbye. Both functions take a name parameter and print a message to the console.

The two functions say_hello and say_goodbye, are then passed as arguments to the say_message_to_naomi function.

When say_message_to_naomi is called with say_hello as an argument, it calls say_hello with the string "Naomi" as its argument, resulting in the message "Hello, Naomi" being printed to the console.

Similarly, when say_message_to_naomi is called with say_goodbye as an argument, it calls say_goodbye with the string "Naomi" as its argument, resulting in the message "Goodbye, Naomi" being printed to the console.

Creating decorators

Now that we have seen how functions can be used as arguments, we can move on to creating decorators. We’ll begin by creating a basic decorator and then explore how Python’s syntactic sugar can simplify the process.

Creating a Simple Decorator

Let’s create a basic decorator that prints a message before and after the decorated function is called.

The following code demonstrates how this can be done.

def my_first_decorator(func): 
    def wrapper(): 
        print("Before the function call") 
        func() 
        print("After the function call") 
    return wrapper 
 
def hello_world(): 
    print("Hello, World!") 
 
hello_world = my_first_decorator(hello_world) 
hello_world()

The my_first_decorator function is defined with a single parameter func, representing the function to be decorated. Inside my_first_decorator, a new function wrapper prints a message before and after the call to func. Finally, wrapper is returned.

The hello_world function is defined, which prints the message "Hello, World!".

The hello_world function is then decorated by calling my_first_decorator with hello_world as an argument. This returns the wrapper function, which is assigned to the hello_world variable. Thus, hello_world now points to the wrapper function returned by my_first_decorator.

When hello_world() is called, it calls the wrapper function, which first prints "Before the function call", then calls the original hello_world function, which prints "Hello, World!" and finally prints "After the function call".

Thus, using a decorator, we added functionality to the hello_world function without modifying the original.

Now let's see how Python has a special syntax for decorators.

Adding Python syntactic sugar

Python provides a special syntax for assigning decorators to functions, using the @ symbol followed by the decorator function name.

This allows us to easily attach a decorator to a function without modifying the original function's code.

The following code demonstrates how the @ symbol can attach the my_first_decorator to the hello_world function.

def my_first_decorator(func): 
    def wrapper(): 
        print("Before the function call") 
        func() 
        print("After the function call") 
    return wrapper 
 
@my_first_decorator 
def hello_world(): 
    print("Hello, World!") 
 
hello_world()

Parameterized decorators

Decorators can be parameterized. This involves an additional level of nesting to handle the arguments.

Parameterized decorators are essentially functions that return decorator functions, which in turn accept the function or class being decorated.

This extra layer allows you to pass arguments to the outermost function, enabling customization of the decorator’s behavior. Here’s an example to illustrate this concept.

def my_parameterized_decorator(arg1, arg2): 
    def decorator_function(func): 
        def wrapper(*args, **kwargs): 
            print(f"Decorator arguments: {arg1}, {arg2}") 
            return func(*args, **kwargs) 
        return wrapper 
    return decorator_function 
 
@my_parameterized_decorator("Hello", "World") 
def my_function(): 
    print("Inside the function") 
 
my_function()

As a side note, it’s worth mentioning that the decorator behavior relies on the closure mechanism for handling arguments, which allows decorators to maintain their state and encapsulate logic effectively.

A template for Python decorators

We can now define a template with boilerplate Python code for creating Python decorators. To illustrate this, consider the following template.

import functools 
 
def my_decorator(func): 
    @functools.wraps(func) 
    def wrapper(*args, **kwargs): 
        # Code to be executed before the decorated function is called 
        result = func(*args, **kwargs) 
        # Code to be executed after the decorated function is called 
        return result 
    return wrapper

While this decorator may look similar to the one we previously created, there are a few key differences that set it apart.

*args and **kwargs

First, we added *args and **kwargs as arguments to the wrapper function. In the context of decorators, *args and **kwargs are often used to allow the decorated function to accept any number of arguments, regardless of their type or number.

By passing *args and **kwargs to the wrapper function inside the decorator, we can ensure that the decorated function can accept any arguments passed to it without specifying them all in advance.

@functools.wraps(func)

A problem that can arise when creating decorators is that the resulting decorated function may retain only some of the metadata from the original function, such as the function name, docstring, parameter list, etc. This can make working with the decorated function more challenging.

The functools.wraps decorator copies the metadata from the original to the new function.

By using functools.wraps in a decorator, we ensure that the resulting decorated function retains all of the metadata from the original function. This makes working with the decorated function easier and helps avoid confusion when debugging or maintaining the code.

return result

The last change we made in the decorator is to capture the return value of the function we’re wrapping and store it in a variable called result.

This ensures that if the decorated function returns a value, we return it from the wrapper function. Finally, we return the result variable at the end of the wrapper function.

Real-world examples of Python decorators

Decorators are used in several Python packages. For example, Flask uses the @app.route decorator. This decorator is used in Flask web applications to map a URL route to a view function that handles requests for that route.

@app.route("/") 
def index(): 
    return "Hello, world!" 
 
@app.route("/about") 
def about(): 
    return "This is the about page."

Django uses the @login_required decorator. This decorator is used in Django web applications to restrict access to certain views to authenticated users only.

@login_required 
def my_view(request): 
    # View code goes here

Pytest is a popular testing framework for Python. It uses decorators extensively to mark test functions and modify their behavior. For example, the @pytest.fixture decorator defines fixtures that provide test data and other resources to test functions.

Practical examples including Naomi’s

This section will explore several use cases for decorators and provide examples of implementing them using the decorator template we introduced earlier.

Specifically, we’ll demonstrate how decorators can be used for logging, timing, retry on failure, and caching. We’ll conclude by sharing a real-world example of how Naomi used a decorator to find a bug in her project.

A decorator for timing and logging function execution

One everyday use case for Python decorators is logging information about a function call, such as input arguments, output value, and execution time.

This can help with debugging or performance analysis. See below a possible implementation of such a decorator.

import functools 
import logging 
import time 
 
def log_function_call(func): 
    @functools.wraps(func) 
    def wrapper(*args, **kwargs): 
        logging.info(f"Calling {func.__name__} with args={args}, kwargs={kwargs}") 
        start_time = time.perf_counter() 
        result = func(*args, **kwargs) 
        end_time = time.perf_counter() 
        elapsed_time = end_time - start_time 
        logging.info(f"Returned {result} from {func.__name__} in {elapsed_time:.6f} seconds") 
        return result 
    return wrapper

A decorator for retrying on failure

Retry on failure is a technique for retrying a function multiple times if it fails.

This can make your implementation more robust. See below a possible implementation of such a decorator.

import time 
 
def retry_on_failure(max_attempts, delay): 
    def decorator(func): 
        @functools.wraps(func) 
        def wrapper(*args, **kwargs): 
            for i in range(max_attempts): 
                try: 
                    result = func(*args, **kwargs) 
                    return result 
                except Exception as e: 
                    if i == max_attempts - 1: 
                        raise e 
                    time.sleep(delay) 
        return wrapper 
    return decorator 
 
@retry_on_failure(max_attempts=3, delay=1) 
def send_email(to, subject, body): 
    # Function code goes here

A decorator for caching the previous function result

Memoization is a technique for optimizing expensive recursive functions by caching the results of last calls.

Here’s an example of a memoization decorator:

def memoize(func): 
    cache = {} 
    @functools.wraps(func) 
    def wrapper(*args): 
        if args in cache: 
            return cache[args] 
        result = func(*args) 
        cache[args] = result 
        return result 
    return wrapper 
 
@memoize 
def fibonacci(n): 
    if n in (0, 1): 
        return n 
    return fibonacci(n - 1) + fibonacci(n - 2)

Naomi’s example use of a decorator

In the introduction, we described a bug my colleague Naomi was trying to find in her code.

She eventually succeeded by using Python decorators, which allowed her to modify the behavior of existing standard Python functions used internally by her code.

In her case, the bug was incorrect usage of the Python len function. We found this by adding the logging and timing decorator, as shown before, to the standard Python len function.

import functools 
import logging 
import time 
 
def log_function_call(func): 
    @functools.wraps(func) 
    def wrapper(*args, **kwargs): 
        logging.info(f"Calling {func.__name__} with args={args}, kwargs={kwargs}") 
        start_time = time.perf_counter() 
        result = func(*args, **kwargs) 
        end_time = time.perf_counter() 
        elapsed_time = end_time - start_time 
        logging.info(f"Returned {result} from {func.__name__} in {elapsed_time:.6f} seconds") 
        return result 
    return wrapper 
 
# Decorate the built-in len function 
len = log_function_call(len) 
 
print(len("Naomi"))

Using decorators for function registration and extensibility

Decorators can also serve purposes other than modifying or wrapping the decorated objects. A possible use case is registering objects, such as functions, within an extensible process.

For instance, consider a framework designed to apply treatments to graphic files based on their format (e.g., PNG, BMP, JPEG). To create an extensible system, treatment functions can be registered in a dictionary with the associated file extension as the key.

A decorator can facilitate this registration process in a developer-friendly and readable manner. By decorating each treatment function with a decorator that takes the file format as an argument, the decorated function can be added to the dictionary with the specified key.

This approach enables the easy addition of new functions to handle various image formats while maintaining clear and organized code.

See the example below for an example.

imgproc_registry = {} 
 
def imgproc(format): 
    def decorator_function(func): 
        imgproc_registry[format] = func 
        return func 
    return decorator_function 
 
@imgproc(format="PNG") 
def process_png(file): 
    # Processing code for PNG files 
 
@imgproc(format="JPEG") 
def process_jpeg(file): 
    # Processing code for JPEG files

Conclusion

Python decorators are a powerful feature that allows you to modify the behavior of functions and classes at runtime.

Using decorators, you can add functionality to your code without modifying the underlying functions or classes, making your code more modular and easier to maintain.

We explored the basics of Python decorators and provided a decorator template that you can use to create your decorators.

We also demonstrated several real-world examples of decorators, including logging, timing, and caching.

The article includes a real-world example of how my colleague, Naomi, used a decorator to find and fix a bug in her code.

By adding a decorator to modify the behavior of an internal standard Python function, she was able to gain insight into its behavior and debug the issue.

Throughout this article, we have primarily discussed and provided examples of function decorators. However, it is important to note that decorators are not limited to functions alone. They can also be used to modify or similarly extend the behavior of classes.

While the syntax and implementation details might differ slightly when working with class decorators, the underlying concept remains the same.

By mastering the art of Python decorators, you can take your Python programming skills to the next level and build more powerful and flexible applications. So go forth and decorate your code!

The Python examples from this article can be found in this GitHub repository.

Special thanks to Eric PASCUAL for offering valuable feedback and contributing the insightful example of using decorators for an extensibility framework.