Use partial() With Django’s transaction.on_commit() to Avoid Late-Binding Bugs

1. A Plain Function

This is the first techinque covered by the Django documentation, and perhaps the easiest to understand.

For example, imagine we were writing a function to grant a user a reward and email them. To send the email after the reward is saved in the database, we can create an inner function, and pass that to transaction.on_commit():

from django.db import transaction

...


def grant_reward(amount: Decimal, user: User) -> None:
    Reward.objects.create(user=user, amount=amount)

    def notify_user() -> None:
        send_mail(
            subject=f"You have received a £{amount} reward",
            message=...,
            from_email=...,
            recipient_list=[user.email],
        )

    transaction.on_commit(notify_user)

I find this method relatively straightforward, as it requires only basic Python concepts. However it does require a few lines to define the function and then pass it to on_commit().

2. A Decorated Function

Python’s decorator syntax allows you to wrap functions using a decorator function. Decorator functions take the function they’re applied to, and return a new function (or any value).

Since on_commit() can take a function, we can use it as a decorator:

from django.db import transaction

...


def grant_reward(amount: Decimal, user: User) -> None:
    Reward.objects.create(user=user, amount=amount)

    @transaction.on_commit
    def notify_user() -> None:
        send_mail(
            subject=f"You have received a £{amount} reward",
            message=...,
            from_email=...,
            recipient_list=[user.email],
        )

This technique is basically a shorter way of passing a plain function, avoiding repeating the name notify_user. The syntax may be a bit unclear though, as it is unusual to use a decorator for its side effect (running the function later), rather than modifying the function.

Also, since on_commit() doesn’t have any return value, the decorator syntax will set notify_user to None. This could be surprising, especially when debugging.

3. A lambda Function

Django’s documentation also covers using a lambda function with transaction.on_commit(). We can adapt our example to use lambda like so:

from django.db import transaction

...


def grant_reward(amount: Decimal, user: User) -> None:
    Reward.objects.create(user=user, amount=amount)

    transaction.on_commit(
        lambda: send_mail(
            subject=f"You have received a £{amount} reward",
            message=...,
            from_email=...,
            recipient_list=[user.email],
        )
    )

The code is a bit shorter, as we don’t need to name the callback function. The execution order is clearer, but I generally find the drawbacks of lambda functions outweigh the benefits:

• I find lambda to be relatively rare in Django projects, so junior developers may not understand the the syntax.
• Lambda functions have the anonymous name <lambda> in stack traces, which can complicate debugging.
• Lambda functions cannot have type hints. Mypy’s strict mode thus prevents using them in many contexts.

4. A partial() Of a Function

Python’s functools.partial() “wraps” a given callable, with some arguments pre-filled. It’s a fantastic tool that I’ve blogged about using with Django before (three use cases, and three more). This is also my preferred technique.

You can use partial() for our example like so:

from functools import partial
from django.db import transaction

...


def grant_reward(amount: Decimal, user: User) -> None:
    Reward.objects.create(user=user, amount=amount)

    transaction.on_commit(
        partial(
            send_mail,
            subject=f"You have received a £{amount} reward",
            message=...,
            from_email=...,
            recipient_list=[user.email],
        )
    )

We pass partial() the function to wrap, send_mail(), and some keyword arguments to pass to it. When the transaction commits, the partial object is later called, which in turn calls send_mail() with the pre-computed arguments.

partial() Prevents a Pesky Bug

At first glance, using partial() may seem to achieve the same as a plain function or lambda, but it requires an import. But, there is one key difference that helps avoid a bug: partial() is early binding, whilst all the prior techniques are late binding.

Early binding here means that function arguments are resolved when calling partial(). Python constructs the f-string for subject, the list containing the user’s email address, and so on, before passing them to partial(). The partial() stores those arguments and uses them later.

In contrast, the late binding in the other techniques does this work later, when their inner functions are called, after the transaction commits. This can lead to bugs when those variables change after the call to transaction.on_commit().

Say we modify our function to grant rewards to multiple users. We might try this with a plain function like so:

from django.db import transaction

...


def grant_rewards(amount: Decimal, users: QuerySet[User]) -> None:
    for user in users:
        Reward.objects.create(user=user, amount=amount)

        def notify_user() -> None:
            send_mail(
                subject=f"You have received a £{amount} reward",
                message=...,
                from_email=...,
                recipient_list=[user.email],
            )

        transaction.on_commit(notify_user)

Due to late binding, this looped version has a bug…

notify_user() uses the user variable created by the loop, but it will only look up this variable when called, after the transaction commits. That is after the loop has completed, when user is assigned to the last user. So, each user will have a Reward created, but only the last user will receive the email… N times!

We accidentally spammed that user! 😳

Let’s instead use partial():

from functools import partial
from django.db import transaction

...


def grant_rewards(amount: Decimal, users: QuerySet[User]) -> None:
    for user in users:
        Reward.objects.create(user=user, amount=amount)

        transaction.on_commit(
            partial(
                send_mail,
                subject=f"You have received a £{amount} reward",
                message=...,
                from_email=...,
                recipient_list=[user.email],
            )
        )

Now the arguments are bound during the loop, when they’re passed to partial(). At commit time, each partial object passes its pre-created arguments to send_mail(). Thus, users receive one email each.

Using partial() saves the day 😅

Due to this clearer behaviour, I would always recommend using partial() with transaction.on_commit().

Linting For Late Binding Bugs with flake8-bugbear

I have seen this late-binding “gotcha” a few times, but to be honest, it has not been top of mind. I’m sure I’ve written or review-missed several instances of the late-binding bug over the years.

Things changed with last month’s 22.7.1 release of flake8-bugbear. This is a flake8 plugin that I already considered indispensable.

The new release added a check, B023, that finds loop variables used in late-bound variables. This check found a bunch of issues on two client projects, which I fixed by converting to use partial(). Thank you very much to Zac Hatfield-Dodds for contributing the new check, and Cooper Lees for reviewing it.

Protect your code by using flake8-bugbear today! It finds so many instances of problematic code patterns, it’s a must-have.

I covered a full setup of pre-commit, flake8, flake8-bugbear, and other tools for Django projects in my book Boost Your Django DX.

Fin

May your transaction.on_commit() callbacks always run as expected,

—Adam