A Problem with Duplicated Mutable Constants
Here’s a small problem I’ve seen where several modules share versions of the same “constant” variable. It came up in the context of a Django project with multiple settings files, but it could happen in different contexts.
Imagine you have two submodules defining API_CONFIG as a “constant” dictinoary. The development submodule should copy the value in base, but use a different value for the "rate_limit" key. You have example/base.py which looks like:
API_CONFIG = {
# ...
"rate_limit": "10/m",
# ...
}
…and example/development.py:
from example.base import API_CONFIG
API_CONFIG["rate_limit"] = "100/m"
By importing from base, the development module doesn’t need to completely redefine API_CONFIG. Great - the redundant repetition is reduced. But can you see the flaw in this approach?
The problem is that API_CONFIG is the same dict in both modules. The change in development “leaks” back to base:
In [1]: from example import base
In [2]: base.API_CONFIG["rate_limit"]
Out[2]: '10/m'
In [3]: from example import development
In [4]: development.API_CONFIG["rate_limit"]
Out[4]: '100/m'
In [5]: base.API_CONFIG["rate_limit"]
Out[5]: '100/m'
Eek!
This happens because Python variables are only names for underlying objects. Pointing another name to the same variable does not create a copy. If this is surprising, Ned Batchelder’s Python Names and Values is a great explainer.
The shared value might not always manifest as a problem. In the context of Django, it’s only possible to activate one settings file per process, so if you activated “development” the values in “base” won’t (normally) be needed. But it could be a problem if you ever need to use both modules, for example whilst debugging or testing.
Let’s look at a couple of ways of solving this problem.
Deep Copies
A simple technique here is to fully copy mutable objects before applying changes. Python lets you do this with copy.deepcopy(). In example/development.py, this looks like:
from copy import deepcopy
from example.base import API_CONFIG
API_CONFIG = deepcopy(API_CONFIG)
API_CONFIG["rate_limit"] = "100/m"
(Note that copy.copy() or dict.copy() won’t work. This is because they only produce shallow copies, which can share nested values.)
Immutable Values
A second solution is to switch to immutable data types. This way changes aren’t even possible to the value. For dicts, you can emulate an immutable dictionary with types.MappingProxyType, as I previous covered.
Using this in example/base.py would look like:
from types import MappingProxyType
API_CONFIG = MappingProxyType(
{
# ...
"rate_limit": "10/m",
# ...
}
)
Then example/development.py can create a copy with changes:
from types import MappingProxyType
from example.base import API_CONFIG
API_CONFIG = MappingProxyType(
API_CONFIG
| {
"rate_limit": "100/m",
}
)
(Using Python 3.9’s dict merge operator.)
Neat.
Sorted
With either version of the above, the two modules preserve their separate values:
In [1]: from example import base
In [2]: base.API_CONFIG["rate_limit"]
Out[2]: '10/m'
In [3]: from example import development
In [4]: development.API_CONFIG["rate_limit"]
Out[4]: '100/m'
In [5]: base.API_CONFIG["rate_limit"]
Out[5]: '10/m'
👍
Read my book Boost Your Django DX, freshly updated in November 2024.
One summary email a week, no spam, I pinky promise.
Related posts: