Untangled Development

Non-Django issues during a Django upgrade

Upgrading from Django 3.0.9 to 3.1 was a breeze.

To be fair upgrading Django is usually a breeze. The exhausive release notes written every time a new version is rolled out is a defining factor here.

What do I mean when I say “a breeze”? I compare Django “upgrades” to something like upgrading a Python 2 codebase to Python 3 😬

So, upgrades being a breeze, why write a post about this topic?

Because a Django application usually does not comprise only of the Django package.

How I upgraded

This section basically is about “how I usually upgrade”.

Made specific for this 3.0.9 to 3.1 upgrade.

Steps

  • Changed Django version in requirements.txt file. I.e. changed entry from Django==3.0.9 to Django==3.1.
  • Ran pip install -r requirements.txt with the updated requirements file. Among the command’s output I spotted:
Installing collected packages: Django
  Found existing installation: Django 3.0.9
    Uninstalling Django-3.0.9:
      Successfully uninstalled Django-3.0.9
Successfully installed Django-3.1
  • Ran ./manage.py check

This yielded several warnings of the type:

$ ./manage.py check
System check identified some issues:

WARNINGS:
app_name.Model.field_name: (fields.W904) django.contrib.postgres.fields.JSONField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0.
    HINT: Use django.db.models.JSONField instead.

...

System check identified 3 issues (0 silenced).

Hence I search/replaced any instances of

from django.contrib.postgres.fields import JSONField

with

from django.db.models import JSONField
  • Ran ./manage.py check again. Got this output:
System check identified no issues (0 silenced)
  • In order to keep the Django models code and database state in sync, I ran makemigrations:
./manage.py makemigrations appname -n move_to_django_jsonfield

Done this for each app containing a JSONField change.

This resulted in AlterField statements for the models/fields whose JSONField type was updated:

migrations.AlterField(
    model_name='mymodel',
    name='myfield',
    field=models.JSONField(),
),
  • Ran tests.

Oh noes! An external Python packages I use only when running tests, django_bakery, does not like Django 3.1. I got this exception:

TypeError: <class 'django.db.models.fields.json.JSONField'> is not supported by baker.

Non-Django issue and what I did about it

The Django application itself works without error. Unit tests that rely on the Django 3.1 JSONField fail. Because the model_bakery package does not support the newly added django.db.models.JSONField yet.

First thought: Am I running the latest package?

The below command didn’t install a new version.

pip install model_bakery -U

So I have the latest.

Second thought: So is this an opportunity for me to write a PR? Finally make a decent contribution to a package I really like? To an opensource project which has given me so much?

Not so fast 😊

I navigated to the pacakge’s github repo and looked for the text JSONField within the closed PRs. And indeed:

Github PR

The Django 3.1 compatibility fix I was going to “contribute” is already done in this PR by Tobias Bengfort. And merged by Bernardo Fontes. Thanks folks!

So the question now is: How do I get to use the package with this commit in my project?

Pinning a github project to a specific commit

This StackOverflow answer presents various ways of achieving this.

The model_bakery requirements file entry I had looked like:

model_bakery==1.1.0

whereas now I have:

git+ssh://git@github.com/model-bakers/model_bakery.git@2dada1f51a3d08bbec7cd8be9da0b103581dc392

2dada1f51a3d08bbec7cd8be9da0b103581dc392 is the git SHA of this specific commit.

Update: 2020-09-11

I ran into an error when trying to install the package above while rolling out a Docker container for local Django development:

Collecting git+ssh://****@github.com/model-bakers/model_bakery.git@2dada1f51a3d08bbec7cd8be9da0b103581dc392 (from -r requirements/local.txt (line 9))
  Cloning ssh://****@github.com/model-bakers/model_bakery.git (to revision 2dada1f51a3d08bbec7cd8be9da0b103581dc392) to /tmp/pip-req-build-zi5byz4k
  Running command git clone -q 'ssh://****@github.com/model-bakers/model_bakery.git' /tmp/pip-req-build-zi5byz4k
  Host key verification failed.
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
ERROR: Command errored out with exit status 128: git clone -q 'ssh://****@github.com/model-bakers/model_bakery.git' /tmp/pip-req-build-zi5byz4k Check the logs for full command output.

Why this error? Because on the Docker container I do not have git set up and configured with a Github user account.

To resolve this I changed the way the repo commit is pinned. The format I’m using does not need pip to rely on git. In fact it relies only on https:

https://github.com/model-bakers/model_bakery/archive/2dada1f51a3d08bbec7cd8be9da0b103581dc392.zip

I.e. the format now is:

https://github.com/model-bakers/model_bakery/archive/commitSHA.zip

This works across all environments, i.e.:

  • when installing the package on its own using pip install [https path to package zip]
  • pip install -r requirements.txt in my local terminal
  • pip install -r requirements.txt inside the Docker container

How to get that zip file URL?

Navigate to the commit’s URL on github. In this case it’s:

https://github.com/model-bakers/model_bakery/tree/2dada1f51a3d08bbec7cd8be9da0b103581dc392

Click the Code button, and copy the link address for the Download ZIP option:

Download zip

Risks

What I did is not best practice. Far from it. But I only use model_bakery package when running unit tests. I’m OK with taking that sort of risk with a package that does not “run” in production. I’m willing to accept that risk.

Conclusion

I find Django a very stable framework. But no open source project can guarantee “zero hiccups” for the whole ecosystem around it.

The approach I’ve taken above is a compromise between:

  • the stability of a mature framework like Django, and
  • utilising the rich ecosystem around it

Any thoughts? Should I have done it another way? Did I ignore any risks?

Comments !