Run your tests against Django's main!

This is the blog version of a talk! If you prefer, watch the recording on YouTube:

Django is known for its stability. The framework makes a strong commitment to API stability and forwards-compatibility, ensuring that developers can rely on it for building long-term, maintainable projects. A key aspect of this commitment involves extensive testing and structured releases—an area where testing by Django users can significantly enhance Django’s reliability. Here’s a closer look at how this works, and how you can contribute 🤝.

How Django stays stable

Django's stability is upheld through rigorous testing. As of Django 5.2, there are more than 18,000 tests run against all officially supported database backends, Python versions, and operating systems. Additionally, Django follows a well-structured deprecation policy, ensuring that public APIs are deprecated over at least two feature releases before being removed.

The feature release schedule is systematic and structured:

1. Active development happens on the main branch.
2. A stable branch (for example stable/5.2.x) is forked when an alpha release is made.
3. After a month, the beta release follows, where only release-blocking bug fixes are allowed.
4. A month later, a release candidate (RC) is published, marking the translation string freeze.
5. If no critical bugs are found, the final release is published after a couple of weeks.

With this structured approach, Django ensures that releases are stable. However, bugs can and do occasionally slip through the cracks!

Catching issues early

The best time to catch issues is before they reach the final release. Ideally, potential bugs should be caught at the pull request stage, but keeping up with all changes is challenging. This is where the community can help—by running their tests with Django's main branch.

How you can help

You can set up your test suite to run with Django's main branch in your tests pipeline. Here's an example using GitHub Actions, a popular Continuous Integration platform:

test:
    runs-on: ubuntu-latest
    continue-on-error: ${{ matrix.experimental }}
    strategy:
      matrix:
        include:
          - python: "3.13"
            django: "git+https://github.com/django/django.git@main#egg=Django"
            experimental: true
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python }}
      - run: pip install -r requirements.txt
      - if: ${{ matrix.experimental }}
        run: pip install "${{ matrix.django }}"
      - run: python -Wd manage.py test
  

If you maintain a Django package, you likely already test with multiple Django versions. Adding the main branch ensures that your project stays ahead of potential breaking changes.

Why this helps you

Running tests with Django main allows you to detect when changes in Django break your project. Sometimes, this happens due to the removal of internal APIs (that were never intended for reuse outside Django 🙈). If your tests fail, you can identify which commit caused the issue and adjust your code accordingly.

For example, on the Wagtail CMS project, recently caught an issue when an internal class, SubqueryConstraint, was removed from Django. This wasn't a bug in Django—it was the removal of an internal workaround that was no longer needed. If your project relies on internal APIs, testing against main is crucial to avoid surprises.

Why this helps Django

Testing with main doesn't just help your project—it helps Django too. Sometimes, your tests may fail due to legitimate regressions in Django that its test suite doesn't cover. Reporting these issues ensures they get fixed before the next release.

For example, just two days before Django 5.2 alpha was released, Wagtail tests on main helped detect a bug where calling .full_clean() on a child model in a multi-table inheritance setup triggered an unintended database query. This regression was promptly fixed, ensuring a smoother release for all users.

Take action: test against Django's main and report issues

By running your tests against Django's main branch and reporting any issues you find, you contribute to a more stable framework for everyone. It's a small step that makes a big impact.

So, take a few minutes today to update your automated tests setup and help keep Django as reliable as ever!