Configure installation of pgh context tracking func post migrate (#140)

* Include more tests for runtime SQL injection

* Configure installation of pgh context tracking func post migrate, plus update other docs

* update lock file

* update release notes

CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## 3.3.0 (2024-08-27)
+
+#### Features
+
+- `PGHISTORY_INSTALL_CONTEXT_FUNC_ON_MIGRATE` setting to configure installation of tracking function after migrations by [@wesleykendall](https://github.com/wesleykendall) in [#140](https://github.com/Opus10/django-pghistory/pull/140).
+- `PGHISTORY_CREATED_AT_FUNCTION` setting to configure the function for determining the current time in history triggers by [@lokhman](https://github.com/lokhman) in [#137](https://github.com/Opus10/django-pghistory/pull/137).
+- Add ASGI support by [@pablogadhi](https://github.com/pablogadhi) in [#127](https://github.com/Opus10/django-pghistory/pull/127).
+
+#### Fixes
+
+- Ensure `BigAutoField`s are properly mirrored in history models by [@tobiasmcnulty](https://github.com/tobiasmcnulty) in [#134](https://github.com/Opus10/django-pghistory/pull/134).
+
+    !!! warning
+
+        If you have event models for models with `BigAutoField` primary keys, you will see new migrations to convert `pgh_obj_id` to a bigint.
+
+- Support filtering event models by referenced proxy models by [@lokhman](https://github.com/lokhman) in [#135](https://github.com/Opus10/django-pghistory/pull/135).
+- Ensure `bytes` representations of SQL are handled by [@tobiasmcnulty](https://github.com/tobiasmcnulty) in [#136](https://github.com/Opus10/django-pghistory/pull/136).
+- Fix setting default trackers with the `PGHISTORY_DEFAULT_TRACKERS` setting by [@SupImDos](https://github.com/SupImDos) in [#133](https://github.com/Opus10/django-pghistory/pull/133).
+- Don't install pghistory's context tracking function on non-postgres databases by [@pmdevita](https://github.com/pmdevita) in [#132](https://github.com/Opus10/django-pghistory/pull/132).
+- Ensure `MiddlewareEvents` doesn't filter out non-middleware events in the admin by [@lokhman](https://github.com/lokhman) in [#130](https://github.com/Opus10/django-pghistory/pull/130).
+- Support custom primary keys for the aggregate `Events` proxy model by [@lokhman](https://github.com/lokhman) in[#128](https://github.com/Opus10/django-pghistory/pull/128).
+
+#### Changes
+
+- Django 5.1 compatibility, dropped Django 3.2 / Postgres 12 support by [@wesleykendall](https://github.com/wesleykendall) in [#139](https://github.com/Opus10/django-pghistory/pull/139).
+- Added section in FAQ for handling issues with concrete inheritance by [@xaitec](https://github.com/xaitec) in [#138](https://github.com/Opus10/django-pghistory/pull/138).
+- Add reference to DjangoCon talk in the docs by [@max-muoto](https://github.com/max-muoto) in [#114](https://github.com/Opus10/django-pghistory/pull/114).
+
 ## 3.2.0 (2024-04-20)
 
 #### Bug

README.md

@@ -77,7 +77,7 @@ There's a [DjangoCon 2023 talk](https://youtu.be/LFIAqFt9z2s?si=GQBchy9bVAk-b9ok
 
 For information on setting up django-pghistory for development and contributing changes, view [CONTRIBUTING.md](CONTRIBUTING.md).
 
-## Creators
+## Creator
 
 - [Wes Kendall](https://github.com/wesleykendall)
 
@@ -91,3 +91,9 @@ For information on setting up django-pghistory for development and contributing
 - @adamchainz
 - @eeriksp
 - @pfouque
+- @tobiasmcnulty
+- @lokhman
+- @SupImDos
+- @pmdevita
+- @pablogadhi
+- @xaitec

docs/faq.md

@@ -20,7 +20,7 @@ For those that are new to triggers and want additional confidence in their behav
 
 Although triggers will be issuing additional SQL statements to write events, keep in mind that this happens within the database instance itself. In other words, writing events does not incur additional expensive round-trip database calls. This results in a reduced performance impact when compared to other history tracking methods implemented in software.
 
-Note that currently `django-pghistory` uses row-level triggers, meaning a bulk update such as `Model.objects.update` over one hundred elements could perform one hundred queries within the database instance. We're planning to address this in a future version of `django-pghistory` by using statement-level triggers instead.
+Note that currently `django-pghistory` uses row-level triggers, meaning a bulk update such as `Model.objects.update` over one hundred elements could perform one hundred queries within the database instance. We're planning to address this in a future version of `django-pghistory` by using statement-level triggers instead. See the [discussion here in django-pgtrigger](https://github.com/Opus10/django-pgtrigger/discussions/166).
 
 See the [Performance and Scaling](performance.md) section for tips and tricks on large history tables.
 
@@ -34,11 +34,9 @@ Add a condition to your tracker. See the [Conditional Tracking](event_tracking.m
 
 ## How do I track models with concrete inheritance?
 
-Currently concrete inheritance isn't well supported since `django-pghistory` simply snapshots the fields on the underlying table. Since concrete inheritance uses foreign keys to other tables, you'll need to set up trackers on all tables.
+`django-pghistory` simply snapshots the fields on the underlying table, meaning you'll need to set up trackers on all inherited tables. We plan to make this easier in the future.
 
-We plan to add a guide on this in the future.
-
-However, a simple example of how it can be achieved as such.
+When tracking child models, remember to set proper reverse foreign key names, otherwise collisions can happen. For example:
 
 ``` python
 @pghistory.track()
@@ -51,15 +49,15 @@ class Child(Parent):
     field_b = models.CharField(default="unknown")
 ```
 
-Given the use case of just two models, a patent and a child as shown above the following error will be raised:
+These models will raise the following:
 ```
 mmp_metadata.ChildEvent.pgh_obj: (fields.E304) Reverse accessor 'Child.events' for 'mmp_metadata.ChildEvent.pgh_obj' clashes with reverse accessor for 'mmp_metadata.ParentEvent.pgh_obj'.
         HINT: Add or change a related_name argument to the definition for 'mmp_metadata.ChildEvent.pgh_obj' or 'mmp_metadata.ParentEvent.pgh_obj'.
 mmp_metadata.ChildEvent.pgh_obj: (fields.E305) Reverse query name for 'mmp_metadata.ChildEvent.pgh_obj' clashes with reverse query name for 'mmp_metadata.ParentEvent.pgh_obj'.
         HINT: Add or change a related_name argument to the definition for 'mmp_metadata.ChildEvent.pgh_obj' or 'mmp_metadata.ParentEvent.pgh_obj'.
 ```
 
-This error is due to multiple foreign keys ahving the same 'default' name. By manualy setting the relation and query names for the event model we can aviod this clash.
+This error is due to multiple foreign keys having the same 'default' name. Manually set the relation and query names to avoid this clash.
 
 ``` python
 @pghistory.track(
@@ -107,6 +105,14 @@ If you need data for fields that have been dropped, we recommend two approaches:
 1. Make the field nullable instead of removing it.
 2. Use [django-pgtrigger](https://github.com/Opus10/django-pgtrigger) to create a custom trigger that dumps a JSON record of the row at that point in time.
 
+## I'm getting `function _pgh_attach_context() does not exist`
+
+If you don't run migrations in your test suite, pghistory's custom context tracking function won't be installed. Set `PGHISTORY_INSTALL_CONTEXT_FUNC_ON_MIGRATE=True` in your test settings.
+
+!!! note
+
+    This is harmless to enable in all environments, but it will issue a redundant SQL statement after running `migrate`.
+
 ## How can I report issues or request features
 
 Open a [discussion](https://github.com/Opus10/django-pghistory/discussions) for a feature request. You're welcome to pair this with a pull request, but it's best to open a discussion first if the feature request is not trivial.

docs/requirements.txt

@@ -1,98 +1,98 @@
 arrow==1.3.0 ; python_version >= "3.8" and python_version < "4"
-asgiref==3.8.1 ; python_version >= "3.8" and python_version < "4"
+asgiref==3.7.2 ; python_version >= "3.8" and python_version < "4"
 astunparse==1.6.3 ; python_version >= "3.8" and python_version < "3.9"
-babel==2.16.0 ; python_version >= "3.8" and python_version < "4"
+babel==2.13.0 ; python_version >= "3.8" and python_version < "4"
 backports-zoneinfo==0.2.1 ; python_version >= "3.8" and python_version < "3.9"
 beautifulsoup4==4.11.1 ; python_full_version >= "3.8.0" and python_version < "4"
 binaryornot==0.4.4 ; python_full_version >= "3.8.0" and python_version < "4"
 black==24.8.0 ; python_version >= "3.8" and python_version < "4"
 cachetools==5.5.0 ; python_version >= "3.8" and python_version < "4"
-certifi==2024.7.4 ; python_version >= "3.8" and python_version < "4"
+certifi==2023.7.22 ; python_version >= "3.8" and python_version < "4"
 chardet==5.2.0 ; python_version >= "3.8" and python_version < "4"
-charset-normalizer==3.3.2 ; python_version >= "3.8" and python_version < "4"
+charset-normalizer==3.3.0 ; python_version >= "3.8" and python_version < "4"
 click==8.1.7 ; python_version >= "3.8" and python_version < "4"
 colorama==0.4.6 ; python_version >= "3.8" and python_version < "4"
 cookiecutter==1.7.3 ; python_full_version >= "3.8.0" and python_version < "4"
-coverage[toml]==7.6.1 ; python_version >= "3.8" and python_version < "4"
-distlib==0.3.8 ; python_version >= "3.8" and python_version < "4"
+coverage[toml]==7.3.2 ; python_version >= "3.8" and python_version < "4"
+distlib==0.3.7 ; python_version >= "3.8" and python_version < "4"
 dj-database-url==2.2.0 ; python_full_version >= "3.8.0" and python_version < "4"
 django-dynamic-fixture==4.0.1 ; python_full_version >= "3.8.0" and python_version < "4"
 django-extensions==3.1.3 ; python_full_version >= "3.8.0" and python_version < "4"
 django-pgtrigger==4.12.0 ; python_full_version >= "3.8.0" and python_version < "4"
 django-stubs-ext==5.0.4 ; python_version >= "3.8" and python_version < "4"
 django-stubs==5.0.4 ; python_version >= "3.8" and python_version < "4"
-django==4.2.15 ; python_version >= "3.8" and python_version < "4"
-exceptiongroup==1.2.2 ; python_version >= "3.8" and python_version < "3.11"
+django==4.2.6 ; python_version >= "3.8" and python_version < "4"
+exceptiongroup==1.1.3 ; python_version >= "3.8" and python_version < "3.11"
 filelock==3.15.4 ; python_version >= "3.8" and python_version < "4"
-footing==0.1.7 ; python_full_version >= "3.8.0" and python_version < "4"
-formaldict==1.0.7 ; python_full_version >= "3.8.0" and python_version < "4"
+footing==0.1.4 ; python_full_version >= "3.8.0" and python_version < "4"
+formaldict==1.0.5 ; python_full_version >= "3.8.0" and python_version < "4"
 ghp-import==2.1.0 ; python_version >= "3.8" and python_version < "4"
 git-tidy==1.2.0 ; python_full_version >= "3.8.0" and python_version < "4"
 griffe==1.2.0 ; python_version >= "3.8" and python_version < "4"
-idna==3.8 ; python_version >= "3.8" and python_version < "4"
-importlib-metadata==8.4.0 ; python_version >= "3.8" and python_version < "3.10"
+idna==3.4 ; python_version >= "3.8" and python_version < "4"
+importlib-metadata==6.8.0 ; python_version >= "3.8" and python_version < "3.10"
 iniconfig==2.0.0 ; python_version >= "3.8" and python_version < "4"
 jinja2-time==0.2.0 ; python_full_version >= "3.8.0" and python_version < "4"
-jinja2==3.1.4 ; python_version >= "3.8" and python_version < "4"
+jinja2==3.1.2 ; python_version >= "3.8" and python_version < "4"
 kmatch==0.5.0 ; python_full_version >= "3.8.0" and python_version < "4"
-markdown==3.7 ; python_version >= "3.8" and python_version < "4"
-markupsafe==2.1.5 ; python_version >= "3.8" and python_version < "4"
+markdown==3.5 ; python_version >= "3.8" and python_version < "4"
+markupsafe==2.1.3 ; python_version >= "3.8" and python_version < "4"
 mergedeep==1.3.4 ; python_version >= "3.8" and python_version < "4"
-mkdocs-autorefs==1.1.0 ; python_version >= "3.8" and python_version < "4"
+mkdocs-autorefs==0.5.0 ; python_version >= "3.8" and python_version < "4"
 mkdocs-get-deps==0.2.0 ; python_version >= "3.8" and python_version < "4"
 mkdocs-material-extensions==1.3.1 ; python_version >= "3.8" and python_version < "4"
 mkdocs-material==9.5.33 ; python_version >= "3.8" and python_version < "4"
 mkdocs==1.6.0 ; python_version >= "3.8" and python_version < "4"
 mkdocstrings-python==1.10.8 ; python_version >= "3.8" and python_version < "4"
 mkdocstrings==0.25.2 ; python_version >= "3.8" and python_version < "4"
 mypy-extensions==1.0.0 ; python_version >= "3.8" and python_version < "4"
-nodeenv==1.9.1 ; python_full_version >= "3.8.0" and python_version < "4"
+nodeenv==1.8.0 ; python_full_version >= "3.8.0" and python_version < "4"
 packaging==24.1 ; python_version >= "3.8" and python_version < "4"
-paginate==0.5.7 ; python_version >= "3.8" and python_version < "4"
-pathspec==0.12.1 ; python_version >= "3.8" and python_version < "4"
+paginate==0.5.6 ; python_version >= "3.8" and python_version < "4"
+pathspec==0.11.2 ; python_version >= "3.8" and python_version < "4"
 pillow==10.0.1 ; python_version >= "3.8" and python_version < "4"
 platformdirs==4.2.2 ; python_version >= "3.8" and python_version < "4"
 pluggy==1.5.0 ; python_version >= "3.8" and python_version < "4"
-poetry-core==1.9.0 ; python_version >= "3.8" and python_version < "4.0"
+poetry-core==1.7.0 ; python_version >= "3.8" and python_version < "4.0"
 poyo==0.5.0 ; python_full_version >= "3.8.0" and python_version < "4"
-prompt-toolkit==3.0.47 ; python_full_version >= "3.8.0" and python_version < "4"
+prompt-toolkit==3.0.39 ; python_full_version >= "3.8.0" and python_version < "4"
 psycopg2-binary==2.9.9 ; python_full_version >= "3.8.0" and python_version < "4"
-pygments==2.18.0 ; python_version >= "3.8" and python_version < "4"
-pymdown-extensions==10.9 ; python_version >= "3.8" and python_version < "4"
+pygments==2.16.1 ; python_version >= "3.8" and python_version < "4"
+pymdown-extensions==10.3 ; python_version >= "3.8" and python_version < "4"
 pyproject-api==1.7.1 ; python_version >= "3.8" and python_version < "4"
 pyright==1.1.377 ; python_full_version >= "3.8.0" and python_version < "4"
 pytest-cov==5.0.0 ; python_version >= "3.8" and python_version < "4"
 pytest-django==4.8.0 ; python_version >= "3.8" and python_version < "4"
 pytest-dotenv==0.5.2 ; python_full_version >= "3.8.0" and python_version < "4"
 pytest-mock==3.6.1 ; python_full_version >= "3.8.0" and python_version < "4"
 pytest==8.3.2 ; python_version >= "3.8" and python_version < "4"
-python-dateutil==2.9.0.post0 ; python_version >= "3.8" and python_version < "4"
-python-dotenv==1.0.1 ; python_version >= "3.8" and python_version < "4"
-python-gitlab==4.9.0 ; python_full_version >= "3.8.0" and python_version < "4"
-python-slugify==8.0.4 ; python_full_version >= "3.8.0" and python_version < "4"
-pytz==2024.1 ; python_version >= "3.8" and python_version < "3.9"
+python-dateutil==2.8.2 ; python_version >= "3.8" and python_version < "4"
+python-dotenv==1.0.0 ; python_version >= "3.8" and python_version < "4"
+python-gitlab==3.15.0 ; python_full_version >= "3.8.0" and python_version < "4"
+python-slugify==8.0.1 ; python_full_version >= "3.8.0" and python_version < "4"
+pytz==2023.3.post1 ; python_version >= "3.8" and python_version < "3.9"
 pyyaml-env-tag==0.1 ; python_version >= "3.8" and python_version < "4"
-pyyaml==5.3.1 ; python_version >= "3.8" and python_version < "4"
-regex==2024.7.24 ; python_version >= "3.8" and python_version < "4"
-requests-file==2.1.0 ; python_version >= "3.8" and python_version < "4"
+pyyaml==6.0.1 ; python_version >= "3.8" and python_version < "4"
+regex==2023.10.3 ; python_version >= "3.8" and python_version < "4"
+requests-file==1.5.1 ; python_version >= "3.8" and python_version < "4"
 requests-toolbelt==1.0.0 ; python_full_version >= "3.8.0" and python_version < "4"
-requests==2.32.3 ; python_version >= "3.8" and python_version < "4"
+requests==2.31.0 ; python_version >= "3.8" and python_version < "4"
 ruff==0.6.2 ; python_full_version >= "3.8.0" and python_version < "4"
-setuptools==73.0.1 ; python_version >= "3.8" and python_version < "4"
+setuptools==68.2.2 ; python_version >= "3.8" and python_version < "4"
 six==1.16.0 ; python_version >= "3.8" and python_version < "4"
-soupsieve==2.6 ; python_version >= "3.8" and python_version < "4"
-sqlparse==0.5.1 ; python_version >= "3.8" and python_version < "4"
+soupsieve==2.5 ; python_version >= "3.8" and python_version < "4"
+sqlparse==0.4.4 ; python_version >= "3.8" and python_version < "4"
 text-unidecode==1.3 ; python_full_version >= "3.8.0" and python_version < "4"
-tldextract==5.1.2 ; python_version >= "3.8" and python_version < "4"
+tldextract==5.0.0 ; python_version >= "3.8" and python_version < "4"
 tomli==2.0.1 ; python_version >= "3.8" and python_full_version <= "3.11.0a6"
 tox==4.18.0 ; python_version >= "3.8" and python_version < "4"
-types-python-dateutil==2.9.0.20240821 ; python_version >= "3.8" and python_version < "4"
-types-pyyaml==6.0.12.20240808 ; python_version >= "3.8" and python_version < "4"
+types-python-dateutil==2.8.19.14 ; python_version >= "3.8" and python_version < "4"
+types-pyyaml==6.0.12.20240311 ; python_version >= "3.8" and python_version < "4"
 typing-extensions==4.12.2 ; python_version >= "3.8" and python_version < "4"
-tzdata==2024.1 ; python_version >= "3.8" and python_version < "4" and sys_platform == "win32"
-urllib3==2.2.2 ; python_version >= "3.8" and python_version < "4"
+tzdata==2023.3 ; python_version >= "3.8" and python_version < "4" and sys_platform == "win32"
+urllib3==2.0.6 ; python_version >= "3.8" and python_version < "4"
 virtualenv==20.26.3 ; python_version >= "3.8" and python_version < "4"
-watchdog==4.0.2 ; python_version >= "3.8" and python_version < "4"
-wcwidth==0.2.13 ; python_full_version >= "3.8.0" and python_version < "4"
-wheel==0.44.0 ; python_version >= "3.8" and python_version < "3.9"
-zipp==3.20.1 ; python_version >= "3.8" and python_version < "3.10"
+watchdog==3.0.0 ; python_version >= "3.8" and python_version < "4"
+wcwidth==0.2.8 ; python_full_version >= "3.8.0" and python_version < "4"
+wheel==0.43.0 ; python_version >= "3.8" and python_version < "3.9"
+zipp==3.17.0 ; python_version >= "3.8" and python_version < "3.10"

docs/settings.md

@@ -14,6 +14,18 @@ The JSON encoder class or class path to use when serializing context.
 
 **Default** `"django.core.serializers.json.DjangoJSONEncoder"`
 
+## PGHISTORY_INSTALL_CONTEXT_FUNC_ON_MIGRATE
+
+Install the Postgres `_pgh_attach_context` function after migrations. Ensures pghistory context tracking works even without running migrations, typically for test suites.
+
+**Default** `False`
+
+## PGHISTORY_CREATED_AT_FUNCTION
+
+Pghistory uses `NOW()` to determine the current time, which is the same timestamp for the entire transaction. Configure this setting to `CLOCK_TIMESTAMP()`, for example, if you'd like to use a different SQL function for determining the current time.
+
+**Default** `"NOW()"`
+
 ## PGHISTORY_BASE_MODEL
 
 The base model to use for event models.

pghistory/apps.py

@@ -1,12 +1,20 @@
 import django.apps
-from django.db.models.signals import class_prepared
+from django.db.models.signals import class_prepared, post_migrate
+
+from pghistory import config
 
 
 def pgh_setup(sender, **kwargs):
     if hasattr(sender, "pghistory_setup"):
         sender.pghistory_setup()
 
 
+def install_on_migrate(using, **kwargs):  # pragma: no cover
+    if config.install_context_func_on_migrate():
+        Context = django.apps.apps.get_model("pghistory", "Context")
+        Context.install_pgh_attach_context_func(using=using)
+
+
 class PGHistoryConfig(django.apps.AppConfig):
     name = "pghistory"
 
@@ -23,3 +31,5 @@ def __init__(self, *args, **kwargs):
     def ready(self):
         # Register custom checks
         from pghistory import checks  # noqa
+
+        post_migrate.connect(install_on_migrate, sender=self)