Django 5 updates; restructure for package installs

[ColeDCrawford]

See also: artshumrc#4

Change Summary

This PR restructures Catchpy so that it can continue to be run as a standalone Django app but makes it possible to install Catchpy into an existing Django application. This greatly facilitates research applications as they can reference annotation data directly from the same database. It also upgrades Catchpy to work with Django 5.x applications.

• Move apps anno and consumer into the project directory: catchpy. This makes is easier to package Catchpy into a Python wheel.
• Rename consumer.Profile to consumer.CatchpyProfile to avoid clashes with existing projects.
• Fixed token creation bug
• Added requirements file as Dockerfile ARG so that local.txt is default, but docker-compose.yml specifies dev.txt, which enables running tests directly in the dev container.
• Converted legacy setup.py to pyproject.toml packaged a distribution with hatch
• Refactor urlpatterns so that the admin URLs are only included when running as a standalone app and the others can be included in an existing project.
• Handle case when JWT is not prefixed with token details
• Ensure that CatchpyProfile one to one relationship exists before attempting to access it.
• Update readme with instructions for adding to existing project and packaging instructions.
• Update tests

Checklist

• PR title is descriptive
• Added/Modified tests for the changes (or explain why not relevant)
• Manually tested the changes and investigated potential regressions
• Documented changes (where applicable)

[ColeDCrawford]

Some questions that can help guide next steps on this PR:

• What Python and PSQL versions do HX and AT need to support?
  • Python 3.8 is EOL, so it probably makes sense to drop that and try to see whether 3.13 works in the test matrix. All of DARTH's apps are on Django 5.x / Python 3.12 at the moment. Python 3.8 and 3.9 are still passing tests (resolving to Django 4.2), but Django 5.x requires Python 3.10.x+
  • PSQL - docker-compose isn't currently pinning a version for this, we should probably choose a major at least
• How are HX and AT deploying CatchPy, so we make sure to keep a variety of infrastructure approaches in mind?

As Naomi mentioned in the meeting, semantic versioning definitely makes sense here. We had some separate tags in our fork but happy to adjust to use a different version number for the upstream release.

I think since this repo is in a personal account there are no teams configured (e.g. @darth-dev). Tagging @arthurian for visibility (Vesna @Tanvez came to the meeting but since he hasn't contributed to the repo I don't think tagging him will work)

[nmaekawa]

python version
we are ok with 3.12

postgres version
django4.2 requires pg12 or higher, and since we finally moved to pg16, we are ok with pg15~16

catchpy deploy for hx
we don't usually package it and pull source from github. An artifact repo would be nice if only available within harvard so we don't set expectations of maintenance in the large interwebs, checking with huit if they have any service that we can use.

semantic version
for our tag system, this merge would be 3.0.0, but ok to skip ahead if it helps DARTH to adjust.

personal git acct
will think about moving to harvardx acct, but not very high priority right now.

[ColeDCrawford]

python version we are ok with 3.12
postgres version django4.2 requires pg12 or higher, and since we finally moved to pg16, we are ok with pg15~16

Great, we can update the Dockerfiles / Compose files to reflect these.

catchpy deploy for hx we don't usually package it and pull source from github. An artifact repo would be nice if only available within harvard so we don't set expectations of maintenance in the large interwebs, checking with huit if they have any service that we can use.

Let us know if you hear anything about this. On our fork, we've just been building the wheels as part of the releases: https://github.com/artshumrc/catchpy/releases. We then include this in project pyproject.toml files as "catchpy @ git+https://github.com/artshumrc/[email protected]". This seems to split the difference of making it easy to include within a project without setting expectations of maintenance (e.g. PyPi).

semantic version for our tag system, this merge would be 3.0.0, but ok to skip ahead if it helps DARTH to adjust.

3.0.0 works for us as well, that makes sense especially if this introduces any backwards incompatible changes that you'd need to update infrastructure to account for.

personal git acct will think about moving to harvardx acct, but not very high priority right now.

Let me know if you have issues here - I've had good experiences with this recently. I moved the artshumrc and shariasource Github orgs to the Harvard Enterprise Github in the past few years. This provided a number of benefits (private runners, internal repos, easier to collaborate and use teams, etc). You need to contact the service owner for this, but it usually only takes a couple of days to be approved and migrated. And I've moved a number of repos from personal to org accounts - this usually takes <15 minutes.