Backend work for public collections: thumbnails, url list, upload pages, and so on

[tw4l]

Fixes #2182

This rather large PR adds the rest of what should be needed for public collections work in the frontend.

New API endpoints

• /public/orgs/{org_slug}/collections/{coll_id}: Public GET endpoint for public collection (returns 404 collection_not_found if org with slug doesn't exist or isn't public, collection id doesn't exist or isn't public)
• • /public/orgs/{org_slug}/collections/{coll_id}/download: Streaming download endpoint for public collection (returns 404 collection_not_found if org with slug doesn't exist or isn't public, collection id doesn't exist or isn't public; 403 if Collection.allowPublicDownload is false)
• /orgs/{oid}/collections/{coll_id}/urls: Paginated GET list of urls in collection sorted descending by snapshot count, with information about each individual snapshot (including page_id, needed for below), for use in selecting a home url/snapshot for collections in the frontend
• /orgs/{oid}/collections/{coll_id}/home-url: POST endpoint to set collection home url/start snapshot by page_id
• /orgs/{oid}/collections/{coll_id}/thumbnail: PUT endpoint to upload collection thumbnail as a stream. Will replace existing thumbnail if called on a collection that already has one.
• /orgs/{oid}/collections/{coll_id}/thumbnail: DELETE endpoint to remove collection thumbnail

Changes to existing API endpoints

• /public-collections/{org_slug}: Public collection list now supports pagination, only returns public fields for collections, and has had the path modified to /public/orgs/{org_slug}/collections following offline feedback
• Several pages endpoints that previously only supported /crawls/ in their path, e.g. /orgs/{oid}/crawls/all/pages/reAdd, now support /uploads/ and /all-crawls/ namespaces as well. This is necessitated by adding pages for uploads to the database (see below). For /orgs/{oid}/namespace/all/pages/reAdd, crawls or uploads will serve as a filter to only affect crawls of that given type. Other endpoints are more liberal at this point, and will perform the same action regardless of the namespace used in the route (we'll likely want to change this in a follow-up to be more consistent, but I didn't want to expand the scope of this PR even larger than it already has been).
• /orgs/{oid}/namespace/all/pages/reAdd now kicks off a background job rather than doing all of the computation in an asyncio task in the backend container. The background job additionally updates collection date ranges, page/size counts, and tags for each collection in the org after pages have been (re)added.
• The collection PATCH endpoint now allows the user to set a default thumbnail that is served by the frontend via the defaultThumbnailName field, as well as the new allowPublicDownload field to determine whether the collection is downloadable when/if it is made public.

Other big changes

• New uploads will now have their pages read into the database! Collection page counts now also include uploads!
• A migration was added to start a background job for each org that will add the pages for previously-uploaded WACZ files to the database and update collections accordingly!
• Adds a new ImageFile subclass of BaseFile for thumbnails that we can use for other user-uploaded image files moving forward, with separate output models for authenticated and public endpoints

Other less big changes

• Collections now have a dateEarliest and dateLatest that we can generate a range from, based on the earliest and latest snapshot in the collection
• Public collections now have their own model that restricts the fields we share to unauthenticated users
• Collections now have a caption field that can be used to set a short caption to use for public collection views (presumably with more visibility than full description)
• The process of adding pages to the database from WACZ files is more flexible to account for older WACZ files whose page lists may deviate from the current norm (e.g. non-UUID page ids)

Testing

Backend tests have been added or modified for all API endpoint changes.

To test the migration:

• Spin up main locally
• Upload some WACZ files in one or more orgs and add them to some collections
• Re-deploy from this branch
• Verify that migration appeared to work successfully in backend logs
• Verify that background job container (in default namespace) logs look good
• Verify that pages have been added for uploads and that collections have been updated accordingly (look at data in Network tab for collection replay.json endpoint on the collection page)

[tw4l]

@SuaYoo I added a max size validation for thumbnails of 2 MB (following Youtube's example). It'll be easy to raise it or set different limits for other types of user-uploaded image files in the future. I'm wondering if we also want to restrict file types? I'm calculating a mime type for the image files based on the filename they're uploaded with, we could use that or try to detect via magic numbers

[SuaYoo]

Based on the availability of API endpoints, this looks like it covers everything we need on the backend so far! I'm also able to get it running locally.

[SuaYoo]

Documenting some feedback from Discord:

• Ran into an issue when using the /urls endpoint with prefix search. urlPrefix doesn't seem to accept values after ?, for example [frontend/src/features/collections/select-collection-start-page.ts](https://en.wikipedia.org/w/index.php?title=Special:CreateAccount&returnto=Main+Page). Passing in an encoded value to urlPrefix doesn't return any results.
• Unlisted collections should still return in the public GET endpoint, regardless of whether the org itself is public.

[tw4l]

Documenting some feedback from Discord:

* Ran into an issue when using the `/urls` endpoint with prefix search. `urlPrefix` doesn't seem to accept values after `?`, for example `[frontend/src/features/collections/select-collection-start-page.ts](https://en.wikipedia.org/w/index.php?title=Special:CreateAccount&returnto=Main+Page)`. Passing in an encoded value to `urlPrefix` doesn't return any results.

* Unlisted collections should still return in the public GET endpoint, regardless of whether the org itself is public.

@SuaYoo these are both now fixed, pushed to dev, and tested

[SuaYoo]

Tested newest changes locally, working well!

[SuaYoo]

@tw4l mind if I merge this into the feature branch?

[tw4l]

@tw4l mind if I merge this into the feature branch?

done! :)