1.0.1

Migrations

None

Documentation

• Doc with Furo (#888)

1.0.0

A word from the maintainer:

This is the first major release for Procrastinate, and the 50th release of the project. The project has been moving a lot up until 2021, and has slowed down since. In the last year or so, it was almost dormant. One of the reasons was the poor state of sync vs async compatibility code, which made new contributions more complex. It took a few month to gather the motivation for solving this, but the release of the excellent Psycopg3, as well as the opportunity to use asgiref's sync/async compatibility code opened the door for new solutions.

This release wouldn't have existed without the help of the users who gave their opinion in #882, and especially @paulzakin and their team who tested beta versions in real-life scenarios.

I'd love for the procrastinate developer team to be bigger, as I'm the only person committing to the project these days (... except for dependency bots, of course). If you use Procrastinate in your life, and would like to give back a little bit of time now and then, please don't hesitate to come say hi (#748).

Happy new year, happy upgrading, and let us know if anything breaks :)
Cheers, Joachim

Migrations

None

Breaking changes

The following is a consequence of the merge of #753 (see also #882)

• Aiopg and Psycopg2 connectors are now contrib. To use them, you'll need to import them with from procrastinate.contrib.aiopg import AiopgConnector and from procrastinate.contrib.psycopg2 import Psycopg2Connector. Also, the dependencies to aiopg and psycopg2 are now optional: use pip install procrastinate[aiopg] or pip install procrastinate[psycopg2] (or pip install procrastinate[aiopg,psycopg2] for both)
• The main supported connector is now the PsycopgConnector that uses Psycopg 3 (and SyncPsycopgConnector). Please note that PsycopgConnector accepts parameters based on psycopg_pool.ConnectionPool, which has a slightly different signature from the psycopg2/aiopg counterpart, the main difference is that the connection arguments are now passed as PsycopgConnector(kwargs={"host": "..."}) (instead of AiopgConnector(host=...)). Other parameters may have changed too, please check the documentation.
• Actually, you probably won't need sync connectors anymore because the Async connectors are now able to derive a sync connector when called in a sync context. You should try defining a single (async) connector in your code such as PsycopgConnector (or AiopgConnector), see if it works with all your existing code and don't hesitate to report potential issues. Synchronous connectors are still available in case you need it, so it should be a workaround in most cases.
• Synchronous tasks are now launched asynchronously in a ThreadPoolExecutor using asgiref.sync_to_async. That said, because of the Global Interpreter Lock (GIL), CPU-consuming tasks will not run faster with parallelization.
• The CLI parser has changed. The main differences should be around the order of arguments vs flag, and around environment variables. If you find something unexpected, please open an issue.
• Opening your app at the same time you instantiate it is now discouraged. Ideally, define your app as a module variable, and open it in the appropriate function (when your process starts). In the future, we may expose new helpers for doing this easily with django. Note that when you use the Procrastinate CLI, it takes care of opening/closing the app properly.
• JobManager.check_connection was an async method. It became check_connection_async for consistency. check_connection was created a the sync counterpart.

Dependencies

• Update Deps with major upgrades (major) (#875)

Kudos:

@paulzakin for testing, and the folks who contributed to #882

0.34.0b2

Fix issue in main

0.34.0b1: A pre-release for testing sync/async changes to come

Migrations

None

Breaking changes

#753 :

• Aiopg and Psycopg2 connectors are now contrib. To use them, you'll need to import them with from procrastinate.contrib.aiopg import AiopgConnector and from procrastinate.contrib.psycopg2 import Psycopg2Connector. Also, the dependencies to aiopg and psycopg2 are now optionnal: use pip install procrastinate[aiopg] or pip install procrastinate[psycopg2] (or pip install procrastinate[aiopg,psycopg2] for both)
• The main supported connector is now the PsycopgConnector that use Psycopg 3 (and SyncPsycopgConnector)
• Actually, you probably won't need sync connectors anymore because the Async connectors are now able to derive a sync connector when called in a sync context. So try defining a single (async) connector in your code such as PsycopgConnector (or AiopgConnector).
• Synchronous tasks are now launched asynchronously in a ThreadPoolExecutor using asgiref.sync_to_async.
• Some CLI commands that used to accept parameter in any order now won't. It should be mainly mixing positional CLI args vs flags: you used to be able to do procrastinate defer x --unknown {} and now you'll have to put all positional arguments first and flags last: procrastinate defer x {} --unknown. We can't guarantee that there won't be other subtle breaks around this (e.g. environment variable support). That said, if you discover a change, feel free to open a ticket, if it's easily fixable, we might fix it.
• Opening your app at the same time you instantiate it is now discouraged. Ideally, define your app as a module variable, and open it in the appropriate function (when your process starts). In the future, we may be exposing helpers for doing this easily with django. Note that when you use the Procrastinate CLI, it takes care of opening/closing the app properly.

Discussion about these changes happen in #882

0.33.0: Fix import error

Migrations

None

Bugfix

• Don't throw an importError if psycopg(3) can't be imported, unless user is actually using it (#881)

Kudos:

@ashleyheath

0.32.0: Better handle BaseException in Worker

Migrations

None

• Ensure jobs are scheduled for retry regardless of exception type raised (#878)

Dependencies

• [pre-commit.ci] pre-commit autoupdate (#877)

Kudos:

@ashleyheath

0.31.0: Same as 0.30.0

(We needed a new version number for administrative reasons)

0.30.0: Psycopg v3 compatibility

Migrations

https://github.com/procrastinate-org/procrastinate/tree/main/procrastinate/sql/migrations

Features

• Add psycopg3 compat (#863)

Kudos:

@aleksandr-shtaub

0.29.1: Open files with explicit encodings

Migrations

None

Bugs

• Set encoding=utf-8 when reading files to avoid warnings (#873)

Dependencies

#868, #870, #867, #866, #864, #862, #861, #860, #858, #859, #857, #855, #856, #853, #851, #850, #847, #848, #872, #849

Kudos:

@ashleyheath

0.30.0rc1: A preview of breaking changes to come in #753

Migrations

None

Breaking changes

#753 brings import changes to the way sync and async work in procrastinate.

• We've tried to remove some place where we "seemlessly" executed async code from sync calls because it wasn't seemless at all. Most of the sync methods that are just wrappers over async methods with syntax sugar to avoid having to run the even loop yourself, have been removed. This means all the sync methods of JobManager and App.check_connection. The AsyncConnector subclasses don't support being called via sync methods anymore. The way to fix this if you were using it is to use async code when interacting with Procrastinate's systems (so from sync code, either use asyncio.run or asgiref.sync.async_to_sync and call async methods).

• The remaining methods on the sync API are:

  • Task.defer which is implemented as a synchronous function and will stay that way
  • App.run_worker has been reimplemented on top of App.run_worker_async. It uses its own event loop. This is probably the only case where it makes sense that Procrastinate provides a sync shortcut because it's a long-lived call. For all other async methods, trying to handle the event loop in Procrastinate is likely to cause more problem than it solves, whereas the caller is in a much better solution to solve it the right way as they're in control of the process they run the code from.
  • Providing your synchronous tasks for Procrastinate to run has a much better support now, thanks to
    asgiref.sync.sync_to_async.

  Those are the 3 only part of the synchronous Procrastinate API that are left. We don't plan to remove them, and they're enough to get almost all of the nice things of procrastinate in your sync project. If Procrastinate doesn't drastically change in the future, there's no reason we'd want to add more sync stuff.

• Due to dropping Click in favor of argparse for async reasons, some commands that accepted parameter in any order now don't. It should be mainly mixing positional CLI args and flags: you used to be able to do procrastinate defer x --unknown {} and now you'll have to put all positional args first: procrastinate defer x {} --unknown. I can't guarantee that there won't be other subtle breaks around this (e.g. environment variable support). That said, if you discover a change, feel free to open a ticket, if it's easily fixable, we might fix it.

• You may have issues if you call App.open right after creating the app.

  • Ideally, open the app only in the context where you know you'll be using it (in a given process, you'll likely either use the sync app or the async app, not both, so try to open only the one you'll use)
  • Ideally, try to open the app at the start of the process and close it at the end. It might not be easy to find the exact right place to do so, but we'll try to give some advice. For Django, for example, that might be an AppConfig.ready() and atexit (from the standard lib). YMMV.
  • For app configured with a sync connectors, use app.open() and app.close() (or with app.open()). With an async connector, use await app.open_async() and await app.close_async() or async with app.open_async(). Calling app.open() on an async app is not supported anymore.

Dependencies

• Lock file maintenance (#856)
• Lock file maintenance (#853)
• Bump urllib3 from 2.0.6 to 2.0.7 (#851)
• Lock file maintenance (#850)
• Update Deps with major upgrades to v2 (major) (#847)
• Lock file maintenance (#848)
• [pre-commit.ci] pre-commit autoupdate (#849)

Kudos:

@dependabot, @dependabot[bot], @ewjoachim, @pre-commit-ci, @pre-commit-ci[bot], @renovate and @renovate[bot]