Update Python dependencies

[jbethune]

I've tried to install DearMEP on a newish Linux machine and I got all kinds of C++ compiler errors when using Poetry.
In order to keep things simple and clean, I used the official docker.io/library/python image as a starting point.

There have been some changes in the world of Python packaging and various packages fail to compile when using the current poetry.lock file. I set the required Python version to 3.13 and I ran poetry update to get newer packages. I also had to install the debian package graphviz-dev for some header files. After that I could run poetry install --all-extras. I also needed to update the python package pillow to version 11 from version 9.5 because of C++ compiler errors (this might be a breaking change).

Long story short: I think it would be good to update the Python dependencies in poetry.lock and also set the Python version to a higher number than ^3.8. Is there something that speaks agains that? Debian bookworm is currently at python 3.11.

[scy]

Thanks for letting us know.

I've tried to install DearMEP on a newish Linux machine and I got all kinds of C++ compiler errors when using Poetry.

"all kinds of errors" is not exactly an actionable bug report. ;)

There have been some changes in the world of Python packaging and various packages fail to compile when using the current poetry.lock file.

The CI builds succeed on 3.8 and 3.11. I haven't tested other versions yet.

I also had to install the debian package graphviz-dev for some header files.

That's for eralchemy2 in the specs extra described in the readme, which is really just a gimmick on top. Normal users won't need it. Once DearMEP is on PyPI, we will make sure to tell users that they don't need all extras.

I set the required Python version to 3.13 and I ran poetry update to get newer packages.
[…]
After that I could run poetry install --all-extras. I also needed to update the python package pillow to version 11 from version 9.5 because of C++ compiler errors (this might be a breaking change).

We're only using Pillow for file format detection (server/dearmep/convert/image.py), it's probably trivial to check whether upgrading Pillow breaks anything.

Long story short: I think it would be good to update the Python dependencies in poetry.lock and also set the Python version to a higher number than ^3.8. Is there something that speaks agains that? Debian bookworm is currently at python 3.11.

My goal is to

• migrate from flake8 to Ruff (and fix or ignore all of the things Ruff currently complains about)
• migrate from Poetry to uv
• change the lowest supported Python version to 3.9 and the highest to 3.13, conforming to current upstream lifecycle

The Ruff task has already started. The others will follow after that. But we should do this one step after the other, or else the merges will be a world of pain.

I'm planning to do all of that in the next couple of weeks, but I'm afraid that it's not work that can be split among multiple people easily.

[scy]

My goal is to

• migrate from flake8 to Ruff (and fix or ignore all of the things Ruff currently complains about)

Done in #247.

• migrate from Poetry to uv

Done in #282.

• change the lowest supported Python version to 3.9

Done in #283.

and the highest to 3.13, conforming to current upstream lifecycle

In Progress in #284.

Once those are all done, I'll tackle security updates.

[scy]

We now support Python 3.13 as well, #284 is done.

It's gonna take me a day or two until I can get back to the security stuff. Also, switching to uv means that Dependabot stopped working; it self-closed all of its pip-based alerts once we no longer had a poetry.lock file. uv support in Dependabot is coming, but there's no ETA yet.

[scy]

uv run --with pip-audit pip-audit -s osv seems to be a good replacement for Dependabot for now.

[scy]

$ uv run --with pip-audit pip-audit -s osv
Found 16 known vulnerabilities in 10 packages
Name             Version   ID                  Fix Versions
---------------- --------- ------------------- -------------
certifi          2022.9.24 PYSEC-2022-42986    2022.12.7
certifi          2022.9.24 PYSEC-2023-135      2023.7.22
certifi          2022.9.24 GHSA-248v-346w-9cwc 2024.7.4
fastapi          0.95.1    PYSEC-2024-38       0.109.1
idna             3.4       PYSEC-2024-60       3.7
jinja2           3.1.2     GHSA-h5c8-rqwp-cp95 3.1.3
jinja2           3.1.2     GHSA-h75v-3vvj-5mfj 3.1.4
pygments         2.14.0    PYSEC-2023-117      2.15.1
python-multipart 0.0.7     GHSA-59g5-xgcq-4qw3 0.0.18
requests         2.31.0    GHSA-9wx4-h78v-vm56 2.32.0
starlette        0.26.1    PYSEC-2023-83       0.27.0
starlette        0.26.1    GHSA-f96h-pmfr-66vw 0.40.0
urllib3          1.26.12   PYSEC-2023-192      1.26.17,2.0.6
urllib3          1.26.12   PYSEC-2023-212      1.26.18,2.0.7
urllib3          1.26.12   GHSA-34jh-p97f-mpxf 2.2.2
zipp             3.15.0    GHSA-jfmj-5v4g-7637 3.19.1

Starting to work on these now.

[scy]

$ uv run --with pip-audit pip-audit -s osv
No known vulnerabilities found

😎