Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tox + uv caching #29

Merged
merged 13 commits into from
Dec 2, 2024
Merged

Fix tox + uv caching #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
0a87824
Split tox setup and run
dfsnow Dec 2, 2024
31d5198
Add env cache
dfsnow Dec 2, 2024
9ff3c3f
Pass UV_CACHE_DIR
dfsnow Dec 2, 2024
fa7169d
Disable cache pruning
dfsnow Dec 2, 2024
a580397
Add full uv workflow and caching
dfsnow Dec 2, 2024
c9ce0de
Finalize GH workflows with uv
dfsnow Dec 2, 2024
cc4dcad
Change uv cache suffixes
dfsnow Dec 2, 2024
7d2d3a5
Use system Python for tox
dfsnow Dec 2, 2024
10322e7
Use uv python for tox
dfsnow Dec 2, 2024
e311aff
Drop caching from pages
dfsnow Dec 2, 2024
71ac9f2
Disable cache on lint workflow
dfsnow Dec 2, 2024
3cda093
Cache for each Python version
dfsnow Dec 2, 2024
55e3225
Drop dedicated linting PR
dfsnow Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 0 additions & 35 deletions .github/workflows/lint.yaml
View file
Open in desktop

This file was deleted.

14 changes: 4 additions & 10 deletions .github/workflows/pages.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ name: pages

env:
PYTHONUNBUFFERED: "1"
UV_SYSTEM_PYTHON: 1

jobs:
build-sphinx-site:
Expand All @@ -22,22 +21,17 @@ jobs:
uses: actions/configure-pages@v5

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: pyproject.toml
cache-suffix: docs
uses: astral-sh/setup-uv@v4
Comment on lines 23 to +24
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually faster to run this without using the cache.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Praise] Common uv W 🚀


- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: pyproject.toml
shell: bash
run: uv venv

- name: Install Python dependencies
run: uv pip install .[docs]

- name: Generate HTML
run: sphinx-build -d _build/doctrees docs/source _build/html
run: uv run sphinx-build -d _build/doctrees docs/source _build/html

- name: Upload artifact
uses: actions/upload-pages-artifact@v3
Expand Down
14 changes: 1 addition & 13 deletions .github/workflows/pypi-publish.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ name: pypi-publish

env:
PYTHONUNBUFFERED: "1"
UV_SYSTEM_PYTHON: 1

jobs:
pypi-publish:
Expand All @@ -23,18 +22,7 @@ jobs:
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: pyproject.toml

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: pyproject.toml

- name: Install Python dependencies
run: uv pip install .
Comment on lines -26 to -37
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of this stuff is actually necessary to build the package, so we can remove it.

uses: astral-sh/setup-uv@v4

- name: Build Python dist
run: uv build
Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/python-build-and-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ on:

name: python-build-and-test

env:
PYTHONUNBUFFERED: "1"

jobs:
build-and-test:
runs-on: ubuntu-latest
Expand All @@ -19,11 +22,11 @@ jobs:
uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
uses: astral-sh/setup-uv@v4
with:
enable-cache: true
cache-dependency-glob: pyproject.toml
cache-suffix: tox
cache-suffix: ${{ matrix.python-version }}-tox
Copy link
Member Author

@dfsnow dfsnow Dec 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the cause of the caching issue identified in ccao-data/ccao#33. Basically, uv disables caching in CI by default because it's slower to recover the cache than it is to just uv pip install (wild).

It's supposed to cache built-from-source wheels even when pruning is enabled. However, there's a race condition in the testing setup since the first-to-finish jobs are the ones to populate the cache, and those jobs won't have the built-from-source wheel.

The easy fix here is to just give each Python version its own cache.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Praise] This is wild 🤯 Great find!


- name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ python =
[testenv]
extras = dev
commands = pytest
passenv =
UV_CACHE_DIR
PYTHONUNBUFFERED
Comment on lines +89 to +91
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly necessary I don't think, but also can't hurt.


[testenv:py{39, 310}-lowest]
uv_resolution = lowest-direct
Expand Down
Loading