Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP - Pydantic v2 support #1238

Merged
merged 97 commits into from
Mar 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
31f9a64
WIP
collerek Sep 19, 2023
c95aafc
WIP - make test_model_definition tests pass
collerek Dec 13, 2023
a1204d8
WIP - make test_model_methods pass
collerek Dec 13, 2023
7f351ec
WIP - make whole test suit at least run - failing 49/443 tests
collerek Dec 13, 2023
ffdbf4a
WIP fix part of the getting pydantic tests as types of fields are now…
collerek Dec 14, 2023
14ffb79
WIP fix validation in update by creating individual fields validators…
collerek Dec 14, 2023
4fa7b31
WIP fix __pydantic_extra__ in intializing model, fix test related to …
collerek Dec 14, 2023
f7a7efb
WIP - fix enum schema in model_json_schema, failing 31/442
collerek Dec 14, 2023
6c2e92a
WIP - fix copying through model, fix setting pydantic fields on throu…
collerek Dec 15, 2023
7e40719
WIP fix tests checking pydantic schema, fix excluding parent fields, …
collerek Dec 15, 2023
6809173
WIP some missed files
collerek Dec 15, 2023
aacfebb
WIP - fix validators inheritance and fix validators in generated pyda…
collerek Dec 15, 2023
0a6ca89
WIP - fix through models setting - only on reverse side of relation, …
collerek Dec 15, 2023
fdb0ad9
WIP - fix through models setting - only on reverse side of relation, …
collerek Dec 15, 2023
cca0c8b
WIP - working on proper populating __dict__ for relations for new sch…
collerek Dec 16, 2023
69660f7
WIP - remove property fields as pydantic has now computed_field on it…
collerek Dec 17, 2023
b91bd49
WIP - fixes in docs, failing 8/442
collerek Dec 17, 2023
8c688b8
WIP - fix tests for largebinary schema, wrapped bytes fields fail in …
collerek Dec 19, 2023
5cb310a
WIP - fix to pk only models in schemas
collerek Jan 17, 2024
6ad086b
Getting test suites to pass (#1249)
TouwaStar Jan 23, 2024
baac771
solve circular refs
collerek Jan 24, 2024
8a799c1
all tests pass :tada:
collerek Jan 25, 2024
a737001
remove 3.7 from tests
collerek Jan 25, 2024
c5a10fb
add lint and type check jobs
collerek Jan 25, 2024
87ecdbc
reforat with ruff, fix jobs
collerek Jan 25, 2024
e4c9778
rename jobs
collerek Jan 25, 2024
b2a2ab6
fix imports
collerek Jan 25, 2024
8d29c1f
fix evaluate in py3.8
collerek Jan 25, 2024
d286de5
partially fix coverage
collerek Jan 25, 2024
f27c1d5
fix coverage, add more tests
collerek Jan 26, 2024
8c75752
fix test ids
collerek Jan 26, 2024
444d418
fix test ids
collerek Jan 26, 2024
e090dc4
fix lint, fix docs, make docs fully working scripts, add test docs job
collerek Jan 26, 2024
ae60373
fix pyproject
collerek Jan 26, 2024
2988c9e
pin py ver in test docs
collerek Jan 26, 2024
8134797
change dir in test docs
collerek Jan 26, 2024
a762f8e
fix pydantic warning hack
collerek Jan 26, 2024
7c27c49
rm poetry call in test_docs
collerek Jan 26, 2024
5e709b6
switch to pathlib in test docs
collerek Jan 26, 2024
9b165d9
remove coverage req test docs
collerek Jan 26, 2024
df852fe
fix type check tests, fix part of types
collerek Jan 29, 2024
7d1e9bc
fix/skip next part of types
collerek Jan 29, 2024
d47132e
fix next part of types
collerek Jan 29, 2024
455e8b5
fix next part of types
collerek Jan 30, 2024
8590b14
fix coverage
collerek Jan 31, 2024
facfbb9
fix coverage
collerek Jan 31, 2024
d38c88e
fix type (bit dirty :shrug:)
collerek Jan 31, 2024
2943975
fix some code smells
collerek Jan 31, 2024
bbe1cde
change pre-commit
collerek Jan 31, 2024
d0b8154
tweak workflows
collerek Feb 2, 2024
7fd76e4
remove no root from tests
collerek Feb 2, 2024
46b39b1
switch to full python path by passing sys.executable
collerek Feb 2, 2024
a8b9fce
some small refactor in new base model, one sample test, change makefile
collerek Feb 2, 2024
e7a5a41
small refactors to reduce complexity of methods
collerek Feb 2, 2024
c65fcc9
temp add tests for prs against pydantic_v2
collerek Feb 5, 2024
5d6540e
remove all references to __fields__
collerek Feb 5, 2024
8e93de6
remove all references to construct, deprecate the method and update m…
collerek Feb 5, 2024
68dbcf6
deprecate dict and add model_dump, todo switch to model_dict in calls
collerek Feb 5, 2024
644de37
fix tests
collerek Feb 6, 2024
606f068
change to union
collerek Feb 6, 2024
08d8b02
change to union
collerek Feb 6, 2024
8ff428b
change to model_dump and model_dump_json from dict and json deprecate…
collerek Feb 6, 2024
1340008
finish switching dict() -> model_dump()
collerek Feb 6, 2024
d55e07b
finish switching json() -> model_dump_json()
collerek Feb 6, 2024
36aa05b
remove fully pydantic_only
collerek Feb 6, 2024
4b6ed02
switch to extra for payment card, change missed json calls
collerek Feb 6, 2024
ee4440b
fix coverage - no more warnings internal
collerek Feb 6, 2024
55f4c13
fix coverage - no more warnings internal - part 2
collerek Feb 6, 2024
b45d776
split model_construct into own and pydantic parts
collerek Feb 7, 2024
e68c3aa
split determine pydantic field type
collerek Feb 7, 2024
ab85c81
change to new field validators
collerek Feb 7, 2024
80a93b4
fix benchmarks, add codspeed instead of pytest-benchmark, add action …
collerek Feb 7, 2024
17bf9d5
restore pytest-benchmark
collerek Feb 7, 2024
ed47db4
remove codspeed
collerek Feb 7, 2024
9303ebf
pin pydantic version, restore codspeed
collerek Feb 8, 2024
851f1f1
change on push to pydantic_v2 to trigger first one
collerek Feb 8, 2024
27d798c
Use lifespan function instead of event (#1259)
mekanix Feb 11, 2024
8258090
check return types
collerek Feb 12, 2024
ead254c
Merge branch 'pydantic_v2' of https://github.com/collerek/ormar into …
collerek Feb 12, 2024
217faae
fix imports order, set warnings=False on json that passes the dict, f…
collerek Feb 12, 2024
cdce29d
remove references to model's meta as it's now ormar config, rename re…
collerek Feb 18, 2024
ac00c78
filter out pydantic serializer warnings
collerek Feb 18, 2024
2d79866
remove choices leftovers
collerek Feb 18, 2024
e7d1bdb
remove leftovers after property_fields, keep only enough to exclude t…
collerek Feb 25, 2024
1bbf908
add migration guide
collerek Feb 25, 2024
7517774
fix conflicts
collerek Mar 10, 2024
c8dc612
fix meta references
collerek Mar 10, 2024
4b0a501
downgrade databases for now
collerek Mar 10, 2024
27a981c
Change line numbers in documentation (#1265)
mekanix Mar 10, 2024
280acc1
proofread and fix the docs, part 1
collerek Mar 10, 2024
6b0245d
proofread and fix the docs for models
collerek Mar 10, 2024
8598de4
proofread and fix the docs for fields
collerek Mar 10, 2024
ef6793c
proofread and fix the docs for relations
collerek Mar 10, 2024
c569870
proofread and fix rest of the docs, add release notes for 0.20
collerek Mar 16, 2024
5700bd6
create tables in new docs src
collerek Mar 16, 2024
6cb0af2
cleanup old deps, uncomment docs publish on tag
collerek Mar 23, 2024
d8db468
fix import reorder
collerek Mar 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .flake8

This file was deleted.

6 changes: 3 additions & 3 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ jobs:
run: |
echo $RELEASE_VERSION
echo ${{ env.RELEASE_VERSION }}
# - name: Deploy
# run: |
# mike deploy --push --update-aliases ${{ env.RELEASE_VERSION }} latest
- name: Deploy
run: |
mike deploy --push --update-aliases ${{ env.RELEASE_VERSION }} latest
44 changes: 44 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: lint

on:
push:
branches-ignore:
- 'gh-pages'
pull_request:
branches: [ master, pydantic_v2 ]

jobs:
lint:
name: "Python ${{ matrix.python-version }}"
runs-on: ubuntu-latest
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'collerek/ormar'
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install Poetry
uses: snok/[email protected]
with:
version: 1.4.2
virtualenvs-create: false

- name: Poetry details
run: |
poetry --version
poetry config --list

- name: Install dependencies
run: poetry install --extras "all" --no-root

- name: Format
run: make fmt

- name: Lint
run: make lint
62 changes: 51 additions & 11 deletions .github/workflows/test-package.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: build
name: test

on:
push:
branches-ignore:
- 'gh-pages'
pull_request:
branches: [ master ]
branches: [ master, pydantic_v2 ]

jobs:
tests:
Expand All @@ -17,7 +17,7 @@ jobs:
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'collerek/ormar'
strategy:
matrix:
python-version: [3.7, 3.8, 3.9, "3.10", 3.11]
python-version: [3.8, 3.9, "3.10", 3.11]
fail-fast: false
services:
mysql:
Expand All @@ -39,35 +39,75 @@ jobs:
POSTGRES_DB: testsuite
ports:
- 5432:5432
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 --name postgres

steps:
- uses: actions/checkout@v3
- name: Checkout
uses: actions/checkout@v3
with:
submodules: false

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies

- name: Install Poetry
uses: snok/[email protected]
with:
version: 1.4.2
virtualenvs-create: false

- name: Poetry details
run: |
python -m pip install poetry==1.4.2
poetry install --extras "all"
env:
POETRY_VIRTUALENVS_CREATE: false
poetry --version
poetry config --list

- name: Install dependencies
run: poetry install --extras "all"

- name: Run mysql
env:
DATABASE_URL: "mysql://username:[email protected]:3306/testsuite"
run: bash scripts/test.sh

- name: Install postgresql-client
run: |
sudo apt-get update
sudo apt-get install --yes postgresql-client

- name: Connect to PostgreSQL with CLI
run: env PGPASSWORD=password psql -h localhost -U username -c 'SELECT VERSION();' testsuite

- name: Show max connections
run: env PGPASSWORD=password psql -h localhost -U username -c 'SHOW max_connections;' testsuite

- name: Alter max connections
run: |

docker exec -i postgres bash << EOF
sed -i -e 's/max_connections = 100/max_connections = 1000/' /var/lib/postgresql/data/postgresql.conf
sed -i -e 's/shared_buffers = 128MB/shared_buffers = 512MB/' /var/lib/postgresql/data/postgresql.conf
EOF
docker restart --time 0 postgres
sleep 5

- name: Show max connections
run: env PGPASSWORD=password psql -h localhost -U username -c 'SHOW max_connections;' testsuite

- name: Run postgres
env:
DATABASE_URL: "postgresql://username:password@localhost:5432/testsuite"
run: bash scripts/test.sh

- name: Run sqlite
env:
DATABASE_URL: "sqlite:///testsuite"
run: bash scripts/test.sh
- run: mypy ormar tests benchmarks

- name: Upload coverage
uses: codecov/[email protected]

- name: Test & publish code coverage
uses: paambaati/[email protected]
if: github.event.pull_request.head.repo.full_name == 'collerek/ormar'
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/test_docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: test_docs

on:
push:
branches-ignore:
- 'gh-pages'
pull_request:
branches: [ master, pydantic_v2 ]

jobs:
tests_docs:
name: "Python ${{ matrix.python-version }}"
runs-on: ubuntu-latest
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'collerek/ormar'
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install dependencies
run: |
python -m pip install poetry==1.4.2
poetry install --extras "all"
env:
POETRY_VIRTUALENVS_CREATE: false
- name: Test docs
run: bash scripts/test_docs.sh
41 changes: 41 additions & 0 deletions .github/workflows/type-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: type_check

on:
push:
branches-ignore:
- 'gh-pages'
pull_request:
branches: [ master, pydantic_v2 ]

jobs:
lint:
name: "Python ${{ matrix.python-version }}"
runs-on: ubuntu-latest
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'collerek/ormar'
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install Poetry
uses: snok/[email protected]
with:
version: 1.4.2
virtualenvs-create: false

- name: Poetry details
run: |
poetry --version
poetry config --list

- name: Install dependencies
run: poetry install --extras "all" --no-root

- name: Type check
run: make type_check
36 changes: 7 additions & 29 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,31 +1,9 @@
repos:
- repo: https://github.com/psf/black
rev: 22.3.0
- repo: local
hooks:
- id: black
exclude: ^(docs_src/|examples/)
- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
exclude: ^(docs_src/|examples/|tests/)
args: [ '--max-line-length=88' ]
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.982
hooks:
- id: mypy
exclude: ^(docs_src/|examples/)
args: [--no-strict-optional, --ignore-missing-imports]
additional_dependencies: [
types-ujson>=0.1.1,
types-PyMySQL>=1.0.2,
types-ipaddress>=1.0.0,
types-enum34>=1.1.0,
types-cryptography>=3.3.5,
types-orjson>=3.6.0,
types-aiofiles>=0.1.9,
types-pkg-resources>=0.1.3,
types-requests>=2.25.9,
types-toml>=0.10.0,
pydantic>=1.8.2
]
- id: pre-commit-local
name: format
entry: make pre-commit
language: python
pass_filenames: false

20 changes: 12 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,22 @@ test_mysql:
test_sqlite:
bash scripts/test.sh -svv

test_docs:
bash scripts/test_docs.sh -svv

test:
pytest
pytest -svv tests/

coverage:
pytest --cov=ormar --cov=tests --cov-fail-under=100 --cov-report=term-missing
pytest --cov=ormar --cov=tests --cov-fail-under=100 --cov-report=term-missing tests

black:
black ormar tests
type_check:
mkdir -p .mypy_cache && poetry run python -m mypy ormar tests --ignore-missing-imports --install-types --non-interactive

lint:
black ormar tests
flake8 ormar
poetry run python -m ruff . --fix

fmt:
poetry run python -m black .

mypy:
mypy ormar tests
pre-commit: fmt lint type_check
Loading
Loading