Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat ppu3 #546

Merged
merged 138 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
bacadb7
starting on auto switching to ppu in calc factors grid
jtwhite79 Sep 5, 2024
dabff2a
initial implementation of ppu calc factors grid
jtwhite79 Sep 5, 2024
c27ad8a
fix in env, starting on new factor file support
jtwhite79 Sep 5, 2024
ec24a41
fix in try_use_ppu
jtwhite79 Sep 6, 2024
04d8740
initial implementatio and testing
jtwhite79 Sep 11, 2024
81dc631
merge
jtwhite79 Sep 12, 2024
18a7b3e
Update environment.yml
jtwhite79 Sep 13, 2024
7a84225
docstring
jtwhite79 Sep 14, 2024
696b128
initial hackery for pp hyper pars, fully untested, just barfed up the…
jtwhite79 Sep 14, 2024
e848920
fix
jtwhite79 Sep 14, 2024
b8d1855
preping tests
jtwhite79 Sep 14, 2024
604e806
converting ppu geostats invest function to test function
jtwhite79 Sep 14, 2024
9f74cd6
Merge branch 'feat_ppu2' of https://github.com/jtwhite79/pyemu into f…
jtwhite79 Sep 14, 2024
5cfea6b
more work on hyper par integration
jtwhite79 Sep 14, 2024
05d507f
fix
jtwhite79 Sep 15, 2024
6bb2771
refactor test to invest
jtwhite79 Sep 15, 2024
ccc8882
fix
jtwhite79 Sep 15, 2024
8f83c7e
more
jtwhite79 Sep 15, 2024
f161d59
starting on apply order and apply function
jtwhite79 Sep 15, 2024
a5c525d
initial apply_order and testing
jtwhite79 Sep 15, 2024
8f43c08
initial apply_order and testing
jtwhite79 Sep 15, 2024
ae2a9d0
working on hyper par test
jtwhite79 Sep 15, 2024
9a474a2
working on hyper par test
jtwhite79 Sep 15, 2024
6867d3e
more work on hyper pars
jtwhite79 Sep 15, 2024
eaabf4a
more tweaks
jtwhite79 Sep 15, 2024
d968de4
fix
jtwhite79 Sep 16, 2024
2d6a441
invest into why tests arent failing
jtwhite79 Sep 21, 2024
7dde798
added flopy to env
jtwhite79 Sep 21, 2024
95acf22
ugh
jtwhite79 Sep 21, 2024
242d5ef
fix for config df arg
jtwhite79 Sep 21, 2024
5684052
should fail
jtwhite79 Sep 21, 2024
e6aff39
starting on thresh pars
jtwhite79 Sep 21, 2024
1b20008
undo should fail
jtwhite79 Sep 23, 2024
e57d03f
undo should fail
jtwhite79 Sep 23, 2024
6772f52
undo should fail
jtwhite79 Sep 23, 2024
2df6380
first pass at ppu build in CI
briochh Oct 16, 2024
e719d01
adding workflow dispatch to CI
briochh Oct 16, 2024
e4b68d1
try build ppu from bash as login
briochh Oct 16, 2024
7765914
something about the env
briochh Oct 16, 2024
77fab28
working-directory issue
briochh Oct 16, 2024
2fe4771
pip install ppu
briochh Oct 16, 2024
d6ef057
notebook version fix
briochh Oct 16, 2024
ce466c7
shell def for pip install
briochh Oct 16, 2024
e52e5cd
expanding to all tests again
briochh Oct 16, 2024
4c05488
Merge branch 'develop' of https://github.com/pypest/pyemu into feat_ppu2
jtwhite79 Oct 16, 2024
7adace9
Merge branch 'feat_ppubuild' of https://github.com/briochh/pyemu into…
jtwhite79 Oct 16, 2024
e7a7627
switch try_use_ppu to default true
jtwhite79 Oct 16, 2024
2d46893
rolled back try_use to false - going to go live in the next pr
jtwhite79 Oct 16, 2024
0c92b34
partial merge with ppu2
jtwhite79 Oct 16, 2024
192456a
first full attempt at ppu2-ppu3 merge
jtwhite79 Oct 16, 2024
08ef83a
freyberg test mod
jtwhite79 Oct 16, 2024
dfa9206
freyberg test mod
jtwhite79 Oct 16, 2024
2d49fab
merge
jtwhite79 Oct 17, 2024
09553a6
Merge remote-tracking branch 'origin/feat_ppu2' into feat_ppu3
jtwhite79 Oct 17, 2024
5e4d13e
more invests for hyper pars and hyper pars with thresh
jtwhite79 Oct 17, 2024
ebef37f
trying try-use true
jtwhite79 Oct 17, 2024
dcb8f17
more work on ppu try_use
jtwhite79 Oct 17, 2024
9d1b2c9
falling down the rat hole...swapped apply thresh to use nan math to h…
jtwhite79 Oct 17, 2024
0dde46f
working on 1-d (unstruct) support in ppu offload
jtwhite79 Oct 17, 2024
4c19318
removed zone array from mf6 freyberg test
jtwhite79 Oct 17, 2024
4cced4d
test with try use = false
jtwhite79 Oct 17, 2024
45addb0
fix for 1-D
jtwhite79 Oct 18, 2024
50212a8
refactor array compare to ignore inactive cells
jtwhite79 Oct 18, 2024
357f2e4
undo test hacking
jtwhite79 Oct 18, 2024
5ece44e
more test playing
jtwhite79 Oct 18, 2024
50fd1b0
undo test hacking
jtwhite79 Oct 18, 2024
dd2659e
reworking thresh process and tols
jtwhite79 Oct 18, 2024
6ca325e
more tinkering with nonstat thresh invest
jtwhite79 Oct 20, 2024
eb43250
tighter solver tols
jtwhite79 Oct 21, 2024
d5236b4
added deprecation logic for pp space and pp zones, more work on faili…
jtwhite79 Oct 21, 2024
21a85c3
added deprecation logic for pp space and pp zones, more work on faili…
jtwhite79 Oct 21, 2024
0a5fbe0
fix in pp_space action
jtwhite79 Oct 21, 2024
f647c81
temp push to test explicit py version on gha
jtwhite79 Oct 21, 2024
dece624
fix env
jtwhite79 Oct 21, 2024
f7d74f7
Update CI to test py3.13
briochh Oct 21, 2024
e5f34cd
fix for py3.13 csv writer change (quotchar!=delim)
briochh Oct 21, 2024
e204ead
Merge branch 'hotfix_py313csv' into feat_ppu3
briochh Oct 21, 2024
c7215b7
same treatment for hfb tpl write
briochh Oct 21, 2024
4498a44
Merge branch 'hotfix_py313csv' into feat_ppu3
briochh Oct 21, 2024
041bfb1
tpl no longer written with random extra space
briochh Oct 22, 2024
6e936da
Merge branch 'hotfix_py313csv' into feat_ppu3
briochh Oct 22, 2024
2fcabf5
hack to test the appearent memory of pp_options between tests
jtwhite79 Oct 22, 2024
9634b32
removed recharge from thresh test
jtwhite79 Oct 22, 2024
ae10f46
merge
jtwhite79 Oct 22, 2024
c8bcd9d
bonehead fix
jtwhite79 Oct 22, 2024
8b6af77
undo
jtwhite79 Oct 22, 2024
9b4fab6
subdir_test path fix
briochh Oct 22, 2024
3235f24
Merge branch 'feat_ppu3' of github.com:jtwhite79/pyemu into feat_ppu3
briochh Oct 22, 2024
6dde2af
default pp_options dict to None
briochh Oct 22, 2024
c34e784
switched illegal char catch to prep_pp_hyperpar() funx
rhugman Oct 23, 2024
9e7e9fb
typo
rhugman Oct 23, 2024
894e1ec
more logical print warning
rhugman Oct 23, 2024
6533801
Merge pull request #4 from briochh/feat_ppu3
jtwhite79 Oct 23, 2024
f49c8e8
Merge branch 'feat_ppu3' of https://github.com/rhugman/pyemu into fea…
jtwhite79 Oct 23, 2024
c0e6126
fix in thresh_par_test - setting seed
jtwhite79 Oct 23, 2024
a8a7228
pp refac WIP
briochh Oct 24, 2024
e55cecd
starting pstfrom add pp pars refactor.
briochh Oct 24, 2024
21b78a9
only check zones if use_pp_zones is true
briochh Oct 24, 2024
523ba93
Merge branch 'feat_ppu3' of github.com:jtwhite79/pyemu into feat_ppu3
briochh Oct 24, 2024
e67e7de
comments
briochh Oct 24, 2024
1aa600a
groupname-file_tag fix
briochh Oct 24, 2024
6959591
Trying to bring pp_options upfront.
briochh Oct 25, 2024
7ff8bb9
Move ppu funcs from pst_from to pp_utils
briochh Oct 25, 2024
4f94279
Merge pull request #6 from briochh/feat_ppu3
jtwhite79 Oct 26, 2024
2f1f318
dialed up a few example notebooks to get us thru until we rework the …
jtwhite79 Nov 5, 2024
d45fc18
Merge branch 'feat_ppu3' of https://github.com/jtwhite79/pyemu into f…
jtwhite79 Nov 5, 2024
8b3f1b0
tweaks to notebooks
jtwhite79 Nov 5, 2024
dd8a839
starting tests
briochh Nov 5, 2024
b84e4a3
Merge branch 'feat_ppu3' of github.com:jtwhite79/pyemu into feat_ppu3
briochh Nov 5, 2024
3072364
hyperpars test
briochh Nov 11, 2024
6967a48
Stripping out try-excepts
briochh Nov 11, 2024
2eff0e9
Merge pull request #7 from briochh/feat_ppu3
jtwhite79 Nov 12, 2024
aecc42f
merge upstream develop
jtwhite79 Nov 12, 2024
2b28549
Merge branch 'feat_ppu3' of https://github.com/jtwhite79/pyemu into f…
jtwhite79 Nov 12, 2024
a421516
investingating windows mem issues on GHA
jtwhite79 Nov 13, 2024
b5a71d0
trying less parallel pytest
jtwhite79 Nov 13, 2024
138766f
ci testing - just windows
briochh Nov 13, 2024
d78cd9f
Merge branch 'develop' of github.com:pypest/pyemu into feat_ppu3
briochh Nov 13, 2024
d5ab594
-just win 3.11
briochh Nov 13, 2024
797a27d
Merge branch 'hotfix_testmem' into feat_ppu3
briochh Nov 13, 2024
0aea0fc
just la_tests
briochh Nov 13, 2024
fd0e174
en and la test
briochh Nov 13, 2024
394c92f
ppu3 all tests, just py3.11
briochh Nov 13, 2024
794a7c4
ppu3 -all windows (again)
briochh Nov 14, 2024
759206b
tmp_path cleanup
briochh Nov 14, 2024
d6e3596
common setup temp in pst_from_tests
briochh Nov 14, 2024
7438af0
some port selection rigour
briochh Nov 14, 2024
d166531
n=1 test
briochh Nov 14, 2024
f36454c
skipping pp_loc test -- back to n=auto
briochh Nov 14, 2024
bf62499
could it be PstFrom chunk_len leading to too much spawning?
briochh Nov 14, 2024
e7551de
more chunk_len tidy -- should track test run times and increase if hi…
briochh Nov 14, 2024
5e57fb3
mac runner back in
briochh Nov 14, 2024
46dc9f9
pst_from_tests tidy
briochh Nov 14, 2024
f859055
Merge remote-tracking branch 'jtw/feat_ppu3' into feat_ppu3
briochh Nov 14, 2024
77adc92
worker root fix for para dsi tests
briochh Nov 14, 2024
b8ef6bf
Auto-merge issues so pre-merging in-file py_func setup
briochh Nov 17, 2024
10fbbab
pypestutils to toml
briochh Nov 17, 2024
7ee96f9
newlining comments in gitgttribs to try avoid warnings
briochh Nov 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 50 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on:
- cron: '0 8 * * *' # run at 8 AM UTC (12 AM PST, 8 PM NZST)
push:
pull_request:
workflow_dispatch:

jobs:
pyemuCI:
Expand All @@ -15,12 +16,12 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest] # , macos-latest]
python-version: [3.8, 3.9, "3.10", "3.11", "3.12"]
python-version: [3.9, "3.10", "3.11", "3.12"]
run-type: [std]
test-path: ["."]
include:
- os: macos-latest
python-version: 3.9
python-version: "3.11"

steps:
- name: Checkout repo
Expand All @@ -43,6 +44,52 @@ jobs:
cache-environment-key: environment-${{ steps.date.outputs.date }}
cache-downloads-key: downloads-${{ steps.date.outputs.date }}

- name: Checkout pypestutils
uses: actions/checkout@v4
with:
repository: pypest/pypestutils
ref: develop
path: pypestutils

# - name: Install MinGW-w64 tools (Windows)
# if: runner.os == 'Windows'
# uses: msys2/setup-msys2@v2
# with:
# msystem: MINGW64
# path-type: inherit
## install: >-
## mingw-w64-x86_64-gcc-fortran
## mingw-w64-x86_64-lapack
## mingw-w64-x86_64-meson
## mingw-w64-x86_64-ninja

- name: Install meson and gfortran (for ppu)
shell: bash -l {0}
run: |
micromamba install meson gfortran

- name: Build pypestutils (Windows)
if: runner.os == 'Windows'
shell: bash -l {0}
working-directory: pypestutils
env:
LDFLAGS: -static-libgcc -static-libgfortran -static-libquadmath -Wl,-Bstatic,--whole-archive -lwinpthread -Wl,--no-whole-archive
run: |
scripts/build_lib.sh

- name: Build pypestutils (non-Windows)
if: runner.os != 'Windows'
shell: bash -l {0}
working-directory: pypestutils
run: |
scripts/build_lib.sh

- name: Install pypestutils
shell: bash -l {0}
working-directory: pypestutils
run: |
pip install -e .

- name: Install pyemu
shell: bash -l {0}
run: |
Expand Down Expand Up @@ -76,7 +123,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Test Notebooks
if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == 3.9 }}
if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == 3.10 }}
shell: bash -l {0}
working-directory: ./examples
run: |
Expand Down
3 changes: 2 additions & 1 deletion autotest/la_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -637,12 +637,13 @@ def ends_run_freyberg_dsi(tmp_path,nst=False,nst_extrap=None,ztz=False,energy=1.
os.remove(filename)
pst = pyemu.Pst(os.path.join(t_d,"dsi.pst"))
pst.control_data.noptmax = -1
pst.pestpp_options["overdue_giveup_fac"] = 1000
pst.write(os.path.join(t_d,"dsi.pst"),version=2)
pyemu.os_utils.run("pestpp-ies dsi.pst",cwd=t_d)

#read in the results
oe = pyemu.ObservationEnsemble.from_csv(pst=pst, filename=os.path.join(t_d,"dsi.0.obs.csv"))
assert oe.shape[0]==50, f"{50-oe.shape} failed runs"
assert oe.shape[0]==50, f"{50-oe.shape[0]} failed runs"
phi_vector = oe.phi_vector.sort_values().values
assert phi_vector[0] != phi_vector[1],phi_vector

Expand Down
Loading
Loading