Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Actions New CI Comparison Method #735

Merged
merged 18 commits into from
May 3, 2024

Conversation

scrasmussen
Copy link
Member

@scrasmussen scrasmussen commented Feb 15, 2024

WARNING: PR HAS CHANGED A LOT, SEE COMMENTS LATER IN PR FOR CURRENT STATUS

TYPE: bug fix

KEYWORDS: CI, Google Drive, Croton Testcase

SOURCE: Soren Rasmussen, NCAR

DESCRIPTION OF CHANGES: Switch to gdown python package to download croton testcase. This new method is easier and better suppported.

TESTS CONDUCTED: Tested new lines in a test Python script and they succesfully download the Croton testcase.

NOTES: I believe the Docker image will need to be changed to make the Python package gdown a requirement so it downloads it. Will add that after submitting this pull request.

@scrasmussen scrasmussen force-pushed the bugfix/gh-actions-untar branch 2 times, most recently from f0b0972 to 0adfbfb Compare April 11, 2024 17:28
…t needed to install NetCDF, there were issues with the Conda installation

- Github actions builds reference and candidate WRF-Hydro versions.
- `make croton-{case}` will download Croton tarball if needed and setup Run directory
- `make Run` will run testcase
@scrasmussen scrasmussen force-pushed the bugfix/gh-actions-untar branch from c43c020 to b95e0ef Compare April 11, 2024 19:04
@scrasmussen scrasmussen marked this pull request as draft April 19, 2024 16:28
@scrasmussen
Copy link
Member Author

scrasmussen commented Apr 24, 2024

During debugging PR has changed a lot from original purpose

DESCRIPTION OF CHANGES:

  • Github Actions no longer use Docker to package dependencies and run tests. It does the installation of dependencies itself with apt-get and pip. It handles the building and running of reference and candidate code, comparison of outputs.
  • Parts no longer used by CI: Docker, wrfhydropy, the following files: test/local/run_tests.py, test/test_*.py. Reduces the runtime by around half
  • tests/compare_output.py has the Python scripts for comparing output. This replaces the nccmp functionality
  • The Croton, NY testcase that is part of WRF-Hydro releases is downloaded from Github. Google Drive is no longer needed and the functionality in Python to get the files off of the drive can be removed
  • CMake adds Makefile targets during the build process to build the code and run with the Croton testcase. Targets follow the $ make croton-gridded, $ make run-croton-gridded and $ make run-croton-gridded-parallel format. This is done for gridded, gridded_no_lakes, nwm, nwm_ana, nwm_long_range, reach, reach_lakes

TESTS CONDUCTED: Current build is comparing reference to reference output since the CI will fail on the candidate code

NOTES: checklist for acceptance and testing of this PR

  • debugging cleanup
  • add back in Run Reference Command Github Action step
  • merge PR
  • create new test PR with small change to produce different output. This will test the reporting capacity of this PR (sometimes the Python scripts fail but it doesn't register that failure in the Github Action step)

@scrasmussen scrasmussen marked this pull request as ready for review April 24, 2024 19:09
- generating nwm_ana and nwm_long_range Makefile targets
- adding more makefile targets
- Bringing over xrcmp to test output of actions. Removing apt-install packages that aren't needed.
@scrasmussen scrasmussen force-pushed the bugfix/gh-actions-untar branch from c0d9bf2 to 0a36750 Compare April 26, 2024 16:57
@scrasmussen
Copy link
Member Author

Reference has been added back in, three github action reports to note

@scrasmussen scrasmussen changed the title Github Actions Download Testcase Fix Github Actions New CI Comparison Method Apr 26, 2024
@rcabell rcabell self-requested a review May 3, 2024 19:07
@scrasmussen scrasmussen merged commit e80c79c into NCAR:main May 3, 2024
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants