METplus Coordinated 5.1.0 Release Candidate Testing #2218
Replies: 20 comments 27 replies
-
|
As we've discovered LAEA grid support is still subject to a bugfix for netcdf! We hadn't immediately spotted it has only been added for grib2. I guess the release notes weren't specific about this... but digging into the issues provided the info. |
Beta Was this translation helpful? Give feedback.
-
|
There seems to be an issue with Python Embedding, specifically with a test I ran using the MET nightly build with Point-Stat. Or perhaps just a general overhaul of the directions on Python embedding is necessary. Passed in a generic GFS file for forecast, doesn't seem to be an issue. But trying a Python script that processes the observations gives this error: With Python Embedding activated, this shouldn't be a tripping point since the level is irrelevant in the field info. The guide actually provides a Python Embedding example in Point-Stat where there is no level information provided. Example test command I used was So not sure why this level check fail is happening. |
Beta Was this translation helpful? Give feedback.
-
|
I'm sorry, I meant to say that the obs field information should match the obs var names that are found in the python embedding results. |
Beta Was this translation helpful? Give feedback.
-
|
Where is the test_input_data.csv file? |
Beta Was this translation helpful? Give feedback.
-
|
I still am not convinced it's getting past reading the forecast data file. I think the input CSV file for Python embedding is here: |
Beta Was this translation helpful? Give feedback.
-
|
There is a bad character on line 68 of /d1/personal/jopatz/workbench/PointStatConfig_DELETE:
should be
When I fix this and set the obs name to a valid value in the obs_var column and level to Z0, it does not produce an error. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for that find, George. It looks like everything else is working fine, so the test passes. |
Beta Was this translation helpful? Give feedback.
-
|
Testing for MET issue #2508 was successfully completed. See comment for further details. |
Beta Was this translation helpful? Give feedback.
-
|
Testing for MET issue #2531 was successfully completed. See comment for further details. |
Beta Was this translation helpful? Give feedback.
-
|
Initial testing for MET issue #1283 was conducted. See comment for further details. |
Beta Was this translation helpful? Give feedback.
-
|
Initial testing for MET issue #2425 was conducted (unsuccessfully). See comment for further details. |
Beta Was this translation helpful? Give feedback.
-
|
George McCabe identified the issue with MET #2425 and now everything works successfully. Comment for details. |
Beta Was this translation helpful? Give feedback.
-
|
EVS components global_det atmos and wave stats both pass this testing with MET and METplus release candidates installed on WCOSS2. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @malloryprow. I have added this as a row in the Integration Testing section. |
Beta Was this translation helpful? Give feedback.
-
|
The EVSv1 subseasonal component passes this testing with the MET and METplus release candidates that are installed on WCOSS2 (confirmed successful generation of grid2grid and grid2obs stats files). |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @ShannonShields-NOAA. I have added this as a row in the Integration Testing section. |
Beta Was this translation helpful? Give feedback.
-
Release Acceptance Testing Summary: Integration Testing for SWPC-RTVersion: METplus-5.1.0-rc1 |
Beta Was this translation helpful? Give feedback.
-
Release Acceptance Testing Summary: Integration Testing for TCDiagVersion: METplus-5.1.0-rc1 |
Beta Was this translation helpful? Give feedback.
-
|
EVS testing for TC component is successful and passes with the MET and METplus release candidates installed on WCOSS2. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @OliviaOstwald-NOAA. I have added this as a row in the Integration Testing section. |
Beta Was this translation helpful? Give feedback.
-
|
@OliviaOstwald-NOAA Can you please confirm if this case tests MET issue #2542? |
Beta Was this translation helpful? Give feedback.
-
|
and/or MET issue #2532? |
Beta Was this translation helpful? Give feedback.
-
|
Confirming this case tests both #2542 and #2532 and both were successful and pass on dogwood. |
Beta Was this translation helpful? Give feedback.
-
|
@OliviaOstwald-NOAA Thank you so much! |
Beta Was this translation helpful? Give feedback.
-
Release Acceptance Testing SummaryVersion: METplus-5.1.0-rc1 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @MarcelCaron-NOAA. I have added this as a row in the Integration Testing section. |
Beta Was this translation helpful? Give feedback.
-
|
Version: METplus-5.1.0-rc1 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @RoshanShrestha-NOAA. I have added this as a row in the Integration Testing section. |
Beta Was this translation helpful? Give feedback.
-
|
Release Acceptance Testing on dtcenter/MET#2452 (a) Use the combination of module load met/11.0.0 and module load metplus/5.0.1 (b) Use the combination of module load met/11.0.1 and module load metplus/5.0.1 (c) Use the combination of module load met/11.1.0-rc1 and module load metplus/5.1.0-rc2 The results indicate that MET#2452 remains fixed in this new version. |
Beta Was this translation helpful? Give feedback.
-
|
Version: Met/11.1.0-rc1/METplus-5.1.0-rc2 |
Beta Was this translation helpful? Give feedback.
-
|
@GwenChen-NOAA, thank you. Can you please post the command you're running on dogwood so that I can take a look? Please ensure to include full paths of any necessary directories. |
Beta Was this translation helpful? Give feedback.
-
|
The run command is: run_metplus.py -c /lfs/h2/emc/vpppg/noscrub/emc.vpppg/EVS/parm/metplus_config/rtofs/metplus_rtofs.conf Below is the diagnosis from Perry: I traced it back to prep. For VDATE=20230709, prep ran as usual with no errors. This ran with metplus 5.0.1 and met 11.0.2. 184 (dlogin08) /lfs/h2/emc/ptmp/emc.vpppg/output > !ls Both argo and ndbc require METplus to process (the others do not). It actually does not show an error in ndbc, but there is definitely an error in argo. That is not present in the earlier runs. Looking at the log file /lfs/h2/emc/ptmp/emc.vpppg/evs/v1.0/logs/rtofs/metplus.log.20230714101049: DEBUG 1: get_python3_script() -> added python path (/apps/ops/para/libs/intel/19.1.3.304/met/11.1.0-rc1/share/met/python) to PYTHONPATH As opposed to an earlier argo metplus prep job that ran file, such as: /lfs/h2/emc/ptmp/emc.vpppg/evs/v1.0/logs/rtofs/metplus.log.20230712100808, with no errors. There would seem to be a problem in the release candidate here. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Gwen, can you try changing the import on line 11 of read_argo_metplus.py from:
to
|
Beta Was this translation helpful? Give feedback.
-
|
Also, the ARGO use case was added to the METplus repository for rc1. The read_argo_metplus.py script can be found in the METplus release at
The script you are using may be out-of-date without the updates needed to run using MET 11.1.0. You may be able to use this script instead of your local copy. |
Beta Was this translation helpful? Give feedback.
-
|
Checked GridStat with 11.1.0-rc1 and 5.1.0-rc2 release and worked without any issue. Checked output netcdf file and it shows latitude, longitude and station identifiers changing accordingly. |
Beta Was this translation helpful? Give feedback.
-
|
TC-pair/stats for ensemble TC track/intensity error/spread ([dtcenter/MET#2429]) are tested on Hera (met/11.1.0-rc1; metplus/5.1.0-rc1) and Wcoss2(met/11.1.0-rc1; metplus/5.1.0-rc2). All tests are passed without any problem. Thanks to DTC for the great support! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @JiayiPeng-NOAA. I have marked this as passing in the table. |
Beta Was this translation helpful? Give feedback.
-
|
My apologies, I finally got around to testing
dtcenter/MET#2514 <dtcenter/MET#2514>
and the testing failed. The associated PR
dtcenter/MET#2556 contained changes in two files:
- src/basic/vx_util/main.cc
- src/basci/vx_log/concat_string.h
and it appears in both versions 11.1.0-rc1 and 11.1.0-rc2 that the main.cc
change got in and the concat_string.h did not get in. Not sure how this
happened.
Please advise on if there is something I need to do to help get this
corrected.
Thanks for your help with this!
Stephen
…On Thu, Jun 22, 2023 at 3:49 PM John Halley Gotway ***@***.***> wrote:
Listed below are summaries of the new features, enhancements, and bugfixes
included in all development cycles (beta1, beta2, and release candidate 1)
for each software component of the *METplus Coordinated 5.1.0 release*.
These tables summarize the status of external acceptance testing.
Note that status of earlier testing for METplus Coordinated 5.1.0-beta1
Testing <#2039> and METplus
Coordinated 5.1.0-beta2 Testing
<#2135> has been copied
into this discussion.
Please see the METplus 5.1 Existing Builds
<https://dtcenter.org/community-code/metplus/metplus-5-1-existing-builds>
page to see where the release candidate versions are already installed.
Instructions
For each line in these tables, the Tester listed should test the *release
candidate* to confirm the changes work as expected in their environment.
- For *Integration Testing* (not specifically tied to a single GitHub
issue):
- *ON THIS DISCUSSION*: Add a comment to document the testing
outcome *using the template provided below*.
- For *Repository Testing* (new features, enhancements, and bugfixes):
- *ON THE ISSUE ITSELF*: Document their testing with a comment *using
the template provided below*.
- *ON THIS DISCUSSION*: Add a comment linking to their issue
comment created above.
Copy/paste the following template into your issue or discussion comment:
<h3>Release Acceptance Testing Summary</h3>
Version: METplus-5.1.0-rc1
Date:
Location:
Status (PASS/FAIL):
Description:
The Tester is set to NA for any issues that do not require external
acceptance testing.
Integration Testing
Recommended METplus integration tests not specifically tied to a single
GitHub issue.
Status
(OPEN/PASS/FAIL) Description Tester Acceptance Testing Comment Link
OPEN Test compilation of MET-11.1.0-RC1 with Clang
OPEN Test SWPC-RT Use Cases with RC1 @jvigh <https://github.com/jvigh>
OPEN Test METplus TCDiag wrapper with MET TC-Diag tool @jvigh
<https://github.com/jvigh>
OPEN Test METplus MODE wrapper with Multi-Variate MODE enhancements
@hertneky <https://github.com/hertneky> Repository Testing: METplus
Wrappers (Project Board <https://github.com/orgs/dtcenter/projects/66>)
Testing of specific METplus Wrapper issues (new features, enhancements,
and bugfixes).
Status
(OPEN/PASS/FAIL) METplus Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METplus#XXXX
*PASS* dtcenter/METplus#XXXX No external testing required Repository
Testing: MET (Project Board <https://github.com/orgs/dtcenter/projects/64>
)
Testing of specific MET issues (new features, enhancements, and bugfixes).
Status
(OPEN/PASS/FAIL) MET Issue Dev Cycle Tester Acceptance Testing Comment
Link
*FAIL* dtcenter/MET#1693 <dtcenter/MET#1693> rc1
@mpm-meto <https://github.com/mpm-meto> #2218 (comment)
<#2218 (comment)>
OPEN dtcenter/MET#2487 <dtcenter/MET#2487> rc1
@CPKalb <https://github.com/CPKalb>
OPEN dtcenter/MET#2508 <dtcenter/MET#2508> rc1
@michelleharrold <https://github.com/michelleharrold>
OPEN dtcenter/MET#1060 <dtcenter/MET#1060> rc1
@Deba57 <https://github.com/Deba57>
OPEN dtcenter/MET#1283 <dtcenter/MET#1283> rc1
@hertneky <https://github.com/hertneky>
OPEN dtcenter/MET#2545 <dtcenter/MET#2545> rc1
@j-opatz <https://github.com/j-opatz>
OPEN dtcenter/MET#2168 <dtcenter/MET#2168> rc1
@musgrave-kate <https://github.com/musgrave-kate> or @robertdemariacira
<https://github.com/robertdemariacira>
OPEN dtcenter/MET#2425 <dtcenter/MET#2425> rc1
@mollybsmith-noaa <https://github.com/mollybsmith-noaa>
OPEN dtcenter/MET#2532 <dtcenter/MET#2532> rc1
@OliviaOstwald-NOAA <https://github.com/OliviaOstwald-NOAA>
OPEN dtcenter/MET#2523 <dtcenter/MET#2523> rc1
@jvigh <https://github.com/jvigh>
OPEN dtcenter/MET#2552 <dtcenter/MET#2552> rc1
@halperin-erau <https://github.com/halperin-erau>
OPEN dtcenter/MET#2514 <dtcenter/MET#2514> rc1
@srherbener <https://github.com/srherbener>
OPEN dtcenter/MET#2542 <dtcenter/MET#2542> rc1
@OliviaOstwald-NOAA <https://github.com/OliviaOstwald-NOAA>
OPEN dtcenter/MET#2575 <dtcenter/MET#2575> rc1
@CPKalb <https://github.com/CPKalb>
*PASS* dtcenter/MET#2485 <dtcenter/MET#2485>
dtcenter/MET#2540 <dtcenter/MET#2540>
dtcenter/MET#2492 <dtcenter/MET#2492>
dtcenter/MET#2513 <dtcenter/MET#2513>
dtcenter/MET#2521 <dtcenter/MET#2521>
dtcenter/MET#2525 <dtcenter/MET#2525> rc1 NA No
External Testing Required
OPEN dtcenter/MET#2285 <dtcenter/MET#2285> beta2
@j-opatz <https://github.com/j-opatz>
OPEN dtcenter/MET#2482 <dtcenter/MET#2482> beta2
@John-Sharples <https://github.com/John-Sharples> Fix included in 11.0.2
as well
*PASS* dtcenter/MET#2493 <dtcenter/MET#2493>
beta2 @perthsb <https://github.com/perthsb> Fix tested and accepted in
11.0.2 by @perthsb <https://github.com/perthsb>
OPEN dtcenter/MET#2525 <dtcenter/MET#2525> beta2
OPEN dtcenter/MET#2531 <dtcenter/MET#2531> beta2
@jprestop <https://github.com/jprestop>
*PASS* dtcenter/MET#2303 <dtcenter/MET#2303>
dtcenter/MET#2460 <dtcenter/MET#2460>
dtcenter/MET#2490 <dtcenter/MET#2490>
dtcenter/MET#2474 <dtcenter/MET#2474>
dtcenter/MET#2504 <dtcenter/MET#2504>
dtcenter/MET#2506 <dtcenter/MET#2506>
dtcenter/MET#2401 <dtcenter/MET#2401>
dtcenter/MET#2518 <dtcenter/MET#2518> beta2 NA No
External Testing Required
OPEN dtcenter/MET#2429 <dtcenter/MET#2429> beta1
@JiayiPeng-NOAA <https://github.com/JiayiPeng-NOAA>
*PASS* dtcenter/MET#2218 <dtcenter/MET#2218>
beta1 @GwenChen-NOAA <https://github.com/GwenChen-NOAA> #2042 (comment)
<#2042 (comment)>
OPEN dtcenter/MET#2380 <dtcenter/MET#2380> beta1
@robdarvell <https://github.com/robdarvell>
*PASS* dtcenter/MET#2383 <dtcenter/MET#2383>
beta1 @perthsb <https://github.com/perthsb> Partha #2042 (comment)
<#2042 (comment)>,
Julie dtcenter/MET#2383 (comment)
<dtcenter/MET#2383 (comment)>
OPEN dtcenter/MET#2387 <dtcenter/MET#2387> beta1
@j-opatz <https://github.com/j-opatz>
*PASS* dtcenter/MET#2390 <dtcenter/MET#2390>
beta1 @HathewayWill <https://github.com/HathewayWill> #2042 (comment)
<#2042 (comment)>
*PASS* dtcenter/MET#2399 <dtcenter/MET#2399>
beta1 @malloryprow <https://github.com/malloryprow> #2042 (comment)
<#2042 (comment)>
OPEN dtcenter/MET#2408 <dtcenter/MET#2408> beta1
@georgemccabe <https://github.com/georgemccabe>
*PASS* dtcenter/MET#2412 <dtcenter/MET#2412>
beta1 @GwenChen-NOAA <https://github.com/GwenChen-NOAA> #2042 (comment)
<#2042 (comment)>
*PASS* dtcenter/MET#2426 <dtcenter/MET#2426>
beta1 @GwenChen-NOAA <https://github.com/GwenChen-NOAA> #2042 (comment)
<#2042 (comment)>
OPEN dtcenter/MET#2428 <dtcenter/MET#2428> beta1
@junmeiban <https://github.com/junmeiban>
OPEN dtcenter/MET#2437 <dtcenter/MET#2437> beta1
@craigr3 <https://github.com/craigr3>
OPEN dtcenter/MET#2449 <dtcenter/MET#2449> beta1
@j-opatz <https://github.com/j-opatz>
*PASS* dtcenter/MET#2452 <dtcenter/MET#2452>
beta1 @Ho-ChunHuang-NOAA <https://github.com/Ho-ChunHuang-NOAA> #2042
(comment)
<#2042 (comment)>
*PASS* dtcenter/MET#2146 <dtcenter/MET#2146>
dtcenter/MET#2386 <dtcenter/MET#2386>
dtcenter/MET#2389 <dtcenter/MET#2389>
dtcenter/MET#2388 <dtcenter/MET#2388>
dtcenter/MET#2402 <dtcenter/MET#2402>
dtcenter/MET#2415 <dtcenter/MET#2415>
dtcenter/MET#2421 <dtcenter/MET#2421>
dtcenter/MET#2461 <dtcenter/MET#2461>
dtcenter/MET#2469 <dtcenter/MET#2469> beta1 NA No
External Testing Required Repository Testing: METplus Analysis (Project
Board <https://github.com/orgs/dtcenter/projects/68>)
Testing of specific METplus Analysis (METdataio, METcalcpy, METplotpy,
METviewer, and METexpress) issues (new features, enhancements, and
bugfixes).
Status
(OPEN/PASS/FAIL) METdataio Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METdataio#XXXX
*PASS* dtcenter/METdataio#XXXX No external testing required
Status
(OPEN/PASS/FAIL) METcalcpy Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METcalcpy#XXXX
*PASS* dtcenter/METcalcpy#XXXX No external testing required
Status
(OPEN/PASS/FAIL) METplotpy Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METplotpy#XXXX
*PASS* dtcenter/METplotpy#XXXX No external testing required
Status
(OPEN/PASS/FAIL) METviewer Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METviewer#XXXX
*PASS* dtcenter/METviewer#XXXX No external testing required
Status
(OPEN/PASS/FAIL) METexpress Issue Dev Cycle Tester Acceptance Testing
Comment Link
OPEN dtcenter/METexpress#XXXX
*PASS* dtcenter/METexpress#XXXX No external testing required
—
Reply to this email directly, view it on GitHub
<#2218>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AH3RATOD24FXWKJSMLRMDODXMS4XFANCNFSM6AAAAAAZHBBMLM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Stephen Herbener
***@***.***
303-497-8650
Joint Center for Satellite Data Assimilation
[image: Image result for jcsda logo]
|
Beta Was this translation helpful? Give feedback.
-
|
Stephen, thank you very much for letting us know about this! While the change to main.cc did stick, the changes to concat_string.h did not. I've been tracing through the history to understand why. Using just develop branch for simplicity (since main_v11.1 was created from develop on June 18th or so):
Now I don't know WHY the second PR would've changed those 2 lines. I would not have expected that behavior. On June 12th, those 2 lines changed in the develop branch, and I would expect the June 13th merge to not change them back. Perhaps both PR's were open at the same time and that caused confusion about the which lines were considered diffs? Suffice it to say this is an easy fix that I'll make to both the main_v11.1 branch (for inclusion in the v11.1.0 release) and the develop branch (for inclusion in future releases). I'm just more concerned about my assumptions for how diffs are handled in PR's! |
Beta Was this translation helpful? Give feedback.
-
|
No worries, I'm glad we caught this. And thank you for fixing this! We are hoping to get this in our spack-stack-1.5.0 release which is currently targeted for Sep 9, 2023. Is it a reasonable expectation to have an 11.1 release with this fix in time for Sep 9. If not, no problem, we will incorporate the 11.1 release when it becomes available and continue with the patches we are currently using in spack-stack. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Yes, we are planning to release MET-11.1.0 on Monday 7/31/23. |
Beta Was this translation helpful? Give feedback.
-
|
Sounds great! Thanks for letting me know. |
Beta Was this translation helpful? Give feedback.
-
|
Have downloaded rc2 and tested the LAEA grid update which was added. [https://github.com/dtcenter/MET/pull/2616] plot_data_plane and point_stat have been tested with a netcdf LAEA file and they both work (we don't have grib2 flavours of the data to hand). Examining the mpr output with some alternate data for the same time period (for temperature) and the results appear to be in the same range for each location. So at this point, it will be an OK to go with it (as it works). Hopefully when we start hitting it with a lot more data then we don't uncover anything more. |
Beta Was this translation helpful? Give feedback.
-
Listed below are summaries of the new features, enhancements, and bugfixes included in all development cycles (beta1, beta2, and release candidate 1) for each software component of the METplus Coordinated 5.1.0 release. These tables summarize the status of external acceptance testing.
Note that status of earlier testing for METplus Coordinated 5.1.0-beta1 Testing and METplus Coordinated 5.1.0-beta2 Testing has been copied into this discussion.
Please see the METplus 5.1 Existing Builds page to see where the release candidate versions are already installed.
Instructions
For each line in these tables, the
Testerlisted should test the release candidate to confirm the changes work as expected in their environment.Copy/paste the following template into your issue or discussion comment:
The
Testeris set toNAfor any issues that do not require external acceptance testing.Integration Testing
Integration Testing refers to testing functionality not specifically tied to a single GitHub issue. The largest category here is testing existing use cases in your environment (e.g. NOAA/EMC testing their EMC Vx System use cases). This also includes testing new features across multiple METplus components.
Users are encouraged to test their existing use cases with the METplus release candidate and report their results as comments on this discussion (see Instructions above). Please indicate the name of the use case if not already listed in the table below. Comments about expected results (PASS) and unexpected results (FAIL) are welcome. The METplus team will add a row for the use case and summarize those comments in the table below.
(OPEN,
PASS,
FAIL)
Repository Testing: METplus Wrappers (Project Board)
Testing of specific METplus Wrapper issues (new features, enhancements, and bugfixes).
(OPEN,
PASS,
FAIL)
The majority of the changes to METplus wrappers were tested when the changes were made, so testing of these specific issues may not be necessary. However, it would be good to perform general testing of existing METplus use cases to ensure that any changes have not caused unexpected bugs. This should be covered in the "Integrated Testing" section.
Repository Testing: MET (Project Board)
Testing of specific MET issues (new features, enhancements, and bugfixes).
(OPEN,
PASS,
FAIL)
Repository Testing: METplus Analysis (Project Board)
Testing of specific METplus Analysis (METdataio, METcalcpy, METplotpy, METviewer, and METexpress) issues (new features, enhancements, and bugfixes).
(OPEN,
PASS,
FAIL)
(OPEN,
PASS,
FAIL)
(OPEN,
PASS,
FAIL)
(OPEN,
PASS,
FAIL)
(OPEN,
PASS,
FAIL)
Beta Was this translation helpful? Give feedback.
All reactions