reliability diagram calculation for ensemble stat

[JiliDong-NOAA]

Hi,

I am trying to plot reliability diagram from MET ensemble stat. As I don't use MetViewer, I am wondering how to extract the observed relative frequency from the ensemble prob *.stat. I see there is a PCT line type. Should I use OY_i/TOTAL (number matched pairs) as the observed frequency for each forecast probability thresholds (THRESH_i) from PCT? Or should it be OY_i/(OY_i+ON_i)?

There is also a PSTD line type which I am not sure if I can get the reliability diagram from or not.

The sample lines are here:

V10.0.0 rrfs_prob NA 370000 20210504_130000 20210504_130000 000000 20210504_130039 20210504_130039 REFC_ENS_FREQ_ge20 dB L0 MergedReflectivityQCComposite dBZ Z500 MRMS CONUS NEAREST 1 ==0.10000 >=20.0 NA NA PCT 1101368 11 0 8245 882545 0.1 6806 96854 0.2 4533 39591 0.3 3911 20725 0.4 4056 11874 0.5 4296 6921 0.6 3279 3760 0.7 1976 1396 0.8 342 217 0.9 38 3 1
V10.0.0 rrfs_prob NA 370000 20210504_130000 20210504_130000 000000 20210504_130039 20210504_130039 REFC_ENS_FREQ_ge20 dB L0 MergedReflectivityQCComposite dBZ Z500 MRMS CONUS NEAREST 1 ==0.10000 >=20.0 NA 0.05 PSTD 1101368 11 0.034032 0.033695 0.034372 0.0048634 0.0053952 0.032874 0.8347 0.032342 0.032003 0.032682 NA NA NA NA 0.016175 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Thanks!

Jili

[bikegeek]

Hi Jili,

I will let a scientist answer your questions about the line types. I wanted to point out that METplotpy has support for the reliability diagram: https://metplotpy.readthedocs.io/en/v1.1.0-beta4/Users_Guide/reliability_diagram.html

So you can hopefully generate a reliability diagram using your MET output without using METviewer.

[CPKalb]

Hi Jili:

When I have plotted them in the past, I used OY_i/(OY_i+ON_i), with the threshold on the x-axis. Is this producing something reasonable?

Thanks,
Christina

[JiliDong-NOAA]

Thanks to both of you!

Christina, it looks more reasonable with OY_i/(OY_i+ON_i). I will mark this as answer.

Thanks to bikegeek for tips on METplotpy too.

Jili