Some PySPEDAS unit tests fail in PyHC Environment

[sapols]

Greetings PySPEDAS team,

I'm opening this issue in response to a conversation we had at the last PyHC core developer tag-up.

The PyHC environment is now deployed in Binder: https://binder.opensci.2i2c.cloud/v2/gh/heliophysicsPy/science-platforms-coordination/pyhc

Included in there is a notebook unit-tests.ipynb which runs core package unit tests.

Currently, 90.62% of PySPEDAS's tests pass when I run that notebook. Will you please look at what fails and help me bring that up to 100%?

You can click the link above to launch the environment and run the unit tests yourself. But if the link hasn't been clicked in a while it can take upwards of ten minutes to spin up. This is unfortunate but normal. For convenience I ran the notebook and uploaded it here so you can view the output without waiting.

Please take a look, see what's failing, and let me know 1) if I should update the environment somehow to get 100% of PySPEDAS's tests passing or 2) if you can fix them in a new release.

Thank you!
Shawn

[jameswilburlewis]

The ELFIN test failure looks like a packaging issue -- on Github, we just do a git clone of the pyspedas repo and run out of that, but one of the test files wasn't being included in the package as built and uploaded to pypi. So that should be relatively easy to fix.

The failures in twavpol are going to be a bit harder to track down. There is a LOT of heavy numpy processing going on (ffts, etc), with many stages of intermediate results, before it gets to the variables that are actually being compared in the test. @sapols, can you refresh my memory about the python and numpy versions in the binder environment? There's a note in the source code that numpy >= 1.20.0 is required, which is reflected in both requirements.txt and pyproject.toml, so that shouldn't be a problem, but thought I'd better check just in case. Also, pip doesn't seem to be on my path when I launch a terminal window. Do I need to activate one of the Python environments first?

[sapols]

Glad one failure is easy to fix, sorry the other isn't.
Right now the environment has Python 3.10.14 and numpy 1.26.4. Although, in the next day or two I'm going to update the Python version to 3.11.11.

I'm not sure why pip isn't recognized in the terminal; working on that. But it does work in notebook cells (e.g. you can run !pip list in a cell).

[jameswilburlewis]

@sapols I just released pyspedas v1.7.2, which should fix the ELFIN issue, and give me a few more diagnostics to look at with the twavpol test failures. It should also pull in an updated release of pytplot-mpl-temp, with some bug fixes and documentation updates. Can you try installing the updates in the PYHC environment and let me know when they're ready to test?

[sapols]

Thanks @jameswilburlewis!

Your new v1.7.2 is in the environment now, and I've upgraded the Python version to 3.11.

Unfortunately when I re-ran your tests, 9 ELFIN tests errored. But I actually believe this is probably just bad timing; their server appears to be down at the moment! (E.g. this URL fails to load when I hit it in my browser, so does data.elfin.ucla.edu and their plot summary page)

Fwiw, the short test output was the following:

=========================== short test summary info ============================
FAILED analysis/tests/test_twavpol.py::TwavpolDataValidation::test_elliptict - AssertionError: 
FAILED analysis/tests/test_twavpol.py::TwavpolDataValidation::test_helict - AssertionError: 
FAILED analysis/tests/test_twavpol.py::TwavpolDataValidation::test_powspec - AssertionError: 
FAILED analysis/tests/test_twavpol.py::TwavpolDataValidation::test_waveangle - AssertionError: 
ERROR projects/elfin/tests/test_epd_l1.py::TestELFL1Validation::test_epd_l1_cps - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l1.py::TestELFL1Validation::test_epd_l1_eflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l1.py::TestELFL1Validation::test_epd_l1_nflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l1.py::TestELFL1Validation::test_epd_l1_raw - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l2.py::TestELFL2Validation::test_epd_l2_fs_eflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l2.py::TestELFL2Validation::test_epd_l2_fs_nflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l2.py::TestELFL2Validation::test_epd_l2_hs_eflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_epd_l2.py::TestELFL2Validation::test_epd_l2_hs_nflux - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
ERROR projects/elfin/tests/test_state.py::TestELFStateValidation::test_state - requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='data.elfin.uc...
== 4 failed, 49 passed, 2 skipped, 2 warnings, 9 errors in 550.53s (0:09:10) ===

And you can see the full notebook output here.

Hopefully their server goes back up soon and we can re-test.

[jiashuwu89]

I'll let them know about the server issue and keep you updated. Thanks @sapols

[jiashuwu89]

Elfin server should be back on now.

[sapols]

Thanks @jiashuwu89. I re-ran it and the ELFIN tests pass. Now the only failures are the 4 twavpol ones.

I put the output in a notebook here if you wanna quickly see it.