Testsuite fails on date/serial conversion routines

[rolandmas]

Describe the bug

As part as the Debian packaging of rosettasciio, I'm running the full testsuite at build time, but the tests at rsciio/tests/utils/test_utils.py that concern the date_time_tools module fail. This is both with Python 3.12 and Python 3.13.

To Reproduce

Steps to reproduce the behavior: python3.13 -m pytest /build/python-rosettasciio-0.6/rsciio/tests/utils/test_utils.py. All four of the tests related to date_time_tools fail; all of these failures seem related to the datetime that's specified as EST. Note that I'm running as CET (although the chroot might be set to UTC, not sure); I did try running the testsuite with TZ=EST but that didn't seem to help. A comprehensive extract of the build logs is pasted further down in this report.

Expected behavior

No failure :-)

Python environement:

• RosettaSciIO version: 0.6
• Python version: 3.12/3.13
• HyperSpy (not installed)

Additional context

Excerpt from the build log I get (filtered to only run the failing tests, but you see both 3.12 and 3.13):

[...]
adding 'rosettasciio-0.0.0.dist-info/WHEEL'
adding 'rosettasciio-0.0.0.dist-info/top_level.txt'
adding 'rosettasciio-0.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built rosettasciio-0.0.0-cp312-cp312-linux_x86_64.whl
I: pybuild plugin_pyproject:144: Unpacking wheel built for python3.12 with "installer" module
   dh_auto_test -O--buildsystem=pybuild
I: pybuild base:311: cd /build/python-rosettasciio-0.6/.pybuild/cpython3_3.13_rosettasciio/build; python3.13 -m pytest /build/python-rosettasciio-0.6/rsciio/tests/utils/test_utils.py
============================= test session starts ==============================
platform linux -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0
rootdir: /build/python-rosettasciio-0.6
configfile: pyproject.toml
plugins: typeguard-4.4.1, xdist-3.6.1
created: 16/16 workers
16 workers [25 items]

.................FFFF....                                                [100%]
=================================== FAILURES ===================================
________________________ test_serial_date_to_ISO_format ________________________
[gw0] linux -- Python 3.13.0 /usr/bin/python3.13

    def test_serial_date_to_ISO_format():
        iso_1 = dtt.serial_date_to_ISO_format(serial1)
        dt1_local = dt1.astimezone(tz.tzlocal())
        assert iso_1[0] == dt1_local.date().isoformat()
        assert iso_1[1] == dt1_local.time().isoformat()
        assert iso_1[2] == dt1_local.tzname()
    
        iso_2 = dtt.serial_date_to_ISO_format(serial2)
        dt2_local = dt2.astimezone(tz.tzlocal())
        assert iso_2[0] == dt2_local.date().isoformat()
        # The below line will/can fail due to accuracy loss when converting to serial date:
        # We therefore truncate milli/micro seconds
>       assert iso_2[1][:8] == dt2_local.time().isoformat()
E       AssertionError: assert '15:04:48' == '10:04:48'
E         
E         - 10:04:48
E         ?  ^
E         + 15:04:48
E         ?  ^

../../../rsciio/tests/utils/test_utils.py:292: AssertionError
________________________ test_ISO_format_to_serial_date ________________________
[gw0] linux -- Python 3.13.0 /usr/bin/python3.13

    def test_ISO_format_to_serial_date():
        res1 = dtt.ISO_format_to_serial_date(
            dt1.date().isoformat(), dt1.time().isoformat(), timezone=dt1.tzname()
        )
        np.testing.assert_allclose(res1, serial1, atol=1e-5)
        dt = dt2.astimezone(tz.tzlocal())
        res2 = dtt.ISO_format_to_serial_date(
            dt.date().isoformat(), dt.time().isoformat(), timezone=dt.tzname()
        )
>       np.testing.assert_allclose(res2, serial2, atol=1e-5)

../../../rsciio/tests/utils/test_utils.py:312: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (<function assert_allclose.<locals>.compare at 0x7f1eddb97d80>, array(81900.42), array(81900.62833333))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-07, atol=1e-05', 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-05
E           
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 0.20833333
E           Max relative difference: 2.54373303e-06
E            x: array(81900.42)
E            y: array(81900.628333)

/usr/lib/python3.13/contextlib.py:85: AssertionError
_________________________ test_datetime_to_serial_date _________________________
[gw0] linux -- Python 3.13.0 /usr/bin/python3.13

    def test_datetime_to_serial_date():
        np.testing.assert_allclose(dtt.datetime_to_serial_date(dt1), serial1, atol=1e-5)
>       np.testing.assert_allclose(dtt.datetime_to_serial_date(dt2), serial2, atol=1e-5)

../../../rsciio/tests/utils/test_utils.py:321: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (<function assert_allclose.<locals>.compare at 0x7f1eddb97880>, array(81900.42), array(81900.62833333))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-07, atol=1e-05', 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-05
E           
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 0.20833333
E           Max relative difference: 2.54373303e-06
E            x: array(81900.42)
E            y: array(81900.628333)

/usr/lib/python3.13/contextlib.py:85: AssertionError
_______________________ test_get_date_time_from_metadata _______________________
[gw0] linux -- Python 3.13.0 /usr/bin/python3.13

    def test_get_date_time_from_metadata():
        assert dtt.get_date_time_from_metadata(md1) == "2014-12-27T00:00:00+00:00"
        assert (
            dtt.get_date_time_from_metadata(md1, formatting="ISO")
            == "2014-12-27T00:00:00+00:00"
        )
        assert dtt.get_date_time_from_metadata(
            md1, formatting="datetime64"
        ) == np.datetime64("2014-12-27T00:00:00.000000")
        assert dtt.get_date_time_from_metadata(md1, formatting="datetime") == dt1
    
>       assert dtt.get_date_time_from_metadata(md2) == "2124-03-25T10:04:48-05:00"
E       AssertionError: assert '2124-03-25T10:04:48+00:00' == '2124-03-25T10:04:48-05:00'
E         
E         - 2124-03-25T10:04:48-05:00
E         ?                    ^ ^
E         + 2124-03-25T10:04:48+00:00
E         ?                    ^ ^

../../../rsciio/tests/utils/test_utils.py:359: AssertionError
=========================== short test summary info ============================
FAILED ../../../rsciio/tests/utils/test_utils.py::test_serial_date_to_ISO_format
FAILED ../../../rsciio/tests/utils/test_utils.py::test_ISO_format_to_serial_date
FAILED ../../../rsciio/tests/utils/test_utils.py::test_datetime_to_serial_date
FAILED ../../../rsciio/tests/utils/test_utils.py::test_get_date_time_from_metadata
========================= 4 failed, 21 passed in 2.00s =========================
E: pybuild pybuild:389: test: plugin pyproject failed with: exit code=1: cd /build/python-rosettasciio-0.6/.pybuild/cpython3_3.13_rosettasciio/build; python3.13 -m pytest {dir}/rsciio/tests/utils/test_utils.py
I: pybuild base:311: cd /build/python-rosettasciio-0.6/.pybuild/cpython3_3.12_rosettasciio/build; python3.12 -m pytest /build/python-rosettasciio-0.6/rsciio/tests/utils/test_utils.py
============================= test session starts ==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
rootdir: /build/python-rosettasciio-0.6
configfile: pyproject.toml
plugins: typeguard-4.4.1, xdist-3.6.1
created: 16/16 workers
16 workers [25 items]

.................FFFF....                                                [100%]
=================================== FAILURES ===================================
________________________ test_serial_date_to_ISO_format ________________________
[gw0] linux -- Python 3.12.7 /usr/bin/python3.12

    def test_serial_date_to_ISO_format():
        iso_1 = dtt.serial_date_to_ISO_format(serial1)
        dt1_local = dt1.astimezone(tz.tzlocal())
        assert iso_1[0] == dt1_local.date().isoformat()
        assert iso_1[1] == dt1_local.time().isoformat()
        assert iso_1[2] == dt1_local.tzname()
    
        iso_2 = dtt.serial_date_to_ISO_format(serial2)
        dt2_local = dt2.astimezone(tz.tzlocal())
        assert iso_2[0] == dt2_local.date().isoformat()
        # The below line will/can fail due to accuracy loss when converting to serial date:
        # We therefore truncate milli/micro seconds
>       assert iso_2[1][:8] == dt2_local.time().isoformat()
E       AssertionError: assert '15:04:48' == '10:04:48'
E         
E         - 10:04:48
E         ?  ^
E         + 15:04:48
E         ?  ^

../../../rsciio/tests/utils/test_utils.py:292: AssertionError
________________________ test_ISO_format_to_serial_date ________________________
[gw0] linux -- Python 3.12.7 /usr/bin/python3.12

    def test_ISO_format_to_serial_date():
        res1 = dtt.ISO_format_to_serial_date(
            dt1.date().isoformat(), dt1.time().isoformat(), timezone=dt1.tzname()
        )
        np.testing.assert_allclose(res1, serial1, atol=1e-5)
        dt = dt2.astimezone(tz.tzlocal())
        res2 = dtt.ISO_format_to_serial_date(
            dt.date().isoformat(), dt.time().isoformat(), timezone=dt.tzname()
        )
>       np.testing.assert_allclose(res2, serial2, atol=1e-5)

../../../rsciio/tests/utils/test_utils.py:312: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (<function assert_allclose.<locals>.compare at 0x7f7b01402200>, array(81900.42), array(81900.62833333))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-07, atol=1e-05', 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-05
E           
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 0.20833333
E           Max relative difference: 2.54373303e-06
E            x: array(81900.42)
E            y: array(81900.628333)

/usr/lib/python3.12/contextlib.py:81: AssertionError
_________________________ test_datetime_to_serial_date _________________________
[gw0] linux -- Python 3.12.7 /usr/bin/python3.12

    def test_datetime_to_serial_date():
        np.testing.assert_allclose(dtt.datetime_to_serial_date(dt1), serial1, atol=1e-5)
>       np.testing.assert_allclose(dtt.datetime_to_serial_date(dt2), serial2, atol=1e-5)

../../../rsciio/tests/utils/test_utils.py:321: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (<function assert_allclose.<locals>.compare at 0x7f7b01403c40>, array(81900.42), array(81900.62833333))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-07, atol=1e-05', 'verbose': True}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=1e-05
E           
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 0.20833333
E           Max relative difference: 2.54373303e-06
E            x: array(81900.42)
E            y: array(81900.628333)

/usr/lib/python3.12/contextlib.py:81: AssertionError
_______________________ test_get_date_time_from_metadata _______________________
[gw0] linux -- Python 3.12.7 /usr/bin/python3.12

    def test_get_date_time_from_metadata():
        assert dtt.get_date_time_from_metadata(md1) == "2014-12-27T00:00:00+00:00"
        assert (
            dtt.get_date_time_from_metadata(md1, formatting="ISO")
            == "2014-12-27T00:00:00+00:00"
        )
        assert dtt.get_date_time_from_metadata(
            md1, formatting="datetime64"
        ) == np.datetime64("2014-12-27T00:00:00.000000")
        assert dtt.get_date_time_from_metadata(md1, formatting="datetime") == dt1
    
>       assert dtt.get_date_time_from_metadata(md2) == "2124-03-25T10:04:48-05:00"
E       AssertionError: assert '2124-03-25T10:04:48+00:00' == '2124-03-25T10:04:48-05:00'
E         
E         - 2124-03-25T10:04:48-05:00
E         ?                    ^ ^
E         + 2124-03-25T10:04:48+00:00
E         ?                    ^ ^

../../../rsciio/tests/utils/test_utils.py:359: AssertionError
=========================== short test summary info ============================
FAILED ../../../rsciio/tests/utils/test_utils.py::test_serial_date_to_ISO_format
FAILED ../../../rsciio/tests/utils/test_utils.py::test_ISO_format_to_serial_date
FAILED ../../../rsciio/tests/utils/test_utils.py::test_datetime_to_serial_date
FAILED ../../../rsciio/tests/utils/test_utils.py::test_get_date_time_from_metadata
========================= 4 failed, 21 passed in 1.94s =========================
E: pybuild pybuild:389: test: plugin pyproject failed with: exit code=1: cd /build/python-rosettasciio-0.6/.pybuild/cpython3_3.12_rosettasciio/build; python3.12 -m pytest {dir}/rsciio/tests/utils/test_utils.py
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p "3.13 3.12" returned exit code 13
make: *** [debian/rules:8: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
�[0;31mE: Failed autobuilding of package�[0m
�[0mI: user script /var/cache/pbuilder/build/cow.820914/tmp/hooks/C50shell-on-failure starting�[0m
I: Build failed, opening shell
�[0mI: user script /var/cache/pbuilder/build/cow.820914/tmp/hooks/C50shell-on-failure finished�[0m
�[0mI: unmounting /home/blagandzig/debian/dependencies filesystem�[0m
�[0mI: unmounting dev/ptmx filesystem�[0m
�[0mI: unmounting dev/pts filesystem�[0m
�[0mI: unmounting dev/shm filesystem�[0m
�[0mI: unmounting proc filesystem�[0m
�[0mI: unmounting sys filesystem�[0m
I: Cleaning COW directory
I: forking: rm -rf /var/cache/pbuilder/build/cow.820914

[ericpre]

Thank you for the bug report. Not sure how quickly this can be looked at, in the meantime, could you skip these tests using -k argument of pytest?

[rolandmas]

Okay, I'll do that if I have your blessing :-) Feel free to ask for help if you need me to test a potential fix.