Warnings

[jgallowa07]

Note to look more closely into the unit test warnings

../../../../mambaforge-pypy3/envs/multidms-dev/lib/python3.12/site-packages/dateutil/tz/tz.py:37
  /home/jared/mambaforge-pypy3/envs/multidms-dev/lib/python3.12/site-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    EPOCH = datetime.datetime.utcfromtimestamp(0)

../../../../mambaforge-pypy3/envs/multidms-dev/lib/python3.12/multiprocessing/popen_fork.py:66: 15 warnings
multidms/data.py: 15 warnings
tests/test_data.py: 210 warnings
  /home/jared/mambaforge-pypy3/envs/multidms-dev/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=99501) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

tests/test_data.py::test_ModelCollection_charts
  /home/jared/MatsenGroup/Projects/multidms/multidms/multidms/model_collection.py:679: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
  The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
  
  For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
  
  
    muts_df_tall.condition.replace(

tests/test_data.py::test_ModelCollection_charts
  /home/jared/MatsenGroup/Projects/multidms/multidms/multidms/model_collection.py:877: DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.
    .apply(sparsity)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

[jgallowa07]

../../../../mambaforge-pypy3/envs/multidms-dev/lib/python3.12/site-packages/dateutil/tz/tz.py:37
  /home/jared/mambaforge-pypy3/envs/multidms-dev/lib/python3.12/site-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    EPOCH = datetime.datetime.utcfromtimestamp(0)

I have not been able to track this down - but it's a 3.12-specific problem so I'm assuming one of my dependencies will fix it in a later release.

../../../../mambaforge-pypy3/envs/multidms-dev/lib/python3.12/multiprocessing/popen_fork.py:66: 15 warnings
multidms/data.py: 15 warnings
tests/test_data.py: 210 warnings
  /home/jared/mambaforge-pypy3/envs/multidms-dev/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=99501) is multi-threaded, use of fork() may lead to deadlocks in the child.
    self.pid = os.fork()

I don't think the fork warnings have to do with multidms.fit_models since we use the "spawn" context. Rather, it may be the case that pandarallel is messing things up since it explicitly sets the context to be "fork". However, I'm not sure I understand how pandarallel forked processes use jax at all. Maybe it's just because jax is loaded in the forked processes? One thing you could look at is replacing pandarallel with swifter.

tests/test_data.py::test_ModelCollection_charts
  /home/jared/MatsenGroup/Projects/multidms/multidms/multidms/model_collection.py:679: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
  The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.
  
  For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.
  
  
    muts_df_tall.condition.replace(

This is fixed! ->

multidms/multidms/model_collection.py

Line 717 in 7f8d675

muts_df_tall = muts_df.melt(

tests/test_data.py::test_ModelCollection_charts
  /home/jared/MatsenGroup/Projects/multidms/multidms/multidms/model_collection.py:877: DeprecationWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.
    .apply(sparsity)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

The code causing the problem:

        sparsity_df = (
            df.drop(columns=to_throw)
            .assign(mut_type=lambda x: x.mutation.apply(mut_type))
            .reset_index()
            .groupby(by=feature_cols)
            .apply(
                sparsity, include_groups=True
            )  # TODO This throws deprecation warning
            .drop(columns=feature_cols + ["mutation"])
            .reset_index(drop=False)
            .melt(id_vars=feature_cols, var_name="mut_param", value_name="sparsity")
        )

could be fixed simply by

        sparsity_df = (
            df.drop(columns=to_throw)
            .assign(mut_type=lambda x: x.mutation.apply(mut_type))
            .reset_index()
            .groupby(by=feature_cols)
            .apply(sparsity, include_groups=True)
            .reset_index(drop=False)
            .melt(id_vars=feature_cols, var_name="mut_param", value_name="sparsity")
        )

but not that this makes the package incompatible with 3.8 (which is okay)

Update: fixed!

[jgallowa07]

Closing as the last outstanding problem with "deadlock" warning has been individually documented in #147