Support for Polars Dataframe

[goelzva]

Hi everyone,

I noticed that torch-frame is using pandas as the base framework for handling dataframes. I think that it would be valuable to replace pandas with polars, since polars is oftentimes 2-5 times faster when performing sql-like calculations. This is especially relevant for use-cases such as relbench, where datasets are up to 20Gb in size. I would actually be happy to take that over, but probably a discussion is needed first.

Happy Holidays!

[akihironitta]

Hi @goelzva, I'm not very familiar with Polars, but it sounds like a great improvement in PyTorch Frame! We are happy to help with this when we have a chance, but feel free to share any findings or plans if you already have :) I'm not sure yet, but we may need some abstraction layer that's agnostic to the two backends (i.e., pandas and polars).