Method equivalent to query/eval in pandas

[mpecovnik]

We very often use the query and eval methods of pandas dataframes to bring in configurability into the code instead of hard-coding filters. A very general example:

import pandas as pd
import numpy as np

import json

# some json-like config
with open(path_to_config, "r") as f:
    config = json.load(f)

df = pd.DataFrame({"x": np.arange(0, 10, 1), "y": np.arange(10, 20, 1)})

# config holds a key 'configurable_query' which holds a string query
# example "x > 10 and y < 15
string_query = config["configurable_query"]

# This let's us select rows in our ML Pipeline in a configurable manner,
# thus making the Pipeline reusable
queried = df.query(string_query)

# do other stuff with 'queried' dataset based on the specific pipeline and other config keys

Stuff like this is very useful for creating reusable configurable ML Pipelines and I can't really imagine my professional life without it. Is something like this possible in your package?

[markfairbanks]

Two steps to convert this to tidypolars:

• .query() is the pandas way of filtering a data frame. That is done using .filter() in tidypolars.
• Your query string needs to use tidypolars col() syntax so you can use it to create an expression

import tidypolars as tp
from tidypolars import col

df = tp.Tibble(x = range(3), y = range(3))

# Query string contained in config
query_string = "(col('x') > 0) & (col('y') < 2)"

# Run eval to turn this into a tidypolars expression
query_expr = eval(query_string)

# Execute the expression inside a tidypolars method
queried = df.filter(query_expr)

queried
┌─────┬─────┐
│ x   ┆ y   │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 1   │
└─────┴─────┘

[mpecovnik]

Thank you very much for this explanation. Basically there is no blocker here... Will try out the package as soon as I can.