[WIP] feat: handle partition filters via kernel expressions

[roeap]

Description

This PR denotes another step in adopting delta-kernel-rs. The main initial intent was to update the partition filtering logic to be based on the expression system from kernel.

To make this work, we also needed to update the ownership model of the iterators we return when we iterate over logical filers from the log. This implies some API changes ...

Still iterating and cleaning up, but wanted to open this PR to have some visibility into the work for anyone interested.

Related Issue(s)

Documentation

[github-actions]

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

[codecov]

Codecov Report

Attention: Patch coverage is 70.71713% with 147 lines in your changes missing coverage. Please review.

Project coverage is 72.37%. Comparing base (a639dea) to head (72b1ec1).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/core/src/kernel/snapshot/mod.rs	57.14%	40 Missing and 23 partials ⚠️
crates/core/src/kernel/snapshot/log_data.rs	80.72%	28 Missing and 9 partials ⚠️
crates/core/src/schema/partitions.rs	57.44%	10 Missing and 10 partials ⚠️
python/src/lib.rs	0.00%	18 Missing ⚠️
crates/core/src/table/state.rs	91.17%	0 Missing and 3 partials ⚠️
crates/core/src/table/state_arrow.rs	75.00%	0 Missing and 3 partials ⚠️
crates/core/src/operations/optimize.rs	93.75%	0 Missing and 2 partials ⚠️
crates/core/src/delta_datafusion/logical.rs	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3099      +/-   ##
==========================================
- Coverage   72.54%   72.37%   -0.18%     
==========================================
  Files         128      128              
  Lines       41201    41333     +132     
  Branches    41201    41333     +132     
==========================================
+ Hits        29890    29913      +23     
- Misses       9409     9486      +77     
- Partials     1902     1934      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.