chore: Update to DataFusion 45.0.0 / arrow/parquet 54.0.0

[alamb]

Description

Upgrade to DataFusion 45

WIP / Testing PR

Related Issue(s)

• related to Release DataFusion 45.0.0 apache/datafusion#14008

Documentation

[github-actions]

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

[alamb]

It appears we will also have to update delta kernel with the new arrow first:
#delta_kernel = { version = "=0.6.0", features = ["default-engine"] }

[rtyler]

Yeah, I have this change which I would like to land to also make this easier

[alamb]

Yeah, I have this change which I would like to land to also make this easier

Turns out 5c5c65d makes it failry easy

[codecov]

Codecov Report

Attention: Patch coverage is 37.50000% with 10 lines in your changes missing coverage. Please review.

Project coverage is 71.75%. Comparing base (64cb62d) to head (9aa5a3d).

Files with missing lines	Patch %	Lines
python/src/filesystem.rs	0.00%	5 Missing ⚠️
python/src/lib.rs	0.00%	4 Missing ⚠️
crates/core/src/writer/json.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3175      +/-   ##
==========================================
- Coverage   71.75%   71.75%   -0.01%     
==========================================
  Files         138      138              
  Lines       44362    44372      +10     
  Branches    44362    44372      +10     
==========================================
+ Hits        31833    31837       +4     
- Misses      10498    10499       +1     
- Partials     2031     2036       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[alamb]

I would say the tests all look good to me -- there looks to be some error with docs, but that doesn't look like any reason to avoid releasing DataFusion. Otherwise things are looking 👌

[ion-elgreco]

I would say the tests all look good to me -- there looks to be some error with docs, but that doesn't look like any reason to avoid releasing DataFusion. Otherwise things are looking 👌

Awesome! Thanks @alamb 🙏

[alamb]

I would say the tests all look good to me -- there looks to be some error with docs, but that doesn't look like any reason to avoid releasing DataFusion. Otherwise things are looking 👌

Awesome! Thanks @alamb 🙏

Thank you for your patience. BTW I still hope to get the fix for this issue into DataFusion 45 as well

• Type Coercion fails for List with inner type struct which has large/view types apache/datafusion#14154