minor: Move file compression #14555
minor: Move file compression #14555
Conversation
|
can I get some help with wasm ci? I have hit a wall there. |
|
cc @alamb |
Looks like that has now been resolved, but there are some other issues that remain |
|
apologies, there were some major fumbles, this should not have taken 11+ commits, |
No worries at all - when we merge to main everything in each PR is squashed into a single commit. |
Thanks. I know but still, it feels off. This is ready for review, let me know if you want some addition or changes. |
There was a problem hiding this comment.
Thanks @logan-keede -- this is looking great. I had a few questions about feature flags but otherwise this is ready to go from my perpective
Thanks again for continuing to push on this
datafusion/core/Cargo.toml
Outdated
| @@ -43,7 +43,7 @@ array_expressions = ["nested_expressions"] | |||
| # Used to enable the avro format | |||
| avro = ["apache-avro", "num-traits", "datafusion-common/avro"] | |||
| backtrace = ["datafusion-common/backtrace"] | |||
| compression = ["xz2", "bzip2", "flate2", "zstd", "async-compression", "tokio-util"] | |||
| compression = ["xz2", "bzip2", "flate2", "zstd"] | |||
There was a problem hiding this comment.
I am surprised this doesn't need to activate the compression feature in datafusion-catalog-listing as well 🤔
datafusion/sqllogictest/Cargo.toml
Outdated
| @@ -43,6 +43,7 @@ chrono = { workspace = true, optional = true } | |||
| clap = { version = "4.5.16", features = ["derive", "env"] } | |||
| datafusion = { workspace = true, default-features = true, features = ["avro"] } | |||
| datafusion-catalog = { workspace = true, default-features = true } | |||
| datafusion-catalog-listing = { workspace = true, features = ["compression"] } | |||
There was a problem hiding this comment.
I don't think this should be necessary -- all the required featrues should be able to be activated via the datafusion crate.
Maybe you can just add compression as a feature abovez
datafusion = { workspace = true, default-features = true, features = ["avro"] }
Which issue does this PR close?
datafusioncrate (datafusion/core) #14444.Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?