Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Enable covering index acceleration for Iceberg tables #719

Open
dai-chen opened this issue Sep 30, 2024 · 1 comment
Open

[FEATURE] Enable covering index acceleration for Iceberg tables #719

dai-chen opened this issue Sep 30, 2024 · 1 comment
Labels
DataSource:Iceberg enhancement New feature or request

Comments

@dai-chen
Copy link
Collaborator

Is your feature request related to a problem?

Currently, the query rewrite optimization introduced in issue #298 is only available for Spark File data source, but it doesn't support Iceberg tables, leading to missed opportunities for optimization.

What solution would you like?

I would like to enable covering index acceleration for Iceberg tables, similar to the optimization introduced in issue #298. This would involve rewriting queries to leverage the covering index where applicable, allowing more efficient access to the indexed columns without scanning the full table data.

What alternatives have you considered?

N/A

Do you have any additional context?

To enable covering index acceleration for Iceberg tables, we specifically need to implement support for the FlintSparkSourceRelationProvider which is abstraction introduced in PR #325.

Additionally, it is important to verify that the partial index optimization and the spark.flint.optimizer.covering.enabled configuration are functioning as expected for Iceberg tables.

@dblock
Copy link
Member

dblock commented Oct 21, 2024

[Catch All Triage - 1, 2]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DataSource:Iceberg enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants