Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce StringView #436

Open
richox opened this issue Apr 7, 2024 · 2 comments
Open

introduce StringView #436

richox opened this issue Apr 7, 2024 · 2 comments
Labels
performance related Potential optimization point stale

Comments

@richox
Copy link
Collaborator

richox commented Apr 7, 2024

Is your feature request related to a problem? Please describe.
arrow-rs is supporting new StringViewArray type (apache/arrow-rs#5374), which is likely to achieve better performance by reducing large memcpys.

Describe the solution you'd like

  1. supports StringView basics like hashing, serialization.
  2. supports StringView in methods inside datafusion-ext-functions.
  3. test/benchmark.

Describe alternatives you've considered

Additional context

@richox richox added the performance related Potential optimization point label Apr 7, 2024
@alamb
Copy link

alamb commented Apr 15, 2024

FWIW I think the main usecases where StringView will Significantly help are:

  1. Low selectivity filters (where most values are not filtered) on string columns
  2. String manipulation functions (like substr) where the output can be calculated by changing the views rather than the underlying string values.

If you are interested and willing to help, getting StringViewArray support into the DataFusion expression evaluator (especially for filtering) would be a huge help

Copy link

github-actions bot commented Jan 8, 2025

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance related Potential optimization point stale
Projects
None yet
Development

No branches or pull requests

2 participants