Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: Fix LOOKUP JOIN with limit #120411

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

alex-spies
Copy link
Contributor

For queries like

... | LOOKUP JOIN lookup_index ON key | LIMIT 10

the limit cannot be simply pushed past the join - but it can be duplicated past the join. The same currently happens with MV_EXPAND.

This draft solves this by leaving an explicit Limit plan node downstream from the Join, but marks it in a way that prevents being duplicated multiple times (which would cause infinite loops).

The alternative approach would be to follow what we do with MvExpand, and internalize the limit into the Join plan node. This works, too, but then any other optimization rule passes that would do anything with the limit won't apply anymore, because it has become part of the Join node.

We should decide on one approach (internalizing the limit or leaving it in place but marked unavailable for pushdown); then I plan to align the code paths for both Joins and MvExpand accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants