Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enumerator for Optimal Plans in DMLProgram #2182

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

min-guk
Copy link
Contributor

@min-guk min-guk commented Jan 20, 2025

Summary

An initial version of the recursive function for finding the optimal plan in a DMLProgram has been implemented. This design is based on FederatedPlannerFedAll. This PR addresses a different topic from the previous one (#2175), despite being related, so it has been created separately.

The currently implemented functionalities are as follows:

  • Recursive traversal (enumeration?) of StatementBlock and Statement within the program.
  • Identification of the optimal federated plan (FedPlan) for each Statement (limited to individual statements, not the entire program).

However, the current implementation does not yet support:

  • Integration and identification of the optimal FedPlan and MemoTable at the StatementBlock or program level.

Questions for Discussion

1. Recursive Integration of Optimal FedPlans and MemoTables Across Statements and StatementBlocks

  • Should the same dynamic programming approach be used for integration?
  • Or would simply summing up the minimal plans of each block/statement suffice?

2. MemoTable Sharing

  • Is there a need to share the MemoTable?
  • Are there data/hop dependencies between statements?

3. Prediction of Iteration Counts for Loops (For and While)

How should the iteration counts for loops be determined?

  • If from/to/increment are constants: Calculations can be done at compile time.
  • If they are variables: Use default values at compile time, adjust at runtime, or predict using ML models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

1 participant