[Dispatch] Add pattern to bubble expand through extract 1/2 #19325
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hanhanW
reviewed
Nov 27, 2024
compiler/src/iree/compiler/DispatchCreation/test/bubble_up_expand_shapes.mlir
Outdated
Show resolved
Hide resolved
compiler/src/iree/compiler/DispatchCreation/test/bubble_up_expand_shapes.mlir
Outdated
Show resolved
Hide resolved
compiler/src/iree/compiler/DispatchCreation/BubbleUpExpandShapes.cpp
Outdated
Show resolved
Hide resolved
Signed-off-by: Ian Wood <[email protected]>
hanhanW
approved these changes
Dec 2, 2024
| // TODO: move this upstream with other tensor bubbling patterns. | ||
| /// Bubbles a `tensor.expand_shape` op through a `tensor.extract_slice` op. This | ||
| /// pattern only gets applied when the `extract_slice` doesn't modify dimensions | ||
| /// that are expanded by the `expand_shape`. |
There was a problem hiding this comment.
IIRC, it only works for static shapes? Dynamic shapes could be very tricky, and I think we're not going to support that in the near future. So we want to document that it only works for static shapes.
There was a problem hiding this comment.
Yeah, only static shapes so I updated the comment to reflect that
Signed-off-by: Ian Wood <[email protected]>
Signed-off-by: Ian Wood <[email protected]>
giacs-epic
pushed a commit
to giacs-epic/iree
that referenced
this pull request
Dec 4, 2024
…#19325) This is the 1/2 changes needed to reland iree-org#18857 (with an open PR iree-org#19113). Adds pattern to bubble up expand shape through extract slice. i.e `expand(extract)` to `extract(expand)`. This only supports the case where the expanded dimensions are not modified by the extract slice and there are no dynamic dimensions. This is important because `tensor.expand_shape` ops _cannot be cloned_ while `tensor.extract_slice` ops _can be cloned_. So, if the `expand_shape` gets stuck on the bottom of the `extract_slice` it will block it from being cloned and the `extract_slice` will have to be put into its own dispatch. --------- Signed-off-by: Ian Wood <[email protected]> Signed-off-by: Giacomo Serafini <[email protected]>
IanWood1
added a commit
that referenced
this pull request
Jan 8, 2025
Reland after fixing sdxl int8 regressions via #19012. Running CI revealed further performance regressions that have pending patches: #19325 and #19326. This reverts commit 8d3faf8. --------- Signed-off-by: Ian Wood <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the 1/2 changes needed to reland #18857 (with an open PR #19113).
Adds pattern to bubble up expand shape through extract slice. i.e
expand(extract)toextract(expand). This only supports the case where the expanded dimensions are not modified by the extract slice and there are no dynamic dimensions.This is important because
tensor.expand_shapeops cannot be cloned whiletensor.extract_sliceops can be cloned. So, if theexpand_shapegets stuck on the bottom of theextract_sliceit will block it from being cloned and theextract_slicewill have to be put into its own dispatch.