[LLVMGPUVectorDistribute] Refactor vector.contract distribute #19631
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
manupak
force-pushed
the
refactor-contract-distribution
branch
from
4379322 to
41bb886
Compare
Groverkss
reviewed
Jan 15, 2025
Comment on lines
891
to
898
| Value isoRankLocalReduced = rewriter.create<vector::ShapeCastOp>( | ||
| loc, partialReducedDistributedType, localContractValue); |
There was a problem hiding this comment.
can we instead do a broadcast + transpose? broadcast + transpose is prefered for unit dimensions because it is easier to unroll / reason about
Comment on lines
918
to
947
| SmallVector<int64_t> subgroupTileLens = | ||
| llvm::to_vector(resLayout.getSubgroupTile()); | ||
| SmallVector<int64_t> batchTileLens = | ||
| llvm::to_vector(resLayout.getBatchTile()); | ||
| SmallVector<int64_t> outerTileLens = | ||
| llvm::to_vector(resLayout.getOuterTile()); | ||
| SmallVector<int64_t> threadTileLens = | ||
| llvm::to_vector(resLayout.getThreadTile()); | ||
| SmallVector<int64_t> elementTileLens = | ||
| llvm::to_vector(resLayout.getElementTile()); | ||
| SmallVector<int64_t> subgroupStrides = | ||
| llvm::to_vector(resLayout.getSubgroupStrides()); | ||
| SmallVector<int64_t> threadStrides = | ||
| llvm::to_vector(resLayout.getThreadStrides()); | ||
| for (int64_t rDim : reducedDims) { | ||
| // thread-local reductions have already been carried out. | ||
| // What is remaining is reductions across threads and | ||
| // subgroups. | ||
| subgroupTileLens.push_back(lhsLayout.getSubgroupTile()[rDim]); | ||
| batchTileLens.push_back(1); | ||
| outerTileLens.push_back(1); | ||
| threadTileLens.push_back(lhsLayout.getThreadTile()[rDim]); | ||
| elementTileLens.push_back(1); | ||
| subgroupStrides.push_back(lhsLayout.getSubgroupStrides()[rDim]); | ||
| threadStrides.push_back(lhsLayout.getThreadStrides()[rDim]); | ||
| } | ||
| reductionLayout = IREE::VectorExt::NestedLayoutAttr::get( | ||
| contractOp.getContext(), subgroupTileLens, batchTileLens, | ||
| outerTileLens, threadTileLens, elementTileLens, subgroupStrides, | ||
| threadStrides); |
There was a problem hiding this comment.
Can we add a "broadcast" function to the layout which takes a dimension and sizes for a single dimension? This would make it easier to write these transformations.
|
@Groverkss spent few cycles here today... I ve simplified the implementation a bit by adding a builder for NestedLayoutAttr that can take an existing layout and append few tiling dims. This API is sufficient to manually infer the partial reduction layout. Then rest of rewrite can be derived from it altogether -- when you have some time PTAL. |
manupak
force-pushed
the
refactor-contract-distribution
branch
from
de22597 to
abaad34
Compare
Groverkss
approved these changes
Jan 21, 2025
manupak
force-pushed
the
refactor-contract-distribution
branch
2 times, most recently
from
0211656 to
4f75304
Compare
Currently, vector.contract distribution is implemented as a standalone distribution closely following vector.multi_reduce. Therefore, we have to duplicate code/effort when we improve either one. This commit changes vector.contract just to distribute the "contract" part of it. Then it creates a new vector.multi_reduce to be re-distributed with partial reduction semantics. Thus, allowing the improvements of vector.multi_reduce to be re-used by vector.contract Signed-off-by: Manupa Karunaratne <manupa.karunaratne@amd.com>
* use this to infer partial reduction layout of vector.contract Signed-off-by: Manupa Karunaratne <manupa.karunaratne@amd.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, vector.contract distribution is implemented as a standalone distribution closely following vector.multi_reduce. Therefore, we have to duplicate code/effort when we improve either one.
This commit changes vector.contract just to distribute the "contract" part of it. Then it creates a new vector.multi_reduce to be re-distributed with partial reduction semantics. Thus, allowing the improvements of vector.multi_reduce to be re-used by vector.contract
closes : #19620