Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Codegen][Tuner] Add support for per-sku tuning spec #19762

Merged
merged 9 commits into from
Jan 24, 2025
Merged

[Codegen][Tuner] Add support for per-sku tuning spec #19762

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
07c47a3
[Codegen][Tuner] Add support for per-sku tuning spec
bangtianliu Jan 22, 2025
992cc26
specify the target and backend
bangtianliu Jan 22, 2025
2e7a041
address reviewer comments
bangtianliu Jan 23, 2025
95a46e2
add sku to TargetChipAttr
bangtianliu Jan 23, 2025
d72dd45
update TargetChipAttr
bangtianliu Jan 23, 2025
fec3cd2
format TargetChipAttr
bangtianliu Jan 23, 2025
74cb915
remove redundant check
bangtianliu Jan 23, 2025
e7dd660
fix ci error
bangtianliu Jan 23, 2025
bde2e4d
format the code
bangtianliu Jan 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions compiler/plugins/target/ROCM/builtins/tuning/test/spec_gfx942.mlir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,23 @@
// RUN: --iree-codegen-notify-transform-strategy-application \
// RUN: --verify-diagnostics %s | FileCheck %s

// RUN: iree-opt --split-input-file --iree-gpu-test-target=mi300x \
// RUN: --pass-pipeline="builtin.module(hal.executable(hal.executable.variant(iree-hal-configure-target-executable-variants{target=rocm})))" \
// RUN: --iree-codegen-enable-default-tuning-specs \
// RUN: --iree-codegen-notify-transform-strategy-application \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed when this flag was added, but it should be named --iree-codegen-test and/or be hidden or something. Not for this PR though.

// RUN: --verify-diagnostics %s | FileCheck %s --check-prefix=MI308X

// Check that the default configuration for mmt_2048x1280x5120_f16_f16_f32
// applies to the `linalg.matmul_transpose_b` below.

// CHECK-LABEL: func.func @mmt_2048x1280x5120_f16_f16_f32
// CHECK: linalg.generic
// CHECK-SAME: __tuning_spec_applied__

// MI308X-LABEL: func.func @mmt_2048x1280x5120_f16_f16_f32
// MI308X: linalg.generic
// MI308X-SAME: __tuning_spec_applied__

#pipeline_layout = #hal.pipeline.layout<bindings = [
#hal.pipeline.binding<storage_buffer>,
#hal.pipeline.binding<storage_buffer>,
Expand Down
19 changes: 16 additions & 3 deletions compiler/src/iree/compiler/Codegen/Common/MaterializeTuningSpecsPass.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,12 +125,25 @@ getDefaultTuningSpec(ModuleOp module,

// Try to look up the default tuning spec for this architecture, if any.
StringRef arch = gpuTarget.getArch();
std::optional<std::string> sku = gpuTarget.getSKU();
std::string defaultTuningSpecName =
llvm::formatv("iree_default_tuning_spec_{}.mlir", arch);
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
std::optional<StringRef> defaultTuningSpecSource;
EmbeddedDataDirectory::withGlobal([&](EmbeddedDataDirectory &dir) {
defaultTuningSpecSource = dir.getFile(defaultTuningSpecName);
});
if (sku) {
std::string defaultSKUTuningSpecName =
llvm::formatv("iree_default_tuning_spec_{}.mlir", sku);
EmbeddedDataDirectory::withGlobal([&](EmbeddedDataDirectory &dir) {
defaultTuningSpecSource = dir.getFile(defaultSKUTuningSpecName);
});
}
if (!defaultTuningSpecSource) {
// If SKU-specific spec is not found, fall back to the default
// architecture-based tuning spec.
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
EmbeddedDataDirectory::withGlobal([&](EmbeddedDataDirectory &dir) {
defaultTuningSpecSource = dir.getFile(defaultTuningSpecName);
});
}

if (!defaultTuningSpecSource) {
// Not all architectures are expected to provide default tuning specs, so
// this shouldn't be considered a hard error (but that's up to the caller).
Expand Down
17 changes: 17 additions & 0 deletions compiler/src/iree/compiler/Codegen/Dialect/GPU/IR/IREEGPUAttrs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1094,6 +1094,23 @@ bool TargetAttr::supportsSyncMMAOps() const {
return false;
}

std::optional<std::string> TargetAttr::getSKU() const {
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
StringRef arch = getArch();
if (arch == "gfx942") {
TargetChipAttr chip = getChip();
if (chip) {
if (chip.getWgpCount() == 304) {
return "mi300x";
} else if (chip.getWgpCount() == 228) {
return "mi300a";
} else if (chip.getWgpCount() == 80) {
return "mi308x";
}
}
}
return std::nullopt;
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
}

//===----------------------------------------------------------------------===//
// Lowering Config Attributes
//===----------------------------------------------------------------------===//
Expand Down
2 changes: 2 additions & 0 deletions compiler/src/iree/compiler/Codegen/Dialect/GPU/IR/IREEGPUAttrs.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,8 @@ def IREEGPU_TargetAttr : AttrDef<IREEGPU_Dialect, "Target"> {
bool supportsTF32InputMMAOps() const;
// Returns true if this target supports TensorCore synchronized MMA ops.
bool supportsSyncMMAOps() const;
// Returns the SKU of the target GPU if available.
std::optional<std::string> getSKU() const;
}];
}

Expand Down
2 changes: 2 additions & 0 deletions compiler/src/iree/compiler/Codegen/Utils/GPUUtils.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -954,6 +954,8 @@ IREE::GPU::TargetAttr getCLGPUTarget(MLIRContext *context) {
backend = "cuda";
else if (StringRef(clTestTarget).starts_with("gfx"))
backend = "hip";
else if (StringRef(clTestTarget).starts_with("mi"))
backend = "hip";
bangtianliu marked this conversation as resolved.
Show resolved Hide resolved
else if (StringRef(clTestTarget).starts_with("adreno"))
backend = "vulkan";
else if (StringRef(clTestTarget).starts_with("apple"))
Expand Down
Loading