Skip to content

Commit 9bb0eed

Browse files
authored
[VPlan] Assign custom opcodes to recipes not mapping to IR opcodes. (#162267)
We can perform CSE on recipes that do not directly map to Instruction opcodes. One example is VPVectorPointerRecipe. Currently this is handled by supporting them in ::canHandle, but currently that means that we return std::nullopt from getOpcodeOrIntrinsicID() for it. This currently only works, because the only case we return std::nullopt and perform CSE is VPVectorPointerRecipe. But that does not work if we support more such recipes, like VPPredInstPHIRecipe (#162110). To fix this, return a custom opcode from getOpcodeOrIntrinsicID for recipes like VPVectorPointerRecipe, using the VPDefID after all regular instruction opcodes. PR: #162267
1 parent 946238e commit 9bb0eed

File tree

2 files changed

+12
-5
lines changed

2 files changed

+12
-5
lines changed

llvm/lib/Transforms/Vectorize/VPlan.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1064,6 +1064,7 @@ class LLVM_ABI_FOR_TEST VPInstruction : public VPRecipeWithIRFlags,
10641064
ResumeForEpilogue,
10651065
/// Returns the value for vscale.
10661066
VScale,
1067+
OpsEnd = VScale,
10671068
};
10681069

10691070
/// Returns true if this VPInstruction generates scalar values for all lanes.

llvm/lib/Transforms/Vectorize/VPlanTransforms.cpp

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1991,6 +1991,13 @@ struct VPCSEDenseMapInfo : public DenseMapInfo<VPSingleDefRecipe *> {
19911991
.Case<VPWidenIntrinsicRecipe>([](auto *I) {
19921992
return std::make_pair(true, I->getVectorIntrinsicID());
19931993
})
1994+
.Case<VPVectorPointerRecipe>([](auto *I) {
1995+
// For recipes that do not directly map to LLVM IR instructions,
1996+
// assign opcodes after the last VPInstruction opcode (which is also
1997+
// after the last IR Instruction opcode), based on the VPDefID.
1998+
return std::make_pair(false,
1999+
VPInstruction::OpsEnd + 1 + I->getVPDefID());
2000+
})
19942001
.Default([](auto *) { return std::nullopt; });
19952002
}
19962003

@@ -2014,11 +2021,8 @@ struct VPCSEDenseMapInfo : public DenseMapInfo<VPSingleDefRecipe *> {
20142021
static bool canHandle(const VPSingleDefRecipe *Def) {
20152022
// We can extend the list of handled recipes in the future,
20162023
// provided we account for the data embedded in them while checking for
2017-
// equality or hashing. We assign VPVectorEndPointerRecipe the GEP opcode,
2018-
// as it is essentially a GEP with different semantics.
2019-
auto C = isa<VPVectorPointerRecipe>(Def)
2020-
? std::make_pair(false, Instruction::GetElementPtr)
2021-
: getOpcodeOrIntrinsicID(Def);
2024+
// equality or hashing.
2025+
auto C = getOpcodeOrIntrinsicID(Def);
20222026

20232027
// The issue with (Insert|Extract)Value is that the index of the
20242028
// insert/extract is not a proper operand in LLVM IR, and hence also not in
@@ -2057,6 +2061,8 @@ struct VPCSEDenseMapInfo : public DenseMapInfo<VPSingleDefRecipe *> {
20572061
vputils::isSingleScalar(L) != vputils::isSingleScalar(R) ||
20582062
!equal(L->operands(), R->operands()))
20592063
return false;
2064+
assert(getOpcodeOrIntrinsicID(L) && getOpcodeOrIntrinsicID(R) &&
2065+
"must have valid opcode info for both recipes");
20602066
if (auto *LFlags = dyn_cast<VPRecipeWithIRFlags>(L))
20612067
if (LFlags->hasPredicate() &&
20622068
LFlags->getPredicate() !=

0 commit comments

Comments
 (0)