Intrinsic: Fix minnum and introduce minimumnum #93373
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
llvmbot added backend:AArch64 backend:RISC-V backend:X86 llvm:globalisel vectorizers llvm:support llvm:SelectionDAG 
Member
| @llvm/pr-subscribers-llvm-ir @llvm/pr-subscribers-backend-aarch64 Author: YunQiang Su (wzssyqa) ChangesCurrently, on different platform, the behaivor of llvm.minnum is different if one operand is sNaN. When we compute sNaN vs NUM ARM/AArch64 follow the IEEE754-2008's minNUM: return qNaN. RV/X86/Hexagon follow the IEEE754-2019's minimumNumber: return NUM. So, let's introduce Since Fixes: #93033 Patch is 117.02 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/93373.diff 60 Files Affected:
diff --git a/llvm/include/llvm/ADT/APFloat.h b/llvm/include/llvm/ADT/APFloat.h index deb74cb2fdeb1..69dcb3d5f2634 100644 --- a/llvm/include/llvm/ADT/APFloat.h +++ b/llvm/include/llvm/ADT/APFloat.h @@ -1422,9 +1422,19 @@ inline APFloat maxnum(const APFloat &A, const APFloat &B) { LLVM_READONLY inline APFloat minimum(const APFloat &A, const APFloat &B) { if (A.isNaN()) - return A; + return A.makeQuiet(); if (B.isNaN()) - return B; + return B.makeQuiet(); + if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative())) + return A.isNegative() ? A : B; + return B < A ? B : A; +} +LLVM_READONLY +inline APFloat minimumnum(const APFloat &A, const APFloat &B) { + if (A.isNaN()) + return B.isNaN() ? B.makeQuiet() : B; + if (B.isNaN()) + return A; if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative())) return A.isNegative() ? A : B; return B < A ? B : A; @@ -1435,9 +1445,19 @@ inline APFloat minimum(const APFloat &A, const APFloat &B) { LLVM_READONLY inline APFloat maximum(const APFloat &A, const APFloat &B) { if (A.isNaN()) - return A; + return A.makeQuiet(); if (B.isNaN()) - return B; + return B.makeQuiet(); + if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative())) + return A.isNegative() ? B : A; + return A < B ? B : A; +} +LLVM_READONLY +inline APFloat maximumnum(const APFloat &A, const APFloat &B) { + if (A.isNaN()) + return B.isNaN() ? B.makeQuiet() : B; + if (B.isNaN()) + return A; if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative())) return A.isNegative() ? B : A; return A < B ? B : A; diff --git a/llvm/include/llvm/Analysis/IVDescriptors.h b/llvm/include/llvm/Analysis/IVDescriptors.h index 5c7b613ac48c4..cba3e568daca4 100644 --- a/llvm/include/llvm/Analysis/IVDescriptors.h +++ b/llvm/include/llvm/Analysis/IVDescriptors.h @@ -48,6 +48,8 @@ enum class RecurKind { FMax, ///< FP max implemented in terms of select(cmp()). FMinimum, ///< FP min with llvm.minimum semantics FMaximum, ///< FP max with llvm.maximum semantics + FMinimumnum, ///< FP min with llvm.minimumnum semantics + FMaximumnum, ///< FP max with llvm.maximumnum semantics FMulAdd, ///< Sum of float products with llvm.fmuladd(a * b + sum). IAnyOf, ///< Any_of reduction with select(icmp(),x,y) where one of (x,y) is ///< loop invariant, and both x and y are integer type. @@ -226,7 +228,8 @@ class RecurrenceDescriptor { /// Returns true if the recurrence kind is a floating-point min/max kind. static bool isFPMinMaxRecurrenceKind(RecurKind Kind) { return Kind == RecurKind::FMin || Kind == RecurKind::FMax || - Kind == RecurKind::FMinimum || Kind == RecurKind::FMaximum; + Kind == RecurKind::FMinimum || Kind == RecurKind::FMaximum || + Kind == RecurKind::FMinimumnum || Kind == RecurKind::FMaximumnum; } /// Returns true if the recurrence kind is any min/max kind. diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.def b/llvm/include/llvm/Analysis/TargetLibraryInfo.def index 717693a7cf63c..d91891a852618 100644 --- a/llvm/include/llvm/Analysis/TargetLibraryInfo.def +++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.def @@ -1314,7 +1314,7 @@ TLI_DEFINE_ENUM_INTERNAL(flsll) TLI_DEFINE_STRING_INTERNAL("flsll") TLI_DEFINE_SIG_INTERNAL(Int, LLong) -// Calls to fmax and fmin library functions expand to the llvm.maxnnum and +// Calls to fmax and fmin library functions expand to the llvm.maxnum and // llvm.minnum intrinsics with the correct parameter types for the arguments // (all types must match). /// double fmax(double x, double y); @@ -1347,6 +1347,72 @@ TLI_DEFINE_ENUM_INTERNAL(fminl) TLI_DEFINE_STRING_INTERNAL("fminl") TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) +// Calls to fmaximum and fminimum library functions expand to the llvm.maximum and +// llvm.minimum intrinsics with the correct parameter types for the arguments +// (all types must match). +/// double fmaximum(double x, double y); +TLI_DEFINE_ENUM_INTERNAL(fmaximum) +TLI_DEFINE_STRING_INTERNAL("fmaximum") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// float fmaximumf(float x, float y); +TLI_DEFINE_ENUM_INTERNAL(fmaximumf) +TLI_DEFINE_STRING_INTERNAL("fmaximumf") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// long double fmaximuml(long double x, long double y); +TLI_DEFINE_ENUM_INTERNAL(fmaximuml) +TLI_DEFINE_STRING_INTERNAL("fmaximuml") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// double fminimum(double x, double y); +TLI_DEFINE_ENUM_INTERNAL(fminimum) +TLI_DEFINE_STRING_INTERNAL("fminimum") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// float fminimumf(float x, float y); +TLI_DEFINE_ENUM_INTERNAL(fminimumf) +TLI_DEFINE_STRING_INTERNAL("fminimumf") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// long double fminimuml(long double x, long double y); +TLI_DEFINE_ENUM_INTERNAL(fminimuml) +TLI_DEFINE_STRING_INTERNAL("fminimuml") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +// Calls to fmaximum_num and fminimum_num library functions expand to the llvm.maximumnum and +// llvm.minimumnum intrinsics with the correct parameter types for the arguments +// (all types must match). +/// double fmaximum_num(double x, double y); +TLI_DEFINE_ENUM_INTERNAL(fmaximum_num) +TLI_DEFINE_STRING_INTERNAL("fmaximum_num") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// float fmaximum_numf(float x, float y); +TLI_DEFINE_ENUM_INTERNAL(fmaximum_numf) +TLI_DEFINE_STRING_INTERNAL("fmaximum_numf") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// long double fmaximum_numl(long double x, long double y); +TLI_DEFINE_ENUM_INTERNAL(fmaximum_numl) +TLI_DEFINE_STRING_INTERNAL("fmaximum_numl") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// double fminimum_num(double x, double y); +TLI_DEFINE_ENUM_INTERNAL(fminimum_num) +TLI_DEFINE_STRING_INTERNAL("fminimum_num") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// float fminimum_numf(float x, float y); +TLI_DEFINE_ENUM_INTERNAL(fminimum_numf) +TLI_DEFINE_STRING_INTERNAL("fminimum_numf") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + +/// long double fminimum_numl(long double x, long double y); +TLI_DEFINE_ENUM_INTERNAL(fminimum_numl) +TLI_DEFINE_STRING_INTERNAL("fminimum_numl") +TLI_DEFINE_SIG_INTERNAL(Floating, Same, Same) + /// double fmod(double x, double y); TLI_DEFINE_ENUM_INTERNAL(fmod) TLI_DEFINE_STRING_INTERNAL("fmod") diff --git a/llvm/include/llvm/Analysis/ValueTracking.h b/llvm/include/llvm/Analysis/ValueTracking.h index 0584b7e29f67b..c8aaa9d75c119 100644 --- a/llvm/include/llvm/Analysis/ValueTracking.h +++ b/llvm/include/llvm/Analysis/ValueTracking.h @@ -1065,6 +1065,10 @@ enum SelectPatternFlavor { SPF_UMAX, /// Unsigned maximum SPF_FMINNUM, /// Floating point minnum SPF_FMAXNUM, /// Floating point maxnum + SPF_FMINIMUM,/// Floating point minnum + SPF_FMAXIMUM,/// Floating point maxnum + SPF_FMINIMUMNUM, /// Floating point minnum + SPF_FMAXIMUMNUM, /// Floating point maxnum SPF_ABS, /// Absolute value SPF_NABS /// Negated absolute value }; diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h index 2091432d4fe27..167cc2d1755a5 100644 --- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h +++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h @@ -1686,6 +1686,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> { case Intrinsic::vector_reduce_fmin: case Intrinsic::vector_reduce_fmaximum: case Intrinsic::vector_reduce_fminimum: + case Intrinsic::vector_reduce_fmaximumnum: + case Intrinsic::vector_reduce_fminimumnum: case Intrinsic::vector_reduce_umax: case Intrinsic::vector_reduce_umin: { IntrinsicCostAttributes Attrs(IID, RetTy, Args[0]->getType(), FMF, I, 1); @@ -2009,6 +2011,12 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> { case Intrinsic::maximum: ISD = ISD::FMAXIMUM; break; + case Intrinsic::minimumnum: + ISD = ISD::FMINIMUMNUM; + break; + case Intrinsic::maximumnum: + ISD = ISD::FMAXIMUMNUM; + break; case Intrinsic::copysign: ISD = ISD::FCOPYSIGN; break; @@ -2090,6 +2098,8 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> { case Intrinsic::vector_reduce_fmin: case Intrinsic::vector_reduce_fmaximum: case Intrinsic::vector_reduce_fminimum: + case Intrinsic::vector_reduce_fmaximumnum: + case Intrinsic::vector_reduce_fminimumnum: return thisT()->getMinMaxReductionCost(getMinMaxReductionIntrinsicOp(IID), VecOpTy, ICA.getFlags(), CostKind); case Intrinsic::abs: { diff --git a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h index 2111e82e1a99d..c6fe31b3cd4fe 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h @@ -947,8 +947,10 @@ class CombinerHelper { /// /// * G_FMAXNUM /// * G_FMAXIMUM + /// * G_FMAXIMUMNUM /// * G_FMINNUM /// * G_FMINIMUM + /// * G_FMINIMUMNUM /// /// Helper function for matchFPSelectToMinMax. unsigned getFPMinMaxOpcForSelect(CmpInst::Predicate Pred, LLT DstTy, diff --git a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h index 2b3efc3b609f0..47535559e77f1 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h @@ -551,6 +551,8 @@ class GVecReduce : public GenericMachineInstr { case TargetOpcode::G_VECREDUCE_FMIN: case TargetOpcode::G_VECREDUCE_FMAXIMUM: case TargetOpcode::G_VECREDUCE_FMINIMUM: + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: case TargetOpcode::G_VECREDUCE_ADD: case TargetOpcode::G_VECREDUCE_MUL: case TargetOpcode::G_VECREDUCE_AND: @@ -589,6 +591,12 @@ class GVecReduce : public GenericMachineInstr { case TargetOpcode::G_VECREDUCE_FMINIMUM: ScalarOpc = TargetOpcode::G_FMINIMUM; break; + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: + ScalarOpc = TargetOpcode::G_FMAXIMUMNUM; + break; + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: + ScalarOpc = TargetOpcode::G_FMINIMUMNUM; + break; case TargetOpcode::G_VECREDUCE_ADD: ScalarOpc = TargetOpcode::G_ADD; break; @@ -669,6 +677,8 @@ class GBinOp : public GenericMachineInstr { case TargetOpcode::G_FMAXNUM_IEEE: case TargetOpcode::G_FMINIMUM: case TargetOpcode::G_FMAXIMUM: + case TargetOpcode::G_FMINIMUMNUM: + case TargetOpcode::G_FMAXIMUMNUM: case TargetOpcode::G_FADD: case TargetOpcode::G_FSUB: case TargetOpcode::G_FMUL: @@ -719,6 +729,8 @@ class GFBinOp : public GBinOp { case TargetOpcode::G_FMAXNUM_IEEE: case TargetOpcode::G_FMINIMUM: case TargetOpcode::G_FMAXIMUM: + case TargetOpcode::G_FMINIMUMNUM: + case TargetOpcode::G_FMAXIMUMNUM: case TargetOpcode::G_FADD: case TargetOpcode::G_FSUB: case TargetOpcode::G_FMUL: diff --git a/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h b/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h index 284f434fbb9b0..8ce6b44c7e1c6 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h @@ -403,6 +403,7 @@ class LegalizerHelper { LegalizeResult lowerMinMax(MachineInstr &MI); LegalizeResult lowerFCopySign(MachineInstr &MI); LegalizeResult lowerFMinNumMaxNum(MachineInstr &MI); + LegalizeResult lowerFMinimumNumMaximumNum(MachineInstr &MI); LegalizeResult lowerFMad(MachineInstr &MI); LegalizeResult lowerIntrinsicRound(MachineInstr &MI); LegalizeResult lowerFFloor(MachineInstr &MI); diff --git a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h index 92e05ee858a75..96cc0f738d258 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h @@ -1723,6 +1723,30 @@ class MachineIRBuilder { return buildInstr(TargetOpcode::G_FMAXNUM_IEEE, {Dst}, {Src0, Src1}, Flags); } + MachineInstrBuilder + buildFMinimum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, + std::optional<unsigned> Flags = std::nullopt) { + return buildInstr(TargetOpcode::G_FMINIMUM, {Dst}, {Src0, Src1}, Flags); + } + + MachineInstrBuilder + buildFMaximum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, + std::optional<unsigned> Flags = std::nullopt) { + return buildInstr(TargetOpcode::G_FMAXIMUM, {Dst}, {Src0, Src1}, Flags); + } + + MachineInstrBuilder + buildFMinimumNUM(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, + std::optional<unsigned> Flags = std::nullopt) { + return buildInstr(TargetOpcode::G_FMINIMUMNUM, {Dst}, {Src0, Src1}, Flags); + } + + MachineInstrBuilder + buildFMaximumNUM(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, + std::optional<unsigned> Flags = std::nullopt) { + return buildInstr(TargetOpcode::G_FMAXIMUMNUM, {Dst}, {Src0, Src1}, Flags); + } + MachineInstrBuilder buildShl(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, std::optional<unsigned> Flags = std::nullopt) { @@ -2074,6 +2098,18 @@ class MachineIRBuilder { return buildInstr(TargetOpcode::G_VECREDUCE_FMINIMUM, {Dst}, {Src}); } + /// Build and insert \p Res = G_VECREDUCE_FMAXIMUMNUM \p Src + MachineInstrBuilder buildVecReduceFMaximumnum(const DstOp &Dst, + const SrcOp &Src) { + return buildInstr(TargetOpcode::G_VECREDUCE_FMAXIMUMNUM, {Dst}, {Src}); + } + + /// Build and insert \p Res = G_VECREDUCE_FMINIMUMNUM \p Src + MachineInstrBuilder buildVecReduceFMinimumnum(const DstOp &Dst, + const SrcOp &Src) { + return buildInstr(TargetOpcode::G_VECREDUCE_FMINIMUMNUM, {Dst}, {Src}); + } + /// Build and insert \p Res = G_VECREDUCE_ADD \p Src MachineInstrBuilder buildVecReduceAdd(const DstOp &Dst, const SrcOp &Src) { return buildInstr(TargetOpcode::G_VECREDUCE_ADD, {Dst}, {Src}); diff --git a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h index 70421a518ab72..c26f8ffd839e7 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h @@ -60,6 +60,8 @@ class APFloat; case TargetOpcode::G_VECREDUCE_FMIN: \ case TargetOpcode::G_VECREDUCE_FMAXIMUM: \ case TargetOpcode::G_VECREDUCE_FMINIMUM: \ + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ case TargetOpcode::G_VECREDUCE_ADD: \ case TargetOpcode::G_VECREDUCE_MUL: \ case TargetOpcode::G_VECREDUCE_AND: \ @@ -77,6 +79,8 @@ class APFloat; case TargetOpcode::G_VECREDUCE_FMIN: \ case TargetOpcode::G_VECREDUCE_FMAXIMUM: \ case TargetOpcode::G_VECREDUCE_FMINIMUM: \ + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ case TargetOpcode::G_VECREDUCE_ADD: \ case TargetOpcode::G_VECREDUCE_MUL: \ case TargetOpcode::G_VECREDUCE_AND: \ diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h index d8af97957e48e..910728e38c1b1 100644 --- a/llvm/include/llvm/CodeGen/ISDOpcodes.h +++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h @@ -435,6 +435,8 @@ enum NodeType { STRICT_LLRINT, STRICT_FMAXIMUM, STRICT_FMINIMUM, + STRICT_FMAXIMUMNUM, + STRICT_FMINIMUMNUM, /// STRICT_FP_TO_[US]INT - Convert a floating point value to a signed or /// unsigned integer. These have the same semantics as fptosi and fptoui @@ -991,6 +993,12 @@ enum NodeType { FMINIMUM, FMAXIMUM, + /// FMINIMUMNUM/FMAXIMUMNUM - minimum/maximum that also treat -0.0 + /// as less than 0.0. While FMINNUM_IEEE/FMAXNUM_IEEE follow IEEE 754-2008 + /// semantics, FMINIMUM/FMAXIMUM follow IEEE 754-2019 semantics. + FMINIMUMNUM, + FMAXIMUMNUM, + /// FSINCOS - Compute both fsin and fcos as a single operation. FSINCOS, @@ -1365,6 +1373,10 @@ enum NodeType { /// llvm.minimum and llvm.maximum semantics. VECREDUCE_FMAXIMUM, VECREDUCE_FMINIMUM, + /// FMINIMUMNUM/FMAXIMUMNUM nodes doesn't propatate NaNs and signed zeroes using the + /// llvm.minimumnum and llvm.maximumnum semantics. + VECREDUCE_FMAXIMUMNUM, + VECREDUCE_FMINIMUMNUM, /// Integer reductions may have a result type larger than the vector element /// type. However, the reduction is performed using the vector element type /// and the value in the top bits is unspecified. diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h index 50a8c7eb75af5..c92eb86c08a48 100644 --- a/llvm/include/llvm/CodeGen/TargetLowering.h +++ b/llvm/include/llvm/CodeGen/TargetLowering.h @@ -2908,6 +2908,8 @@ class TargetLoweringBase { case ISD::FMAXNUM_IEEE: case ISD::FMINIMUM: case ISD::FMAXIMUM: + case ISD::FMINIMUMNUM: + case ISD::FMAXIMUMNUM: case ISD::AVGFLOORS: case ISD::AVGFLOORU: case ISD::AVGCEILS: @@ -5116,6 +5118,8 @@ class TargetLowering : public TargetLoweringBase { /// through to the default expansion/soften to libcall, we might introduce a /// link-time dependency on libm into a file that originally did not have one. SDValue createSelectForFMINNUM_FMAXNUM(SDNode *Node, SelectionDAG &DAG) const; + SDValue createSelectForFMINIMUM_FMAXIMUM(SDNode *Node, SelectionDAG &DAG) const; + SDValue createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, SelectionDAG &DAG) const; /// Return a reciprocal estimate value for the input operand. /// \p Enabled is a ReciprocalEstimate enum with value either 'Unspecified' or @@ -5247,6 +5251,9 @@ class TargetLowering : public TargetLoweringBase { /// Expand fminimum/fmaximum into multiple comparison with selects. SDValue expandFMINIMUM_FMAXIMUM(SDNode *N, SelectionDAG &DAG) const; + /// Expand fminimumnum/fmaximumnum into multiple comparison with selects. + SDValue expandFMINIMUMNUM_FMAXIMUMNUM(SDNode *N, SelectionDAG &DAG) const; + /// Expand FP_TO_[US]INT_SAT into FP_TO_[US]INT and selects or min/max. /// \param N Node to expand /// \returns The expansion result diff --git a/llvm/include/llvm/IR/ConstrainedOps.def b/llvm/include/llvm/IR/ConstrainedOps.def index 41aa44de957f9..361a050cd2355 100644 --- a/llvm/include/llvm/IR/ConstrainedOps.def +++ b/llvm/include/llvm/IR/ConstrainedOps.def @@ -86,6 +86,8 @@ DAG_FUNCTION(maxnum, 2, 0, experimental_constrained_maxnum, FMAXNUM DAG_FUNCTION(minnum, 2, 0, experimental_constrained_minnum, FMINNUM) DAG_FUNCTION(maximum, 2, 0, experimental_constrained_maximum, FMAXIMUM) DAG_FUNCTION(minimum, 2, 0, experimental_constrained_minimum, FMINIMUM) +DAG_FUNCTION(maximumnum, 2, 0, experimental_constrained_maximumnum, FMAXIMUMNUM) +DAG_FUNCTION(minimumnum, 2, 0, experimental_constrained_minimumnum, FMINIMUMNUM) DAG_FUNCTION(nearbyint, 1, 1, experimental_constrained_nearbyint, FNEARBYINT) DAG_FUNCTION(pow, 2, 1, experimental_constrained_pow, FPOW) DAG_FUNCTION(powi, 2, 1, experimental_constrained_powi, FPOWI) diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h index 40a9cf507248a..34394c6478d16 100644 --- a/llvm/include/llvm/IR/IRBuilder.h +++ b/llvm/include/llvm/IR/IRBuilder.h @@ -1018,6 +1018,16 @@ class IRBuilderBase { ... [truncated] |
You can test this locally with the following command:git-clang-format --diff 0bf181eb34bbac1a65522dbbef42c81a705d3102 13ff1371e22f45d52529e62a977b44d2f605fab6 -- llvm/include/llvm/ADT/APFloat.h llvm/include/llvm/Analysis/IVDescriptors.h llvm/include/llvm/Analysis/ValueTracking.h llvm/include/llvm/CodeGen/BasicTTIImpl.h llvm/include/llvm/CodeGen/GlobalISel/CombinerHelper.h llvm/include/llvm/CodeGen/GlobalISel/GenericMachineInstrs.h llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h llvm/include/llvm/CodeGen/GlobalISel/Utils.h llvm/include/llvm/CodeGen/ISDOpcodes.h llvm/include/llvm/CodeGen/TargetLowering.h llvm/include/llvm/IR/IRBuilder.h llvm/include/llvm/IR/IntrinsicInst.h llvm/lib/Analysis/ConstantFolding.cpp llvm/lib/Analysis/IVDescriptors.cpp llvm/lib/Analysis/InstructionSimplify.cpp llvm/lib/Analysis/ValueTracking.cpp llvm/lib/Analysis/VectorUtils.cpp llvm/lib/CodeGen/ExpandVectorPredication.cpp llvm/lib/CodeGen/GlobalISel/CSEMIRBuilder.cpp llvm/lib/CodeGen/GlobalISel/CombinerHelper.cpp llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp llvm/lib/CodeGen/GlobalISel/Utils.cpp llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp llvm/lib/CodeGen/TargetLoweringBase.cpp llvm/lib/Target/AArch64/AArch64ISelLowering.cpp llvm/lib/Target/Hexagon/HexagonISelLowering.cpp llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp llvm/lib/Target/RISCV/GISel/RISCVLegalizerInfo.cpp llvm/lib/Target/RISCV/GISel/RISCVRegisterBankInfo.cpp llvm/lib/Target/RISCV/RISCVISelLowering.cpp llvm/lib/Target/X86/X86ISelLowering.cpp llvm/lib/Target/X86/X86TargetTransformInfo.cpp llvm/lib/Transforms/Vectorize/SLPVectorizer.cppView the diff from clang-format here.diff --git a/llvm/include/llvm/Analysis/IVDescriptors.h b/llvm/include/llvm/Analysis/IVDescriptors.h index cba3e568da..107c9625f4 100644 --- a/llvm/include/llvm/Analysis/IVDescriptors.h +++ b/llvm/include/llvm/Analysis/IVDescriptors.h @@ -32,29 +32,29 @@ class StoreInst; /// These are the kinds of recurrences that we support. enum class RecurKind { - None, ///< Not a recurrence. - Add, ///< Sum of integers. - Mul, ///< Product of integers. - Or, ///< Bitwise or logical OR of integers. - And, ///< Bitwise or logical AND of integers. - Xor, ///< Bitwise or logical XOR of integers. - SMin, ///< Signed integer min implemented in terms of select(cmp()). - SMax, ///< Signed integer max implemented in terms of select(cmp()). - UMin, ///< Unsigned integer min implemented in terms of select(cmp()). - UMax, ///< Unsigned integer max implemented in terms of select(cmp()). - FAdd, ///< Sum of floats. - FMul, ///< Product of floats. - FMin, ///< FP min implemented in terms of select(cmp()). - FMax, ///< FP max implemented in terms of select(cmp()). - FMinimum, ///< FP min with llvm.minimum semantics - FMaximum, ///< FP max with llvm.maximum semantics + None, ///< Not a recurrence. + Add, ///< Sum of integers. + Mul, ///< Product of integers. + Or, ///< Bitwise or logical OR of integers. + And, ///< Bitwise or logical AND of integers. + Xor, ///< Bitwise or logical XOR of integers. + SMin, ///< Signed integer min implemented in terms of select(cmp()). + SMax, ///< Signed integer max implemented in terms of select(cmp()). + UMin, ///< Unsigned integer min implemented in terms of select(cmp()). + UMax, ///< Unsigned integer max implemented in terms of select(cmp()). + FAdd, ///< Sum of floats. + FMul, ///< Product of floats. + FMin, ///< FP min implemented in terms of select(cmp()). + FMax, ///< FP max implemented in terms of select(cmp()). + FMinimum, ///< FP min with llvm.minimum semantics + FMaximum, ///< FP max with llvm.maximum semantics FMinimumnum, ///< FP min with llvm.minimumnum semantics FMaximumnum, ///< FP max with llvm.maximumnum semantics - FMulAdd, ///< Sum of float products with llvm.fmuladd(a * b + sum). - IAnyOf, ///< Any_of reduction with select(icmp(),x,y) where one of (x,y) is - ///< loop invariant, and both x and y are integer type. - FAnyOf ///< Any_of reduction with select(fcmp(),x,y) where one of (x,y) is - ///< loop invariant, and both x and y are integer type. + FMulAdd, ///< Sum of float products with llvm.fmuladd(a * b + sum). + IAnyOf, ///< Any_of reduction with select(icmp(),x,y) where one of (x,y) is + ///< loop invariant, and both x and y are integer type. + FAnyOf ///< Any_of reduction with select(fcmp(),x,y) where one of (x,y) is + ///< loop invariant, and both x and y are integer type. // TODO: Any_of reduction need not be restricted to integer type only. }; diff --git a/llvm/include/llvm/Analysis/ValueTracking.h b/llvm/include/llvm/Analysis/ValueTracking.h index c8aaa9d75c..8e6239e8e0 100644 --- a/llvm/include/llvm/Analysis/ValueTracking.h +++ b/llvm/include/llvm/Analysis/ValueTracking.h @@ -1059,18 +1059,18 @@ bool mustExecuteUBIfPoisonOnPathTo(Instruction *Root, /// Specific patterns of select instructions we can match. enum SelectPatternFlavor { SPF_UNKNOWN = 0, - SPF_SMIN, /// Signed minimum - SPF_UMIN, /// Unsigned minimum - SPF_SMAX, /// Signed maximum - SPF_UMAX, /// Unsigned maximum - SPF_FMINNUM, /// Floating point minnum - SPF_FMAXNUM, /// Floating point maxnum - SPF_FMINIMUM,/// Floating point minnum - SPF_FMAXIMUM,/// Floating point maxnum + SPF_SMIN, /// Signed minimum + SPF_UMIN, /// Unsigned minimum + SPF_SMAX, /// Signed maximum + SPF_UMAX, /// Unsigned maximum + SPF_FMINNUM, /// Floating point minnum + SPF_FMAXNUM, /// Floating point maxnum + SPF_FMINIMUM, /// Floating point minnum + SPF_FMAXIMUM, /// Floating point maxnum SPF_FMINIMUMNUM, /// Floating point minnum SPF_FMAXIMUMNUM, /// Floating point maxnum - SPF_ABS, /// Absolute value - SPF_NABS /// Negated absolute value + SPF_ABS, /// Absolute value + SPF_NABS /// Negated absolute value }; /// Behavior when a floating point min/max is given one NaN and one diff --git a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h index 96cc0f738d..fd3aafe620 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h @@ -1725,13 +1725,13 @@ public: MachineInstrBuilder buildFMinimum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, - std::optional<unsigned> Flags = std::nullopt) { + std::optional<unsigned> Flags = std::nullopt) { return buildInstr(TargetOpcode::G_FMINIMUM, {Dst}, {Src0, Src1}, Flags); } MachineInstrBuilder buildFMaximum(const DstOp &Dst, const SrcOp &Src0, const SrcOp &Src1, - std::optional<unsigned> Flags = std::nullopt) { + std::optional<unsigned> Flags = std::nullopt) { return buildInstr(TargetOpcode::G_FMAXIMUM, {Dst}, {Src0, Src1}, Flags); } @@ -2100,13 +2100,13 @@ public: /// Build and insert \p Res = G_VECREDUCE_FMAXIMUMNUM \p Src MachineInstrBuilder buildVecReduceFMaximumnum(const DstOp &Dst, - const SrcOp &Src) { + const SrcOp &Src) { return buildInstr(TargetOpcode::G_VECREDUCE_FMAXIMUMNUM, {Dst}, {Src}); } /// Build and insert \p Res = G_VECREDUCE_FMINIMUMNUM \p Src MachineInstrBuilder buildVecReduceFMinimumnum(const DstOp &Dst, - const SrcOp &Src) { + const SrcOp &Src) { return buildInstr(TargetOpcode::G_VECREDUCE_FMINIMUMNUM, {Dst}, {Src}); } diff --git a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h index c26f8ffd83..b762944396 100644 --- a/llvm/include/llvm/CodeGen/GlobalISel/Utils.h +++ b/llvm/include/llvm/CodeGen/GlobalISel/Utils.h @@ -60,8 +60,8 @@ class APFloat; case TargetOpcode::G_VECREDUCE_FMIN: \ case TargetOpcode::G_VECREDUCE_FMAXIMUM: \ case TargetOpcode::G_VECREDUCE_FMINIMUM: \ - case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ - case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ case TargetOpcode::G_VECREDUCE_ADD: \ case TargetOpcode::G_VECREDUCE_MUL: \ case TargetOpcode::G_VECREDUCE_AND: \ @@ -79,8 +79,8 @@ class APFloat; case TargetOpcode::G_VECREDUCE_FMIN: \ case TargetOpcode::G_VECREDUCE_FMAXIMUM: \ case TargetOpcode::G_VECREDUCE_FMINIMUM: \ - case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ - case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMAXIMUMNUM: \ + case TargetOpcode::G_VECREDUCE_FMINIMUMNUM: \ case TargetOpcode::G_VECREDUCE_ADD: \ case TargetOpcode::G_VECREDUCE_MUL: \ case TargetOpcode::G_VECREDUCE_AND: \ diff --git a/llvm/include/llvm/CodeGen/ISDOpcodes.h b/llvm/include/llvm/CodeGen/ISDOpcodes.h index 910728e38c..140c3e83fb 100644 --- a/llvm/include/llvm/CodeGen/ISDOpcodes.h +++ b/llvm/include/llvm/CodeGen/ISDOpcodes.h @@ -1373,7 +1373,8 @@ enum NodeType { /// llvm.minimum and llvm.maximum semantics. VECREDUCE_FMAXIMUM, VECREDUCE_FMINIMUM, - /// FMINIMUMNUM/FMAXIMUMNUM nodes doesn't propatate NaNs and signed zeroes using the + /// FMINIMUMNUM/FMAXIMUMNUM nodes doesn't propatate NaNs and signed zeroes + /// using the /// llvm.minimumnum and llvm.maximumnum semantics. VECREDUCE_FMAXIMUMNUM, VECREDUCE_FMINIMUMNUM, diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h index c92eb86c08..5008ed1388 100644 --- a/llvm/include/llvm/CodeGen/TargetLowering.h +++ b/llvm/include/llvm/CodeGen/TargetLowering.h @@ -5118,8 +5118,10 @@ public: /// through to the default expansion/soften to libcall, we might introduce a /// link-time dependency on libm into a file that originally did not have one. SDValue createSelectForFMINNUM_FMAXNUM(SDNode *Node, SelectionDAG &DAG) const; - SDValue createSelectForFMINIMUM_FMAXIMUM(SDNode *Node, SelectionDAG &DAG) const; - SDValue createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, SelectionDAG &DAG) const; + SDValue createSelectForFMINIMUM_FMAXIMUM(SDNode *Node, + SelectionDAG &DAG) const; + SDValue createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, + SelectionDAG &DAG) const; /// Return a reciprocal estimate value for the input operand. /// \p Enabled is a ReciprocalEstimate enum with value either 'Unspecified' or diff --git a/llvm/include/llvm/IR/IRBuilder.h b/llvm/include/llvm/IR/IRBuilder.h index 34394c6478..54768f0b06 100644 --- a/llvm/include/llvm/IR/IRBuilder.h +++ b/llvm/include/llvm/IR/IRBuilder.h @@ -1020,12 +1020,14 @@ public: /// Create call to the minimumnum intrinsic. Value *CreateMinimumNum(Value *LHS, Value *RHS, const Twine &Name = "") { - return CreateBinaryIntrinsic(Intrinsic::minimumnum, LHS, RHS, nullptr, Name); + return CreateBinaryIntrinsic(Intrinsic::minimumnum, LHS, RHS, nullptr, + Name); } /// Create call to the maximum intrinsic. Value *CreateMaximumNum(Value *LHS, Value *RHS, const Twine &Name = "") { - return CreateBinaryIntrinsic(Intrinsic::maximumnum, LHS, RHS, nullptr, Name); + return CreateBinaryIntrinsic(Intrinsic::maximumnum, LHS, RHS, nullptr, + Name); } /// Create call to the copysign intrinsic. diff --git a/llvm/lib/Analysis/InstructionSimplify.cpp b/llvm/lib/Analysis/InstructionSimplify.cpp index 8644e8fc5b..f5d50876e0 100644 --- a/llvm/lib/Analysis/InstructionSimplify.cpp +++ b/llvm/lib/Analysis/InstructionSimplify.cpp @@ -6633,7 +6633,8 @@ Value *llvm::simplifyBinaryIntrinsic(Intrinsic::ID IID, Type *ReturnType, return Op0; bool PropagateNaN = IID == Intrinsic::minimum || IID == Intrinsic::maximum; - bool IsMin = IID == Intrinsic::minimum || IID == Intrinsic::minimumnum || IID == Intrinsic::minnum; + bool IsMin = IID == Intrinsic::minimum || IID == Intrinsic::minimumnum || + IID == Intrinsic::minnum; // minnum(X, nan) -> X // maxnum(X, nan) -> X diff --git a/llvm/lib/Analysis/ValueTracking.cpp b/llvm/lib/Analysis/ValueTracking.cpp index 2de6a48179..2c66f0773b 100644 --- a/llvm/lib/Analysis/ValueTracking.cpp +++ b/llvm/lib/Analysis/ValueTracking.cpp @@ -5003,7 +5003,8 @@ void computeKnownFPClass(const Value *V, const APInt &DemandedElts, // If either operand is not NaN, the result is not NaN. if (NeverNaN && (IID == Intrinsic::minnum || IID == Intrinsic::maxnum)) Known.knownNot(fcNan); - if (NeverNaN && (IID == Intrinsic::minimumnum || IID == Intrinsic::maximumnum)) + if (NeverNaN && + (IID == Intrinsic::minimumnum || IID == Intrinsic::maximumnum)) Known.knownNot(fcNan); if (IID == Intrinsic::maxnum || IID == Intrinsic::maximumnum) { @@ -5062,15 +5063,19 @@ void computeKnownFPClass(const Value *V, const APInt &DemandedElts, Known.signBitMustBeOne(); else Known.signBitMustBeZero(); - } else if ((IID == Intrinsic::maximum || IID == Intrinsic::minimum || IID == Intrinsic::maximumnum || IID == Intrinsic::minimumnum) || + } else if ((IID == Intrinsic::maximum || IID == Intrinsic::minimum || + IID == Intrinsic::maximumnum || + IID == Intrinsic::minimumnum) || ((KnownLHS.isKnownNeverNegZero() || KnownRHS.isKnownNeverPosZero()) && (KnownLHS.isKnownNeverPosZero() || KnownRHS.isKnownNeverNegZero()))) { - if ((IID == Intrinsic::maximum || IID == Intrinsic::maximumnum|| IID == Intrinsic::maxnum) && + if ((IID == Intrinsic::maximum || IID == Intrinsic::maximumnum || + IID == Intrinsic::maxnum) && (KnownLHS.SignBit == false || KnownRHS.SignBit == false)) Known.signBitMustBeZero(); - else if ((IID == Intrinsic::minimum || IID == Intrinsic::minimumnum || IID == Intrinsic::minnum) && + else if ((IID == Intrinsic::minimum || IID == Intrinsic::minimumnum || + IID == Intrinsic::minnum) && (KnownLHS.SignBit == true || KnownRHS.SignBit == true)) Known.signBitMustBeOne(); } @@ -8567,8 +8572,10 @@ Intrinsic::ID llvm::getInverseMinMaxIntrinsic(Intrinsic::ID MinMaxID) { case Intrinsic::umin: return Intrinsic::umax; // Please note that next four intrinsics may produce the same result for // original and inverted case even if X != Y due to NaN is handled specially. - case Intrinsic::maximumnum: return Intrinsic::minimumnum; - case Intrinsic::minimumnum: return Intrinsic::maximumnum; + case Intrinsic::maximumnum: + return Intrinsic::minimumnum; + case Intrinsic::minimumnum: + return Intrinsic::maximumnum; case Intrinsic::maximum: return Intrinsic::minimum; case Intrinsic::minimum: return Intrinsic::maximum; case Intrinsic::maxnum: return Intrinsic::minnum; diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp index e1133cd232..6a96c4295a 100644 --- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp +++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp @@ -7271,8 +7271,9 @@ LegalizerHelper::lowerFMinNumMaxNum(MachineInstr &MI) { LegalizerHelper::LegalizeResult LegalizerHelper::lowerFMinimumNumMaximumNum(MachineInstr &MI) { - unsigned NewOp = MI.getOpcode() == TargetOpcode::G_FMINIMUMNUM ? - TargetOpcode::G_FMINNUM_IEEE : TargetOpcode::G_FMAXNUM_IEEE; + unsigned NewOp = MI.getOpcode() == TargetOpcode::G_FMINIMUMNUM + ? TargetOpcode::G_FMINNUM_IEEE + : TargetOpcode::G_FMAXNUM_IEEE; auto [Dst, Src0, Src1] = MI.getFirst3Regs(); LLT Ty = MRI.getType(Dst); diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp index 1504495a91..d99377ae5b 100644 --- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp +++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp @@ -18005,7 +18005,8 @@ SDValue DAGCombiner::visitFMinMax(SDNode *N) { const SDNodeFlags Flags = N->getFlags(); unsigned Opc = N->getOpcode(); bool PropagatesNaN = Opc == ISD::FMINIMUM || Opc == ISD::FMAXIMUM; - bool IsMin = Opc == ISD::FMINNUM || Opc == ISD::FMINIMUM || Opc == ISD::FMINIMUMNUM; + bool IsMin = + Opc == ISD::FMINNUM || Opc == ISD::FMINIMUM || Opc == ISD::FMINIMUMNUM; SelectionDAG::FlagInserter FlagsInserter(DAG, N); // Constant fold. diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp index 0772e33c4f..5ed6b4601f 100644 --- a/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp +++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp @@ -322,45 +322,37 @@ SDValue DAGTypeLegalizer::SoftenFloatRes_FMAXNUM(SDNode *N) { SDValue DAGTypeLegalizer::SoftenFloatRes_FMINIMUM(SDNode *N) { if (SDValue SelCC = TLI.createSelectForFMINIMUM_FMAXIMUM(N, DAG)) return SoftenFloatRes_SELECT_CC(SelCC.getNode()); - return SoftenFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMINIMUM_F32, - RTLIB::FMINIMUM_F64, - RTLIB::FMINIMUM_F80, - RTLIB::FMINIMUM_F128, - RTLIB::FMINIMUM_PPCF128)); + return SoftenFloatRes_Binary( + N, GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUM_F32, + RTLIB::FMINIMUM_F64, RTLIB::FMINIMUM_F80, + RTLIB::FMINIMUM_F128, RTLIB::FMINIMUM_PPCF128)); } SDValue DAGTypeLegalizer::SoftenFloatRes_FMAXIMUM(SDNode *N) { if (SDValue SelCC = TLI.createSelectForFMINIMUM_FMAXIMUM(N, DAG)) return SoftenFloatRes_SELECT_CC(SelCC.getNode()); - return SoftenFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMAXIMUM_F32, - RTLIB::FMAXIMUM_F64, - RTLIB::FMAXIMUM_F80, - RTLIB::FMAXIMUM_F128, - RTLIB::FMAXIMUM_PPCF128)); + return SoftenFloatRes_Binary( + N, GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUM_F32, + RTLIB::FMAXIMUM_F64, RTLIB::FMAXIMUM_F80, + RTLIB::FMAXIMUM_F128, RTLIB::FMAXIMUM_PPCF128)); } SDValue DAGTypeLegalizer::SoftenFloatRes_FMINIMUMNUM(SDNode *N) { if (SDValue SelCC = TLI.createSelectForFMINIMUMNUM_FMAXIMUMNUM(N, DAG)) return SoftenFloatRes_SELECT_CC(SelCC.getNode()); - return SoftenFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMINIMUMNUM_F32, - RTLIB::FMINIMUMNUM_F64, - RTLIB::FMINIMUMNUM_F80, - RTLIB::FMINIMUMNUM_F128, - RTLIB::FMINIMUMNUM_PPCF128)); + return SoftenFloatRes_Binary( + N, GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUMNUM_F32, + RTLIB::FMINIMUMNUM_F64, RTLIB::FMINIMUMNUM_F80, + RTLIB::FMINIMUMNUM_F128, RTLIB::FMINIMUMNUM_PPCF128)); } SDValue DAGTypeLegalizer::SoftenFloatRes_FMAXIMUMNUM(SDNode *N) { if (SDValue SelCC = TLI.createSelectForFMINIMUMNUM_FMAXIMUMNUM(N, DAG)) return SoftenFloatRes_SELECT_CC(SelCC.getNode()); - return SoftenFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMAXIMUMNUM_F32, - RTLIB::FMAXIMUMNUM_F64, - RTLIB::FMAXIMUMNUM_F80, - RTLIB::FMAXIMUMNUM_F128, - RTLIB::FMAXIMUMNUM_PPCF128)); + return SoftenFloatRes_Binary( + N, GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUMNUM_F32, + RTLIB::FMAXIMUMNUM_F64, RTLIB::FMAXIMUMNUM_F80, + RTLIB::FMAXIMUMNUM_F128, RTLIB::FMAXIMUMNUM_PPCF128)); } SDValue DAGTypeLegalizer::SoftenFloatRes_FADD(SDNode *N) { @@ -1403,13 +1395,21 @@ void DAGTypeLegalizer::ExpandFloatResult(SDNode *N, unsigned ResNo) { case ISD::STRICT_FMAXNUM: case ISD::FMAXNUM: ExpandFloatRes_FMAXNUM(N, Lo, Hi); break; case ISD::STRICT_FMINIMUM: - case ISD::FMINIMUM: ExpandFloatRes_FMINIMUM(N, Lo, Hi); break; + case ISD::FMINIMUM: + ExpandFloatRes_FMINIMUM(N, Lo, Hi); + break; case ISD::STRICT_FMAXIMUM: - case ISD::FMAXIMUM: ExpandFloatRes_FMAXIMUM(N, Lo, Hi); break; + case ISD::FMAXIMUM: + ExpandFloatRes_FMAXIMUM(N, Lo, Hi); + break; case ISD::STRICT_FMINIMUMNUM: - case ISD::FMINIMUMNUM: ExpandFloatRes_FMINIMUMNUM(N, Lo, Hi); break; + case ISD::FMINIMUMNUM: + ExpandFloatRes_FMINIMUMNUM(N, Lo, Hi); + break; case ISD::STRICT_FMAXIMUMNUM: - case ISD::FMAXIMUMNUM: ExpandFloatRes_FMAXIMUMNUM(N, Lo, Hi); break; + case ISD::FMAXIMUMNUM: + ExpandFloatRes_FMAXIMUMNUM(N, Lo, Hi); + break; case ISD::STRICT_FADD: case ISD::FADD: ExpandFloatRes_FADD(N, Lo, Hi); break; case ISD::FCBRT: ExpandFloatRes_FCBRT(N, Lo, Hi); break; @@ -1553,35 +1553,43 @@ void DAGTypeLegalizer::ExpandFloatRes_FMAXNUM(SDNode *N, SDValue &Lo, } void DAGTypeLegalizer::ExpandFloatRes_FMINIMUM(SDNode *N, SDValue &Lo, - SDValue &Hi) { - ExpandFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMINIMUM_F32, RTLIB::FMINIMUM_F64, - RTLIB::FMINIMUM_F80, RTLIB::FMINIMUM_F128, - RTLIB::FMINIMUM_PPCF128), Lo, Hi); + SDValue &Hi) { + ExpandFloatRes_Binary(N, + GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUM_F32, + RTLIB::FMINIMUM_F64, RTLIB::FMINIMUM_F80, + RTLIB::FMINIMUM_F128, + RTLIB::FMINIMUM_PPCF128), + Lo, Hi); } void DAGTypeLegalizer::ExpandFloatRes_FMAXIMUM(SDNode *N, SDValue &Lo, - SDValue &Hi) { - ExpandFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMAXIMUM_F32, RTLIB::FMAXIMUM_F64, - RTLIB::FMAXIMUM_F80, RTLIB::FMAXIMUM_F128, - RTLIB::FMAXIMUM_PPCF128), Lo, Hi); + SDValue &Hi) { + ExpandFloatRes_Binary(N, + GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUM_F32, + RTLIB::FMAXIMUM_F64, RTLIB::FMAXIMUM_F80, + RTLIB::FMAXIMUM_F128, + RTLIB::FMAXIMUM_PPCF128), + Lo, Hi); } void DAGTypeLegalizer::ExpandFloatRes_FMINIMUMNUM(SDNode *N, SDValue &Lo, - SDValue &Hi) { - ExpandFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMINIMUMNUM_F32, RTLIB::FMINIMUMNUM_F64, - RTLIB::FMINIMUMNUM_F80, RTLIB::FMINIMUMNUM_F128, - RTLIB::FMINIMUMNUM_PPCF128), Lo, Hi); + SDValue &Hi) { + ExpandFloatRes_Binary( + N, + GetFPLibCall(N->getValueType(0), RTLIB::FMINIMUMNUM_F32, + RTLIB::FMINIMUMNUM_F64, RTLIB::FMINIMUMNUM_F80, + RTLIB::FMINIMUMNUM_F128, RTLIB::FMINIMUMNUM_PPCF128), + Lo, Hi); } void DAGTypeLegalizer::ExpandFloatRes_FMAXIMUMNUM(SDNode *N, SDValue &Lo, - SDValue &Hi) { - ExpandFloatRes_Binary(N, GetFPLibCall(N->getValueType(0), - RTLIB::FMAXIMUMNUM_F32, RTLIB::FMAXIMUMNUM_F64, - RTLIB::FMAXIMUMNUM_F80, RTLIB::FMAXIMUMNUM_F128, - RTLIB::FMAXIMUMNUM_PPCF128), Lo, Hi); + SDValue &Hi) { + ExpandFloatRes_Binary( + N, + GetFPLibCall(N->getValueType(0), RTLIB::FMAXIMUMNUM_F32, + RTLIB::FMAXIMUMNUM_F64, RTLIB::FMAXIMUMNUM_F80, + RTLIB::FMAXIMUMNUM_F128, RTLIB::FMAXIMUMNUM_PPCF128), + Lo, Hi); } void DAGTypeLegalizer::ExpandFloatRes_FADD(SDNode *N, SDValue &Lo, diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h b/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h index 2472595ff5..a194f923d9 100644 --- a/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h +++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h @@ -623,10 +623,10 @@ private: void ExpandFloatRes_FABS (SDNode *N, SDValue &Lo, SDValue &Hi); void ExpandFloatRes_FMINNUM (SDNode *N, SDValue &Lo, SDValue &Hi); void ExpandFloatRes_FMAXNUM (SDNode *N, SDValue &Lo, SDValue &Hi); - void ExpandFloatRes_FMINIMUM (SDNode *N, SDValue &Lo, SDValue &Hi); - void ExpandFloatRes_FMAXIMUM (SDNode *N, SDValue &Lo, SDValue &Hi); - void ExpandFloatRes_FMINIMUMNUM (SDNode *N, SDValue &Lo, SDValue &Hi); - void ExpandFloatRes_FMAXIMUMNUM (SDNode *N, SDValue &Lo, SDValue &Hi); + void ExpandFloatRes_FMINIMUM(SDNode *N, SDValue &Lo, SDValue &Hi); + void ExpandFloatRes_FMAXIMUM(SDNode *N, SDValue &Lo, SDValue &Hi); + void ExpandFloatRes_FMINIMUMNUM(SDNode *N, SDValue &Lo, SDValue &Hi); + void ExpandFloatRes_FMAXIMUMNUM(SDNode *N, SDValue &Lo, SDValue &Hi); void ExpandFloatRes_FADD (SDNode *N, SDValue &Lo, SDValue &Hi); void ExpandFloatRes_FCBRT (SDNode *N, SDValue &Lo, SDValue &Hi); void ExpandFloatRes_FCEIL (SDNode *N, SDValue &Lo, SDValue &Hi); diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp index 9353d6b921..e5c0eb985f 100644 --- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp +++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp @@ -1178,10 +1178,14 @@ void DAGTypeLegalizer::SplitVectorResult(SDNode *N, unsigned ResNo) { case ISD::FMUL: case ISD::VP_FMUL: case ISD::FMINNUM: case ISD::VP_FMINNUM: case ISD::FMAXNUM: case ISD::VP_FMAXNUM: - case ISD::FMINIMUM: case ISD::FMAXIMUM: - case ISD::VP_FMINIMUM: case ISD::VP_FMAXIMUM: - case ISD::FMINIMUMNUM: case ISD::FMAXIMUMNUM: - case ISD::VP_FMINIMUMNUM: case ISD::VP_FMAXIMUMNUM: + case ISD::FMINIMUM: + case ISD::FMAXIMUM: + case ISD::VP_FMINIMUM: + case ISD::VP_FMAXIMUM: + case ISD::FMINIMUMNUM: + case ISD::FMAXIMUMNUM: + case ISD::VP_FMINIMUMNUM: + case ISD::VP_FMAXIMUMNUM: case ISD::SDIV: case ISD::VP_SDIV: case ISD::UDIV: case ISD::VP_UDIV: case ISD::FDIV: case ISD::VP_FDIV: @@ -4245,10 +4249,14 @@ void DAGTypeLegalizer::WidenVectorResult(SDNode *N, unsigned ResNo) { case ISD::SRL: case ISD::VP_LSHR: case ISD::FMINNUM: case ISD::VP_FMINNUM: case ISD::FMAXNUM: case ISD::VP_FMAXNUM: - case ISD::FMINIMUM: case ISD::FMAXIMUM: - case ISD::VP_FMINIMUM: case ISD::VP_FMAXIMUM: - case ISD::FMINIMUMNUM: case ISD::FMAXIMUMNUM: - case ISD::VP_FMINIMUMNUM: case ISD::VP_FMAXIMUMNUM: + case ISD::FMINIMUM: + case ISD::FMAXIMUM: + case ISD::VP_FMINIMUM: + case ISD::VP_FMAXIMUM: + case ISD::FMINIMUMNUM: + case ISD::FMAXIMUMNUM: + case ISD::VP_FMINIMUMNUM: + case ISD::VP_FMAXIMUMNUM: case ISD::SMIN: case ISD::VP_SMIN: case ISD::SMAX: case ISD::VP_SMAX: case ISD::UMIN: case ISD::VP_UMIN: diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp index 5c33d6c7f5..0b6f6fad7f 100644 --- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp +++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp @@ -528,8 +528,10 @@ std::string SDNode::getOperationName(const SelectionDAG *G) const { case ISD::VECREDUCE_FMIN: return "vecreduce_fmin"; case ISD::VECREDUCE_FMAXIMUM: return "vecreduce_fmaximum"; case ISD::VECREDUCE_FMINIMUM: return "vecreduce_fminimum"; - case ISD::VECREDUCE_FMAXIMUMNUM: return "vecreduce_fmaximumnum"; - case ISD::VECREDUCE_FMINIMUMNUM: return "vecreduce_fminimumnum"; + case ISD::VECREDUCE_FMAXIMUMNUM: + return "vecreduce_fmaximumnum"; + case ISD::VECREDUCE_FMINIMUMNUM: + return "vecreduce_fminimumnum"; case ISD::STACKMAP: return "stackmap"; case ISD::PATCHPOINT: diff --git a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp index c3b1bf0f4e..d04d475c72 100644 --- a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp +++ b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp @@ -8378,7 +8378,8 @@ SDValue TargetLowering::expandFMINNUM_FMAXNUM(SDNode *Node, "Expanding fminnum/fmaxnum for scalable vectors is undefined."); if (isOperationLegalOrCustom(NewOp, VT)) - return DAG.getNode(NewOp, dl, VT, Node->getOperand(0), Node->getOperand(1), Node->getFlags()); + return DAG.getNode(NewOp, dl, VT, Node->getOperand(0), Node->getOperand(1), + Node->getFlags()); // If the target has FMINIMUM/FMAXIMUM but not FMINNUM/FMAXNUM use that // instead if there are no NaNs and there can't be an incompatible zero @@ -8404,7 +8405,7 @@ SDValue TargetLowering::expandFMINNUM_FMAXNUM(SDNode *Node, SDValue TargetLowering::createSelectForFMINIMUM_FMAXIMUM(SDNode *Node, - SelectionDAG &DAG) const { + SelectionDAG &DAG) const { unsigned Opcode = Node->getOpcode(); assert((Opcode == ISD::FMINIMUM || Opcode == ISD::FMAXIMUM || Opcode == ISD::STRICT_FMINIMUM || Opcode == ISD::STRICT_FMAXIMUM) && @@ -8490,12 +8491,12 @@ SDValue TargetLowering::expandFMINIMUM_FMAXIMUM(SDNode *N, return MinMax; } -SDValue -TargetLowering::createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, - SelectionDAG &DAG) const { +SDValue TargetLowering::createSelectForFMINIMUMNUM_FMAXIMUMNUM( + SDNode *Node, SelectionDAG &DAG) const { unsigned Opcode = Node->getOpcode(); assert((Opcode == ISD::FMINIMUMNUM || Opcode == ISD::FMAXIMUMNUM || - Opcode == ISD::STRICT_FMINIMUMNUM || Opcode == ISD::STRICT_FMAXIMUMNUM) && + Opcode == ISD::STRICT_FMINIMUMNUM || + Opcode == ISD::STRICT_FMAXIMUMNUM) && "Wrong opcode"); if (Node->getFlags().hasNoNaNs()) { @@ -8515,10 +8516,10 @@ TargetLowering::createSelectForFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, } SDValue TargetLowering::expandFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, - SelectionDAG &DAG) const { + SelectionDAG &DAG) const { SDLoc dl(Node); - unsigned NewOp = Node->getOpcode() == ISD::FMINIMUMNUM ? - ISD::FMINNUM_IEEE : ISD::FMAXNUM_IEEE; + unsigned NewOp = Node->getOpcode() == ISD::FMINIMUMNUM ? ISD::FMINNUM_IEEE + : ISD::FMAXNUM_IEEE; EVT VT = Node->getValueType(0); if (VT.isScalableVector()) @@ -8533,12 +8534,12 @@ SDValue TargetLowering::expandFMINIMUMNUM_FMAXIMUMNUM(SDNode *Node, // Insert canonicalizes if it's possible we need to quiet to get correct // sNaN behavior. if (!DAG.isKnownNeverSNaN(Quiet0)) { - Quiet0 = DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet0, - Node->getFlags()); + Quiet0 = + DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet0, Node->getFlags()); } if (!DAG.isKnownNeverSNaN(Quiet1)) { - Quiet1 = DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet1, - Node->getFlags()); + Quiet1 = + DAG.getNode(ISD::FCANONICALIZE, dl, VT, Quiet1, Node->getFlags()); } } @@ -11117,27 +11118,30 @@ SDValue TargetLowering::expandFP_TO_INT_SAT(SDNode *Node, bool MinMaxLegal = isOperationLegal(ISD::FMINNUM, SrcVT) && isOperationLegal(ISD::FMAXNUM, SrcVT); bool MinMaxIEEELegal = isOperationLegal(ISD::FMINNUM_IEEE, SrcVT) && - isOperationLegal(ISD::FMAXNUM_IEEE, SrcVT); + isOperationLegal(ISD::FMAXNUM_IEEE, SrcVT); bool MinMaxNumLegal = isOperationLegal(ISD::FMINIMUMNUM, SrcVT) && - isOperationLegal(ISD::FMAXIMUMNUM, SrcVT); + isOperationLegal(ISD::FMAXIMUMNUM, SrcVT); bool CanonLegal = isOperationLegal(ISD::FCANONICALIZE, SrcVT); - if (AreExactFloatBounds && (MinMaxNumLegal || ((MinMaxIEEELegal || MinMaxLegal) && CanonLegal))) { + if (AreExactFloatBounds && + (MinMaxNumLegal || ((MinMaxIEEELegal || MinMaxLegal) && CanonLegal))) { SDValue Clamped = Src; if (MinMaxNumLegal) { - Clamped = DAG.getNode(ISD::FMAXIMUMNUM, dl, SrcVT, Clamped, MinFloatNode); - Clamped = DAG.getNode(ISD::FMINIMUMNUM, dl, SrcVT, Clamped, MaxFloatNode); + Clamped = DAG.getNode(ISD::FMAXIMUMNUM, dl, SrcVT, Clamped, MinFloatNode); + Clamped = DAG.getNode(ISD::FMINIMUMNUM, dl, SrcVT, Clamped, MaxFloatNode); } else if (MinMaxIEEELegal && CanonLegal) { - Clamped = DAG.getNode(ISD::FCANONICALIZE, dl, SrcVT, Clamped); - Clamped = DAG.getNode(ISD::FMAXNUM_IEEE, dl, SrcVT, Clamped, MinFloatNode); - Clamped = DAG.getNode(ISD::FMINNUM_IEEE, dl, SrcVT, Clamped, MaxFloatNode); + Clamped = DAG.getNode(ISD::FCANONICALIZE, dl, SrcVT, Clamped); + Clamped = + DAG.getNode(ISD::FMAXNUM_IEEE, dl, SrcVT, Clamped, MinFloatNode); + Clamped = + DAG.getNode(ISD::FMINNUM_IEEE, dl, SrcVT, Clamped, MaxFloatNode); } else if (MinMaxLegal && CanonLegal) { - // Clamp Src by MinFloat from below. If Src is NaN the result is MinFloat. - Clamped = DAG.getNode(ISD::FMAXNUM, dl, SrcVT, Clamped, MinFloatNode); - // Clamp by MaxFloat from above. NaN cannot occur. - Clamped = DAG.getNode(ISD::FMINNUM, dl, SrcVT, Clamped, MaxFloatNode); - }else{ - llvm_unreachable("No usable fmax/fmin instructions!"); + // Clamp Src by MinFloat from below. If Src is NaN the result is MinFloat. + Clamped = DAG.getNode(ISD::FMAXNUM, dl, SrcVT, Clamped, MinFloatNode); + // Clamp by MaxFloat from above. NaN cannot occur. + Clamped = DAG.getNode(ISD::FMINNUM, dl, SrcVT, Clamped, MaxFloatNode); + } else { + llvm_unreachable("No usable fmax/fmin instructions!"); } // Convert clamped value to integer. SDValue FpToInt = DAG.getNode(IsSigned ? ISD::FP_TO_SINT : ISD::FP_TO_UINT, diff --git a/llvm/lib/CodeGen/TargetLoweringBase.cpp b/llvm/lib/CodeGen/TargetLoweringBase.cpp index fd8e476471..3c5859fcb8 100644 --- a/llvm/lib/CodeGen/TargetLoweringBase.cpp +++ b/llvm/lib/CodeGen/TargetLoweringBase.cpp @@ -908,7 +908,7 @@ void TargetLoweringBase::initActions() { ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE, ISD::FMINIMUM, ISD::FMAXIMUM, - ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, + ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::FMAD, ISD::SMIN, ISD::SMAX, ISD::UMIN, ISD::UMAX, ISD::ABS, diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp index b8af1404eb..dd6ce35ad7 100644 --- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp +++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp @@ -808,17 +808,38 @@ AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM, setOperationAction(ISD::FP_ROUND, MVT::v4bf16, Custom); // AArch64 has implementations of a lot of rounding-like FP operations. - for (auto Op : - {ISD::FFLOOR, ISD::FNEARBYINT, ISD::FCEIL, - ISD::FRINT, ISD::FTRUNC, ISD::FROUND, - ISD::FROUNDEVEN, ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINNUM_IEEE, ISD::FMAXNUM_IEEE, - ISD::FMINIMUM, ISD::FMAXIMUM, ISD::LROUND, - ISD::LLROUND, ISD::LRINT, ISD::LLRINT, - ISD::STRICT_FFLOOR, ISD::STRICT_FCEIL, ISD::STRICT_FNEARBYINT, - ISD::STRICT_FRINT, ISD::STRICT_FTRUNC, ISD::STRICT_FROUNDEVEN, - ISD::STRICT_FROUND, ISD::STRICT_FMINNUM, ISD::STRICT_FMAXNUM, - ISD::STRICT_FMINIMUM, ISD::STRICT_FMAXIMUM, ISD::STRICT_LROUND, - ISD::STRICT_LLROUND, ISD::STRICT_LRINT, ISD::STRICT_LLRINT}) { + for (auto Op : {ISD::FFLOOR, + ISD::FNEARBYINT, + ISD::FCEIL, + ISD::FRINT, + ISD::FTRUNC, + ISD::FROUND, + ISD::FROUNDEVEN, + ISD::FMINNUM, + ISD::FMAXNUM, + ISD::FMINNUM_IEEE, + ISD::FMAXNUM_IEEE, + ISD::FMINIMUM, + ISD::FMAXIMUM, + ISD::LROUND, + ISD::LLROUND, + ISD::LRINT, + ISD::LLRINT, + ISD::STRICT_FFLOOR, + ISD::STRICT_FCEIL, + ISD::STRICT_FNEARBYINT, + ISD::STRICT_FRINT, + ISD::STRICT_FTRUNC, + ISD::STRICT_FROUNDEVEN, + ISD::STRICT_FROUND, + ISD::STRICT_FMINNUM, + ISD::STRICT_FMAXNUM, + ISD::STRICT_FMINIMUM, + ISD::STRICT_FMAXIMUM, + ISD::STRICT_LROUND, + ISD::STRICT_LLROUND, + ISD::STRICT_LRINT, + ISD::STRICT_LLRINT}) { for (MVT Ty : {MVT::f32, MVT::f64}) setOperationAction(Op, Ty, Legal); if (Subtarget->hasFullFP16()) diff --git a/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp b/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp index 3874d5cb32..caa3d133b4 100644 --- a/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp +++ b/llvm/lib/Target/Hexagon/HexagonISelLowering.cpp @@ -1629,28 +1629,77 @@ HexagonTargetLowering::HexagonTargetLowering(const TargetMachine &TM, // Set the action for vector operations to "expand", then override it with // either "custom" or "legal" for specific cases. static const unsigned VectExpOps[] = { - // Integer arithmetic: - ISD::ADD, ISD::SUB, ISD::MUL, ISD::SDIV, ISD::UDIV, - ISD::SREM, ISD::UREM, ISD::SDIVREM, ISD::UDIVREM, ISD::SADDO, - ISD::UADDO, ISD::SSUBO, ISD::USUBO, ISD::SMUL_LOHI, ISD::UMUL_LOHI, - // Logical/bit: - ISD::AND, ISD::OR, ISD::XOR, ISD::ROTL, ISD::ROTR, - ISD::CTPOP, ISD::CTLZ, ISD::CTTZ, ISD::BSWAP, ISD::BITREVERSE, - // Floating point arithmetic/math functions: - ISD::FADD, ISD::FSUB, ISD::FMUL, ISD::FMA, ISD::FDIV, - ISD::FREM, ISD::FNEG, ISD::FABS, ISD::FSQRT, ISD::FSIN, - ISD::FCOS, ISD::FPOW, ISD::FLOG, ISD::FLOG2, - ISD::FLOG10, ISD::FEXP, ISD::FEXP2, ISD::FCEIL, ISD::FTRUNC, - ISD::FRINT, ISD::FNEARBYINT, ISD::FROUND, ISD::FFLOOR, - ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::FSINCOS, ISD::FLDEXP, - // Misc: - ISD::BR_CC, ISD::SELECT_CC, ISD::ConstantPool, - // Vector: - ISD::BUILD_VECTOR, ISD::SCALAR_TO_VECTOR, - ISD::EXTRACT_VECTOR_ELT, ISD::INSERT_VECTOR_ELT, - ISD::EXTRACT_SUBVECTOR, ISD::INSERT_SUBVECTOR, - ISD::CONCAT_VECTORS, ISD::VECTOR_SHUFFLE, - ISD::SPLAT_VECTOR, + // Integer arithmetic: + ISD::ADD, + ISD::SUB, + ISD::MUL, + ISD::SDIV, + ISD::UDIV, + ISD::SREM, + ISD::UREM, + ISD::SDIVREM, + ISD::UDIVREM, + ISD::SADDO, + ISD::UADDO, + ISD::SSUBO, + ISD::USUBO, + ISD::SMUL_LOHI, + ISD::UMUL_LOHI, + // Logical/bit: + ISD::AND, + ISD::OR, + ISD::XOR, + ISD::ROTL, + ISD::ROTR, + ISD::CTPOP, + ISD::CTLZ, + ISD::CTTZ, + ISD::BSWAP, + ISD::BITREVERSE, + // Floating point arithmetic/math functions: + ISD::FADD, + ISD::FSUB, + ISD::FMUL, + ISD::FMA, + ISD::FDIV, + ISD::FREM, + ISD::FNEG, + ISD::FABS, + ISD::FSQRT, + ISD::FSIN, + ISD::FCOS, + ISD::FPOW, + ISD::FLOG, + ISD::FLOG2, + ISD::FLOG10, + ISD::FEXP, + ISD::FEXP2, + ISD::FCEIL, + ISD::FTRUNC, + ISD::FRINT, + ISD::FNEARBYINT, + ISD::FROUND, + ISD::FFLOOR, + ISD::FMINNUM, + ISD::FMAXNUM, + ISD::FMINIMUMNUM, + ISD::FMAXIMUMNUM, + ISD::FSINCOS, + ISD::FLDEXP, + // Misc: + ISD::BR_CC, + ISD::SELECT_CC, + ISD::ConstantPool, + // Vector: + ISD::BUILD_VECTOR, + ISD::SCALAR_TO_VECTOR, + ISD::EXTRACT_VECTOR_ELT, + ISD::INSERT_VECTOR_ELT, + ISD::EXTRACT_SUBVECTOR, + ISD::INSERT_SUBVECTOR, + ISD::CONCAT_VECTORS, + ISD::VECTOR_SHUFFLE, + ISD::SPLAT_VECTOR, }; for (MVT VT : MVT::fixedlen_vector_valuetypes()) { diff --git a/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp b/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp index fc3f01b322..939974b9e9 100644 --- a/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp +++ b/llvm/lib/Target/Hexagon/HexagonISelLoweringHVX.cpp @@ -129,8 +129,8 @@ HexagonTargetLowering::initializeHVXLowering() { setOperationAction(ISD::FMUL, T, Legal); setOperationAction(ISD::FMINNUM, T, Legal); setOperationAction(ISD::FMAXNUM, T, Legal); - setOperationAction(ISD::FMINIMUMNUM, T, Legal); - setOperationAction(ISD::FMAXIMUMNUM, T, Legal); + setOperationAction(ISD::FMINIMUMNUM, T, Legal); + setOperationAction(ISD::FMAXIMUMNUM, T, Legal); setOperationAction(ISD::INSERT_SUBVECTOR, T, Custom); setOperationAction(ISD::EXTRACT_SUBVECTOR, T, Custom); @@ -168,8 +168,8 @@ HexagonTargetLowering::initializeHVXLowering() { setOperationAction(ISD::FMUL, P, Custom); setOperationAction(ISD::FMINNUM, P, Custom); setOperationAction(ISD::FMAXNUM, P, Custom); - setOperationAction(ISD::FMINIMUMNUM, P, Custom); - setOperationAction(ISD::FMAXIMUMNUM, P, Custom); + setOperationAction(ISD::FMINIMUMNUM, P, Custom); + setOperationAction(ISD::FMAXIMUMNUM, P, Custom); setOperationAction(ISD::SETCC, P, Custom); setOperationAction(ISD::VSELECT, P, Custom); diff --git a/llvm/lib/Target/RISCV/GISel/RISCVLegalizerInfo.cpp b/llvm/lib/Target/RISCV/GISel/RISCVLegalizerInfo.cpp index 951580cc94..a8397301d2 100644 --- a/llvm/lib/Target/RISCV/GISel/RISCVLegalizerInfo.cpp +++ b/llvm/lib/Target/RISCV/GISel/RISCVLegalizerInfo.cpp @@ -372,7 +372,8 @@ RISCVLegalizerInfo::RISCVLegalizerInfo(const RISCVSubtarget &ST) // FP Operations getActionDefinitionsBuilder({G_FADD, G_FSUB, G_FMUL, G_FDIV, G_FMA, G_FNEG, - G_FABS, G_FSQRT, G_FMAXNUM, G_FMINNUM, G_FMAXIMUMNUM, G_FMINIMUMNUM}) + G_FABS, G_FSQRT, G_FMAXNUM, G_FMINNUM, + G_FMAXIMUMNUM, G_FMINIMUMNUM}) .legalIf(typeIsScalarFPArith(0, ST)); getActionDefinitionsBuilder(G_FCOPYSIGN) diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp index 0097d2a6ab..c1edb32e16 100644 --- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp +++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp @@ -431,12 +431,13 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, } static const unsigned FPLegalNodeTypes[] = { - ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::LRINT, - ISD::LLRINT, ISD::LROUND, ISD::LLROUND, - ISD::STRICT_LRINT, ISD::STRICT_LLRINT, ISD::STRICT_LROUND, - ISD::STRICT_LLROUND, ISD::STRICT_FMA, ISD::STRICT_FADD, - ISD::STRICT_FSUB, ISD::STRICT_FMUL, ISD::STRICT_FDIV, - ISD::STRICT_FSQRT, ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS}; + ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, + ISD::FMAXIMUMNUM, ISD::LRINT, ISD::LLRINT, + ISD::LROUND, ISD::LLROUND, ISD::STRICT_LRINT, + ISD::STRICT_LLRINT, ISD::STRICT_LROUND, ISD::STRICT_LLROUND, + ISD::STRICT_FMA, ISD::STRICT_FADD, ISD::STRICT_FSUB, + ISD::STRICT_FMUL, ISD::STRICT_FDIV, ISD::STRICT_FSQRT, + ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS}; static const ISD::CondCode FPCCToExpand[] = { ISD::SETOGT, ISD::SETOGE, ISD::SETONE, ISD::SETUEQ, ISD::SETUGT, @@ -455,15 +456,20 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, setOperationAction(ISD::BITCAST, MVT::i16, Custom); static const unsigned ZfhminZfbfminPromoteOps[] = { - ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::FADD, - ISD::FSUB, ISD::FMUL, ISD::FMA, - ISD::FDIV, ISD::FSQRT, ISD::FABS, - ISD::FNEG, ISD::STRICT_FMA, ISD::STRICT_FADD, - ISD::STRICT_FSUB, ISD::STRICT_FMUL, ISD::STRICT_FDIV, - ISD::STRICT_FSQRT, ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS, - ISD::SETCC, ISD::FCEIL, ISD::FFLOOR, - ISD::FTRUNC, ISD::FRINT, ISD::FROUND, - ISD::FROUNDEVEN, ISD::SELECT}; + ISD::FMINNUM, ISD::FMAXNUM, + ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, + ISD::FADD, ISD::FSUB, + ISD::FMUL, ISD::FMA, + ISD::FDIV, ISD::FSQRT, + ISD::FABS, ISD::FNEG, + ISD::STRICT_FMA, ISD::STRICT_FADD, + ISD::STRICT_FSUB, ISD::STRICT_FMUL, + ISD::STRICT_FDIV, ISD::STRICT_FSQRT, + ISD::STRICT_FSETCC, ISD::STRICT_FSETCCS, + ISD::SETCC, ISD::FCEIL, + ISD::FFLOOR, ISD::FTRUNC, + ISD::FRINT, ISD::FROUND, + ISD::FROUNDEVEN, ISD::SELECT}; if (Subtarget.hasStdExtZfbfmin()) { setOperationAction(ISD::BITCAST, MVT::i16, Custom); @@ -700,21 +706,46 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, ISD::VP_SADDSAT, ISD::VP_UADDSAT, ISD::VP_SSUBSAT, ISD::VP_USUBSAT, ISD::VP_CTTZ_ELTS, ISD::VP_CTTZ_ELTS_ZERO_UNDEF}; - static const unsigned FloatingPointVPOps[] = { - ISD::VP_FADD, ISD::VP_FSUB, ISD::VP_FMUL, - ISD::VP_FDIV, ISD::VP_FNEG, ISD::VP_FABS, - ISD::VP_FMA, ISD::VP_REDUCE_FADD, ISD::VP_REDUCE_SEQ_FADD, - ISD::VP_REDUCE_FMIN, ISD::VP_REDUCE_FMAX, ISD::VP_MERGE, - ISD::VP_SELECT, ISD::VP_SINT_TO_FP, ISD::VP_UINT_TO_FP, - ISD::VP_SETCC, ISD::VP_FP_ROUND, ISD::VP_FP_EXTEND, - ISD::VP_SQRT, ISD::VP_FMINNUM, ISD::VP_FMAXNUM, - ISD::VP_FCEIL, ISD::VP_FFLOOR, ISD::VP_FROUND, - ISD::VP_FROUNDEVEN, ISD::VP_FCOPYSIGN, ISD::VP_FROUNDTOZERO, - ISD::VP_FRINT, ISD::VP_FNEARBYINT, ISD::VP_IS_FPCLASS, - ISD::VP_FMINIMUM, ISD::VP_FMAXIMUM, ISD::VP_FMINIMUMNUM, ISD::VP_FMAXIMUMNUM, ISD::VP_LRINT, - ISD::VP_LLRINT, ISD::EXPERIMENTAL_VP_REVERSE, - ISD::EXPERIMENTAL_VP_SPLICE, ISD::VP_REDUCE_FMINIMUM, - ISD::VP_REDUCE_FMAXIMUM}; + static const unsigned FloatingPointVPOps[] = {ISD::VP_FADD, + ISD::VP_FSUB, + ISD::VP_FMUL, + ISD::VP_FDIV, + ISD::VP_FNEG, + ISD::VP_FABS, + ISD::VP_FMA, + ISD::VP_REDUCE_FADD, + ISD::VP_REDUCE_SEQ_FADD, + ISD::VP_REDUCE_FMIN, + ISD::VP_REDUCE_FMAX, + ISD::VP_MERGE, + ISD::VP_SELECT, + ISD::VP_SINT_TO_FP, + ISD::VP_UINT_TO_FP, + ISD::VP_SETCC, + ISD::VP_FP_ROUND, + ISD::VP_FP_EXTEND, + ISD::VP_SQRT, + ISD::VP_FMINNUM, + ISD::VP_FMAXNUM, + ISD::VP_FCEIL, + ISD::VP_FFLOOR, + ISD::VP_FROUND, + ISD::VP_FROUNDEVEN, + ISD::VP_FCOPYSIGN, + ISD::VP_FROUNDTOZERO, + ISD::VP_FRINT, + ISD::VP_FNEARBYINT, + ISD::VP_IS_FPCLASS, + ISD::VP_FMINIMUM, + ISD::VP_FMAXIMUM, + ISD::VP_FMINIMUMNUM, + ISD::VP_FMAXIMUMNUM, + ISD::VP_LRINT, + ISD::VP_LLRINT, + ISD::EXPERIMENTAL_VP_REVERSE, + ISD::EXPERIMENTAL_VP_SPLICE, + ISD::VP_REDUCE_FMINIMUM, + ISD::VP_REDUCE_FMAXIMUM}; static const unsigned IntegerVecReduceOps[] = { ISD::VECREDUCE_ADD, ISD::VECREDUCE_AND, ISD::VECREDUCE_OR, @@ -942,25 +973,45 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, // TODO: support more ops. static const unsigned ZvfhminPromoteOps[] = { - ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::FADD, ISD::FSUB, - ISD::FMUL, ISD::FMA, ISD::FDIV, ISD::FSQRT, - ISD::FABS, ISD::FNEG, ISD::FCOPYSIGN, ISD::FCEIL, - ISD::FFLOOR, ISD::FROUND, ISD::FROUNDEVEN, ISD::FRINT, - ISD::FNEARBYINT, ISD::IS_FPCLASS, ISD::SETCC, ISD::FMAXIMUM, - ISD::FMINIMUM, ISD::STRICT_FADD, ISD::STRICT_FSUB, ISD::STRICT_FMUL, - ISD::STRICT_FDIV, ISD::STRICT_FSQRT, ISD::STRICT_FMA}; + ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, + ISD::FADD, ISD::FSUB, ISD::FMUL, ISD::FMA, + ISD::FDIV, ISD::FSQRT, ISD::FABS, ISD::FNEG, + ISD::FCOPYSIGN, ISD::FCEIL, ISD::FFLOOR, ISD::FROUND, + ISD::FROUNDEVEN, ISD::FRINT, ISD::FNEARBYINT, ISD::IS_FPCLASS, + ISD::SETCC, ISD::FMAXIMUM, ISD::FMINIMUM, ISD::STRICT_FADD, + ISD::STRICT_FSUB, ISD::STRICT_FMUL, ISD::STRICT_FDIV, ISD::STRICT_FSQRT, + ISD::STRICT_FMA}; // TODO: support more vp ops. - static const unsigned ZvfhminPromoteVPOps[] = { - ISD::VP_FADD, ISD::VP_FSUB, ISD::VP_FMUL, - ISD::VP_FDIV, ISD::VP_FNEG, ISD::VP_FABS, - ISD::VP_FMA, ISD::VP_REDUCE_FADD, ISD::VP_REDUCE_SEQ_FADD, - ISD::VP_REDUCE_FMIN, ISD::VP_REDUCE_FMAX, ISD::VP_SQRT, - ISD::VP_FMINNUM, ISD::VP_FMAXNUM, ISD::VP_FMINIMUMNUM, ISD::VP_FMAXIMUMNUM, ISD::VP_FCEIL, - ISD::VP_FFLOOR, ISD::VP_FROUND, ISD::VP_FROUNDEVEN, - ISD::VP_FCOPYSIGN, ISD::VP_FROUNDTOZERO, ISD::VP_FRINT, - ISD::VP_FNEARBYINT, ISD::VP_SETCC, ISD::VP_FMINIMUM, - ISD::VP_FMAXIMUM, ISD::VP_REDUCE_FMINIMUM, ISD::VP_REDUCE_FMAXIMUM}; + static const unsigned ZvfhminPromoteVPOps[] = {ISD::VP_FADD, + ISD::VP_FSUB, + ISD::VP_FMUL, + ISD::VP_FDIV, + ISD::VP_FNEG, + ISD::VP_FABS, + ISD::VP_FMA, + ISD::VP_REDUCE_FADD, + ISD::VP_REDUCE_SEQ_FADD, + ISD::VP_REDUCE_FMIN, + ISD::VP_REDUCE_FMAX, + ISD::VP_SQRT, + ISD::VP_FMINNUM, + ISD::VP_FMAXNUM, + ISD::VP_FMINIMUMNUM, + ISD::VP_FMAXIMUMNUM, + ISD::VP_FCEIL, + ISD::VP_FFLOOR, + ISD::VP_FROUND, + ISD::VP_FROUNDEVEN, + ISD::VP_FCOPYSIGN, + ISD::VP_FROUNDTOZERO, + ISD::VP_FRINT, + ISD::VP_FNEARBYINT, + ISD::VP_SETCC, + ISD::VP_FMINIMUM, + ISD::VP_FMAXIMUM, + ISD::VP_REDUCE_FMINIMUM, + ISD::VP_REDUCE_FMAXIMUM}; // Sets common operation actions on RVV floating-point vector types. const auto SetCommonVFPActions = [&](MVT VT) { @@ -1357,8 +1408,9 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, setOperationAction({ISD::FADD, ISD::FSUB, ISD::FMUL, ISD::FDIV, ISD::FNEG, ISD::FABS, ISD::FCOPYSIGN, ISD::FSQRT, - ISD::FMA, ISD::FMINNUM, ISD::FMAXNUM, ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, - ISD::IS_FPCLASS, ISD::FMAXIMUM, ISD::FMINIMUM}, + ISD::FMA, ISD::FMINNUM, ISD::FMAXNUM, + ISD::FMINIMUMNUM, ISD::FMAXIMUMNUM, ISD::IS_FPCLASS, + ISD::FMAXIMUM, ISD::FMINIMUM}, VT, Custom); setOperationAction({ISD::FP_ROUND, ISD::FP_EXTEND}, VT, Custom); @@ -1448,7 +1500,8 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM, setTargetDAGCombine(ISD::SRA); if (Subtarget.hasStdExtFOrZfinx()) - setTargetDAGCombine({ISD::FADD, ISD::FMAXNUM, ISD::FMINNUM, ISD::FMAXIMUMNUM, ISD::FMINIMUMNUM}); + setTargetDAGCombine({ISD::FADD, ISD::FMAXNUM, ISD::FMINNUM, + ISD::FMAXIMUMNUM, ISD::FMINIMUMNUM}); if (Subtarget.hasStdExtZbb()) setTargetDAGCombine({ISD::UMAX, ISD::UMIN, ISD::SMAX, ISD::SMIN}); diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp index ab2816e480..80d28f5294 100644 --- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp +++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp @@ -4285,7 +4285,8 @@ X86TTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA, // If there are no NANs to deal with, then these are reduced to a // single MIN** or MAX** instruction instead of the MIN/CMP/SELECT that we // assume is used in the non-fast case. - if (ISD == ISD::FMAXNUM || ISD == ISD::FMINNUM || ISD == ISD::FMAXNUM || ISD == ISD::FMINNUM) { + if (ISD == ISD::FMAXNUM || ISD == ISD::FMINNUM || ISD == ISD::FMAXNUM || + ISD == ISD::FMINNUM) { if (FMF.noNaNs()) return LegalizationCost * 1; } @@ -5424,11 +5425,11 @@ X86TTIImpl::getMinMaxReductionCost(Intrinsic::ID IID, VectorType *ValTy, assert(ValTy->isFPOrFPVectorTy() && "Expected float point or integer vector type."); if (IID == Intrinsic::minimumnum || IID == Intrinsic::maximumnum) -ISD = ISD::FMINIMUMNUM; + ISD = ISD::FMINIMUMNUM; else - ISD = (IID == Intrinsic::minnum || IID == Intrinsic::maxnum) - ? ISD::FMINNUM - : ISD::FMINIMUM; + ISD = (IID == Intrinsic::minnum || IID == Intrinsic::maxnum) + ? ISD::FMINNUM + : ISD::FMINIMUM; } // We use the Intel Architecture Code Analyzer(IACA) to measure the throughput |
Contributor Author
| These ports need to update:
ARM64/MIPSr6/LoongArch have no TODOs. Clang needs update to recognize |
Contributor
| Needs LangRef update and an RFC. I only briefly looked at this, but I think we can probably get away with just changing minnum to follow IEEE behavior and then make use of the following general rule about our FP operations:
The effective outcome of this would be that both behaviors become valid. The sNaN -> qNaN case would be the plain intrinsic definition and the sNaN -> other operand case would be via sNaN -> qNaN relaxation. Only the constrained minnum intrinsics would require precise sNaN/qNaN distinction, as usual. I think this would be in the spirit of the general FP handling in LLVM. |
Contributor Author
RISC-V/Hexagon has IEEE754-2019
My major concern is C code like double fmin(double, double); double f(double a, double b) { return fmin(a, b); }This code may expect the same behavior as libc's fmin(3), while we convert it to
|
arsenm reviewed May 28, 2024
Contributor
arsenm left a comment
arsenm left a comment There was a problem hiding this comment.
I need to find the time to do a proper review, but the change in behavior should be a separate PR from the introduction of a new intrinsic
arsenm reviewed May 28, 2024
llvm/include/llvm/ADT/APFloat.h Outdated
Comment on lines 1424 to 1430
| if (A.isNaN()) | ||
| return A; | ||
| return A.makeQuiet(); | ||
| if (B.isNaN()) | ||
| return B; | ||
| return B.makeQuiet(); | ||
| if (A.isZero() && B.isZero() && (A.isNegative() != B.isNegative())) | ||
| return A.isNegative() ? A : B; | ||
| return B < A ? B : A; |
Contributor
There was a problem hiding this comment.
Changing minimum and maximum constant folding should be a separate PR with InstSImplify tests showing constant folding of signaling nan
Contributor
jcranmer-intel left a comment
jcranmer-intel left a comment There was a problem hiding this comment.
Also probably worth picking one of the intrinsics to canonicalize to in the nnan case.
llvm/include/llvm/ADT/APFloat.h Outdated
| return B < A ? B : A; | ||
| } | ||
| LLVM_READONLY | ||
| inline APFloat minimumnum(const APFloat &A, const APFloat &B) { |
Contributor
There was a problem hiding this comment.
The APFloat changes need unittests.
llvm/include/llvm/IR/Intrinsics.td Outdated
| def int_maximumnum : DefaultAttrsIntrinsic<[llvm_anyfloat_ty], | ||
| [LLVMMatchType<0>, LLVMMatchType<0>], | ||
| [IntrNoMem, IntrSpeculatable, IntrWillReturn, Commutative] | ||
| >; |
Contributor
There was a problem hiding this comment.
Intrinsics need LangRef documentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Currently, on different platform, the behaivor of llvm.minnum is different if one operand is sNaN.
When we compute sNaN vs NUM
ARM/AArch64 follow the IEEE754-2008's minNUM: return qNaN.
LIBCALL: returns qNaN.
RV/X86/Hexagon follow the IEEE754-2019's minimumNumber: return NUM.
MIPS/LoongArch/Generic: return NUM.
So, let's introduce
llvm.minmumnum/llvm.maximumnum, which always follow IEEE754-2019's minimumNumber/maximumNumber.Since
llvm.minnumshares the same name with IEEE754-2008's minNUM, and currently, it is used for fmin(3), we should ask all ports to switch the behavior to minNUM.Fixes: #93033