[CIR] Upstream AddressSpace conversions support #161212
Open
+290 −11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This stack of pull requests is managed by Graphite. Learn more about stacking. |
| ✅ With the latest revision this PR passed the C/C++ code formatter. |
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch 2 times, most recently from 52f8f0f to a631340 Compare RiverDave requested review from andykaylor, bcardosolopes, lanza and xlauko as code owners 
llvmbot added clang RiverDave changed the title AddressSpace conversions support 
| @llvm/pr-subscribers-clang Author: David Rivera (RiverDave) ChangesAdd support for address space conversions in CIR.
Full diff: https://github.com/llvm/llvm-project/pull/161212.diff 11 Files Affected:
diff --git a/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h b/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h index cef8624e65d57..bf4a9b8438982 100644 --- a/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h +++ b/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h @@ -424,6 +424,15 @@ class CIRBaseBuilderTy : public mlir::OpBuilder { return createBitcast(src, getPointerTo(newPointeeTy)); } + mlir::Value createAddrSpaceCast(mlir::Location loc, mlir::Value src, + mlir::Type newTy) { + return createCast(loc, cir::CastKind::address_space, src, newTy); + } + + mlir::Value createAddrSpaceCast(mlir::Value src, mlir::Type newTy) { + return createAddrSpaceCast(src.getLoc(), src, newTy); + } + //===--------------------------------------------------------------------===// // Binary Operators //===--------------------------------------------------------------------===// diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp index cf17de144f4d9..95e392d860518 100644 --- a/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp @@ -58,6 +58,24 @@ static RValue emitBuiltinBitOp(CIRGenFunction &cgf, const CallExpr *e, return RValue::get(result); } +// Initialize the alloca with the given size and alignment according to the lang +// opts. Supporting only the trivial non-initialization for now. +static void initializeAlloca(CIRGenFunction &CGF, + [[maybe_unused]] mlir::Value AllocaAddr, + [[maybe_unused]] mlir::Value Size, + [[maybe_unused]] CharUnits AlignmentInBytes) { + + switch (CGF.getLangOpts().getTrivialAutoVarInit()) { + case LangOptions::TrivialAutoVarInitKind::Uninitialized: + // Nothing to initialize. + return; + case LangOptions::TrivialAutoVarInitKind::Zero: + case LangOptions::TrivialAutoVarInitKind::Pattern: + assert(false && "unexpected trivial auto var init kind NYI"); + return; + } +} + RValue CIRGenFunction::emitRotate(const CallExpr *e, bool isRotateLeft) { mlir::Value input = emitScalarExpr(e->getArg(0)); mlir::Value amount = emitScalarExpr(e->getArg(1)); @@ -172,21 +190,8 @@ RValue CIRGenFunction::emitBuiltinExpr(const GlobalDecl &gd, unsigned builtinID, builder.getUInt8Ty(), "bi_alloca", suitableAlignmentInBytes, size); // Initialize the allocated buffer if required. - if (builtinID != Builtin::BI__builtin_alloca_uninitialized) { - // Initialize the alloca with the given size and alignment according to - // the lang opts. Only the trivial non-initialization is supported for - // now. - - switch (getLangOpts().getTrivialAutoVarInit()) { - case LangOptions::TrivialAutoVarInitKind::Uninitialized: - // Nothing to initialize. - break; - case LangOptions::TrivialAutoVarInitKind::Zero: - case LangOptions::TrivialAutoVarInitKind::Pattern: - cgm.errorNYI("trivial auto var init"); - break; - } - } + if (builtinID != Builtin::BI__builtin_alloca_uninitialized) + initializeAlloca(*this, allocaAddr, size, suitableAlignmentInBytes); // An alloca will always return a pointer to the alloca (stack) address // space. This address space need not be the same as the AST / Language @@ -194,6 +199,12 @@ RValue CIRGenFunction::emitBuiltinExpr(const GlobalDecl &gd, unsigned builtinID, // the AST level this is handled within CreateTempAlloca et al., but for the // builtin / dynamic alloca we have to handle it here. assert(!cir::MissingFeatures::addressSpace()); + cir::AddressSpace aas = getCIRAllocaAddressSpace(); + cir::AddressSpace eas = cir::toCIRAddressSpace( + e->getType()->getPointeeType().getAddressSpace()); + if (eas != aas) { + assert(false && "Non-default address space for alloca NYI"); + } // Bitcast the alloca to the expected type. return RValue::get( diff --git a/clang/lib/CIR/CodeGen/CIRGenExpr.cpp b/clang/lib/CIR/CodeGen/CIRGenExpr.cpp index 44626bbdd1dfa..f65d502a12318 100644 --- a/clang/lib/CIR/CodeGen/CIRGenExpr.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenExpr.cpp @@ -1190,7 +1190,19 @@ LValue CIRGenFunction::emitCastLValue(const CastExpr *e) { case CK_Dynamic: case CK_ToUnion: case CK_BaseToDerived: - case CK_AddressSpaceConversion: + case CK_AddressSpaceConversion: { + LValue lv = emitLValue(e->getSubExpr()); + QualType destTy = getContext().getPointerType(e->getType()); + cir::AddressSpace srcAS = + cir::toCIRAddressSpace(e->getSubExpr()->getType().getAddressSpace()); + cir::AddressSpace destAS = + cir::toCIRAddressSpace(e->getType().getAddressSpace()); + mlir::Value V = getTargetHooks().performAddrSpaceCast( + *this, lv.getPointer(), srcAS, destAS, convertType(destTy)); + return makeAddrLValue(Address(V, convertTypeForMem(e->getType()), + lv.getAddress().getAlignment()), + e->getType(), lv.getBaseInfo()); + } case CK_ObjCObjectLValueCast: case CK_VectorSplat: case CK_ConstructorConversion: @@ -2283,7 +2295,10 @@ Address CIRGenFunction::createTempAlloca(mlir::Type ty, CharUnits align, // be different from the type defined by the language. For example, // in C++ the auto variables are in the default address space. Therefore // cast alloca to the default address space when necessary. - assert(!cir::MissingFeatures::addressSpace()); + if (auto astAS = cir::toCIRAddressSpace(cgm.getLangTempAllocaAddressSpace()); + getCIRAllocaAddressSpace() != astAS) { + llvm_unreachable("Requires address space cast which is NYI"); + } return Address(v, ty, align); } diff --git a/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp b/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp index bd09d78cd0eb6..48ffaf9770e7c 100644 --- a/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp @@ -87,6 +87,7 @@ class ScalarExprEmitter : public StmtVisitor<ScalarExprEmitter, mlir::Value> { //===--------------------------------------------------------------------===// // Utilities //===--------------------------------------------------------------------===// + mlir::Type convertType(QualType ty) { return cgf.convertType(ty); } mlir::Value emitComplexToScalarConversion(mlir::Location loc, mlir::Value value, CastKind kind, @@ -1862,6 +1863,27 @@ mlir::Value ScalarExprEmitter::VisitCastExpr(CastExpr *ce) { return cgf.getBuilder().createBitcast(cgf.getLoc(subExpr->getSourceRange()), src, dstTy); } + case CK_AddressSpaceConversion: { + Expr::EvalResult result; + if (subExpr->EvaluateAsRValue(result, cgf.getContext()) && + result.Val.isNullPointer()) { + // If E has side effect, it is emitted even if its final result is a + // null pointer. In that case, a DCE pass should be able to + // eliminate the useless instructions emitted during translating E. + if (result.HasSideEffects) + Visit(subExpr); + return cgf.cgm.emitNullConstant(destTy, + cgf.getLoc(subExpr->getExprLoc())); + } + // Since target may map different address spaces in AST to the same address + // space, an address space conversion may end up as a bitcast. + cir::AddressSpace srcAS = cir::toCIRAddressSpace( + subExpr->getType()->getPointeeType().getAddressSpace()); + cir::AddressSpace destAS = + cir::toCIRAddressSpace(destTy->getPointeeType().getAddressSpace()); + return cgf.cgm.getTargetCIRGenInfo().performAddrSpaceCast( + cgf, Visit(subExpr), srcAS, destAS, convertType(destTy)); + } case CK_AtomicToNonAtomic: { cgf.getCIRGenModule().errorNYI(subExpr->getSourceRange(), diff --git a/clang/lib/CIR/CodeGen/CIRGenFunction.h b/clang/lib/CIR/CodeGen/CIRGenFunction.h index 166435f9e7e9e..f1158c7488f14 100644 --- a/clang/lib/CIR/CodeGen/CIRGenFunction.h +++ b/clang/lib/CIR/CodeGen/CIRGenFunction.h @@ -160,6 +160,10 @@ class CIRGenFunction : public CIRGenTypeCache { const TargetInfo &getTarget() const { return cgm.getTarget(); } mlir::MLIRContext &getMLIRContext() { return cgm.getMLIRContext(); } + const TargetCIRGenInfo &getTargetHooks() const { + return cgm.getTargetCIRGenInfo(); + } + // --------------------- // Opaque value handling // --------------------- diff --git a/clang/lib/CIR/CodeGen/CIRGenModule.cpp b/clang/lib/CIR/CodeGen/CIRGenModule.cpp index eef23a0ebda7f..5ff7c6cf25f80 100644 --- a/clang/lib/CIR/CodeGen/CIRGenModule.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenModule.cpp @@ -1363,6 +1363,23 @@ CIRGenModule::getAddrOfConstantStringFromLiteral(const StringLiteral *s, return builder.getGlobalViewAttr(ptrTy, gv); } +// TODO(cir): this could be a common AST helper for both CIR and LLVM codegen. +LangAS CIRGenModule::getLangTempAllocaAddressSpace() const { + if (getLangOpts().OpenCL) + return LangAS::opencl_private; + + // For temporaries inside functions, CUDA treats them as normal variables. + // LangAS::cuda_device, on the other hand, is reserved for those variables + // explicitly marked with __device__. + if (getLangOpts().CUDAIsDevice) + return LangAS::Default; + + if (getLangOpts().SYCLIsDevice || + (getLangOpts().OpenMP && getLangOpts().OpenMPIsTargetDevice)) + llvm_unreachable("NYI"); + return LangAS::Default; +} + void CIRGenModule::emitExplicitCastExprType(const ExplicitCastExpr *e, CIRGenFunction *cgf) { if (cgf && e->getType()->isVariablyModifiedType()) diff --git a/clang/lib/CIR/CodeGen/CIRGenModule.h b/clang/lib/CIR/CodeGen/CIRGenModule.h index 073e8d96b773b..d8b6f90d37b39 100644 --- a/clang/lib/CIR/CodeGen/CIRGenModule.h +++ b/clang/lib/CIR/CodeGen/CIRGenModule.h @@ -270,6 +270,12 @@ class CIRGenModule : public CIRGenTypeCache { getAddrOfConstantStringFromLiteral(const StringLiteral *s, llvm::StringRef name = ".str"); + /// Returns the address space for temporary allocations in the language. This + /// ensures that the allocated variable's address space matches the + /// expectations of the AST, rather than using the target's allocation address + /// space, which may lead to type mismatches in other parts of the IR. + LangAS getLangTempAllocaAddressSpace() const; + /// Set attributes which are common to any form of a global definition (alias, /// Objective-C method, function, global variable). /// diff --git a/clang/lib/CIR/CodeGen/CIRGenTypes.cpp b/clang/lib/CIR/CodeGen/CIRGenTypes.cpp index e65896a9ff109..c35143238a0f1 100644 --- a/clang/lib/CIR/CodeGen/CIRGenTypes.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenTypes.cpp @@ -405,7 +405,7 @@ mlir::Type CIRGenTypes::convertType(QualType type) { const ReferenceType *refTy = cast<ReferenceType>(ty); QualType elemTy = refTy->getPointeeType(); auto pointeeType = convertTypeForMem(elemTy); - resultType = builder.getPointerTo(pointeeType); + resultType = builder.getPointerTo(pointeeType, elemTy.getAddressSpace()); assert(resultType && "Cannot get pointer type?"); break; } diff --git a/clang/lib/CIR/CodeGen/TargetInfo.cpp b/clang/lib/CIR/CodeGen/TargetInfo.cpp index 62a8c59abe604..26caa4a576abb 100644 --- a/clang/lib/CIR/CodeGen/TargetInfo.cpp +++ b/clang/lib/CIR/CodeGen/TargetInfo.cpp @@ -1,5 +1,7 @@ #include "TargetInfo.h" #include "ABIInfo.h" +#include "CIRGenFunction.h" +#include "clang/CIR/Dialect/IR/CIRDialect.h" using namespace clang; using namespace clang::CIRGen; @@ -68,3 +70,14 @@ bool TargetCIRGenInfo::isNoProtoCallVariadic( // For everything else, we just prefer false unless we opt out. return false; } + +mlir::Value TargetCIRGenInfo::performAddrSpaceCast( + CIRGenFunction &cgf, mlir::Value src, cir::AddressSpace srcAS, + cir::AddressSpace destAS, mlir::Type destTy, bool isNonNull) const { + // Since target may map different address spaces in AST to the same address + // space, an address space conversion may end up as a bitcast. + if (cir::GlobalOp globalOp = src.getDefiningOp<cir::GlobalOp>()) + llvm_unreachable("Global ops addrspace cast NYI"); + // Try to preserve the source's name to make IR more readable. + return cgf.getBuilder().createAddrSpaceCast(src, destTy); +} \ No newline at end of file diff --git a/clang/lib/CIR/CodeGen/TargetInfo.h b/clang/lib/CIR/CodeGen/TargetInfo.h index 1c3ba0b9971b3..0ba12ef7bb1c8 100644 --- a/clang/lib/CIR/CodeGen/TargetInfo.h +++ b/clang/lib/CIR/CodeGen/TargetInfo.h @@ -45,6 +45,18 @@ class TargetCIRGenInfo { /// Returns ABI info helper for the target. const ABIInfo &getABIInfo() const { return *info; } + /// Perform address space cast of an expression of pointer type. + /// \param V is the value to be casted to another address space. + /// \param SrcAddr is the CIR address space of \p V. + /// \param DestAddr is the targeted CIR address space. + /// \param DestTy is the destination pointer type. + /// \param IsNonNull is the flag indicating \p V is known to be non null. + virtual mlir::Value performAddrSpaceCast(CIRGenFunction &cgf, mlir::Value v, + cir::AddressSpace srcAS, + cir::AddressSpace destAS, + mlir::Type destTy, + bool isNonNull = false) const; + /// Determine whether a call to an unprototyped functions under /// the given calling convention should use the variadic /// convention or the non-variadic convention. diff --git a/clang/test/CIR/address-space-conversion.cpp b/clang/test/CIR/address-space-conversion.cpp new file mode 100644 index 0000000000000..0f600e52d24da --- /dev/null +++ b/clang/test/CIR/address-space-conversion.cpp @@ -0,0 +1,68 @@ +// RUN: %clang_cc1 -std=c++17 -triple x86_64-unknown-linux-gnu -fclangir -emit-cir %s -o %t.cir +// RUN: FileCheck --input-file=%t.cir %s -check-prefix=CIR +// RUN: %clang_cc1 -std=c++17 -triple x86_64-unknown-linux-gnu -fclangir -emit-llvm %s -o %t.ll +// RUN: FileCheck --input-file=%t.ll %s -check-prefix=LLVM + +using pi1_t = int __attribute__((address_space(1))) *; +using pi2_t = int __attribute__((address_space(2))) *; + +using ri1_t = int __attribute__((address_space(1))) &; +using ri2_t = int __attribute__((address_space(2))) &; + +// CIR: cir.func dso_local @{{.*test_ptr.*}} +// LLVM: define dso_local void @{{.*test_ptr.*}} +void test_ptr() { + pi1_t ptr1; + pi2_t ptr2 = (pi2_t)ptr1; + // CIR: %[[#PTR1:]] = cir.load{{.*}} %{{[0-9]+}} : !cir.ptr<!cir.ptr<!s32 + // CIR-NEXT: %[[#CAST:]] = cir.cast(address_space, %[[#PTR1]] : !cir.ptr<!s32i, addrspace(target<1>)>), !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %[[#PTR1:]] = load ptr addrspace(1), ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: %[[#CAST:]] = addrspacecast ptr addrspace(1) %[[#PTR1]] to ptr addrspace(2) + // LLVM-NEXT: store ptr addrspace(2) %[[#CAST]], ptr %{{[0-9]+}}, align 8 +} + +// CIR: cir.func dso_local @{{.*test_ref.*}} +// LLVM: define dso_local void @{{.*test_ref.*}} +void test_ref() { + pi1_t ptr; + ri1_t ref1 = *ptr; + ri2_t ref2 = (ri2_t)ref1; + // CIR: %[[#DEREF:]] = cir.load deref{{.*}} %{{[0-9]+}} : !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>>, !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: cir.store{{.*}} %[[#DEREF]], %[[#ALLOCAREF1:]] : !cir.ptr<!s32i, addrspace(target<1>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>> + // CIR-NEXT: %[[#REF1:]] = cir.load{{.*}} %[[#ALLOCAREF1]] : !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>>, !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: %[[#CAST:]] = cir.cast(address_space, %[[#REF1]] : !cir.ptr<!s32i, addrspace(target<1>)>), !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %[[#DEREF:]] = load ptr addrspace(1), ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: store ptr addrspace(1) %[[#DEREF]], ptr %[[#ALLOCAREF1:]], align 8 + // LLVM-NEXT: %[[#REF1:]] = load ptr addrspace(1), ptr %[[#ALLOCAREF1]], align 8 + // LLVM-NEXT: %[[#CAST:]] = addrspacecast ptr addrspace(1) %[[#REF1]] to ptr addrspace(2) + // LLVM-NEXT: store ptr addrspace(2) %[[#CAST]], ptr %{{[0-9]+}}, align 8 +} + +// CIR: cir.func dso_local @{{.*test_nullptr.*}} +// LLVM: define dso_local void @{{.*test_nullptr.*}} +void test_nullptr() { + constexpr pi1_t null1 = nullptr; + pi2_t ptr = (pi2_t)null1; + // CIR: %[[#NULL1:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: cir.store{{.*}} %[[#NULL1]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<1>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>> + // CIR-NEXT: %[[#NULL2:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#NULL2]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: store ptr addrspace(1) null, ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: store ptr addrspace(2) null, ptr %{{[0-9]+}}, align 8 +} + +void test_side_effect(pi1_t b) { + pi2_t p = (pi2_t)(*b++, (int*)0); + // CIR: %{{[0-9]+}} = cir.ptr_stride(%{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<1>)>, %{{[0-9]+}} : !s32i), !cir.ptr<!s32i, addrspace(target<1>)> + // CIR: %[[#CAST:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %{{[0-9]+}} = getelementptr i32, ptr addrspace(1) %{{[0-9]+}}, i64 1 + // LLVM: store ptr addrspace(2) null, ptr %{{[0-9]+}}, align 8 + +} |
| @llvm/pr-subscribers-clangir Author: David Rivera (RiverDave) ChangesAdd support for address space conversions in CIR.
Full diff: https://github.com/llvm/llvm-project/pull/161212.diff 11 Files Affected:
diff --git a/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h b/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h index cef8624e65d57..bf4a9b8438982 100644 --- a/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h +++ b/clang/include/clang/CIR/Dialect/Builder/CIRBaseBuilder.h @@ -424,6 +424,15 @@ class CIRBaseBuilderTy : public mlir::OpBuilder { return createBitcast(src, getPointerTo(newPointeeTy)); } + mlir::Value createAddrSpaceCast(mlir::Location loc, mlir::Value src, + mlir::Type newTy) { + return createCast(loc, cir::CastKind::address_space, src, newTy); + } + + mlir::Value createAddrSpaceCast(mlir::Value src, mlir::Type newTy) { + return createAddrSpaceCast(src.getLoc(), src, newTy); + } + //===--------------------------------------------------------------------===// // Binary Operators //===--------------------------------------------------------------------===// diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp index cf17de144f4d9..95e392d860518 100644 --- a/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenBuiltin.cpp @@ -58,6 +58,24 @@ static RValue emitBuiltinBitOp(CIRGenFunction &cgf, const CallExpr *e, return RValue::get(result); } +// Initialize the alloca with the given size and alignment according to the lang +// opts. Supporting only the trivial non-initialization for now. +static void initializeAlloca(CIRGenFunction &CGF, + [[maybe_unused]] mlir::Value AllocaAddr, + [[maybe_unused]] mlir::Value Size, + [[maybe_unused]] CharUnits AlignmentInBytes) { + + switch (CGF.getLangOpts().getTrivialAutoVarInit()) { + case LangOptions::TrivialAutoVarInitKind::Uninitialized: + // Nothing to initialize. + return; + case LangOptions::TrivialAutoVarInitKind::Zero: + case LangOptions::TrivialAutoVarInitKind::Pattern: + assert(false && "unexpected trivial auto var init kind NYI"); + return; + } +} + RValue CIRGenFunction::emitRotate(const CallExpr *e, bool isRotateLeft) { mlir::Value input = emitScalarExpr(e->getArg(0)); mlir::Value amount = emitScalarExpr(e->getArg(1)); @@ -172,21 +190,8 @@ RValue CIRGenFunction::emitBuiltinExpr(const GlobalDecl &gd, unsigned builtinID, builder.getUInt8Ty(), "bi_alloca", suitableAlignmentInBytes, size); // Initialize the allocated buffer if required. - if (builtinID != Builtin::BI__builtin_alloca_uninitialized) { - // Initialize the alloca with the given size and alignment according to - // the lang opts. Only the trivial non-initialization is supported for - // now. - - switch (getLangOpts().getTrivialAutoVarInit()) { - case LangOptions::TrivialAutoVarInitKind::Uninitialized: - // Nothing to initialize. - break; - case LangOptions::TrivialAutoVarInitKind::Zero: - case LangOptions::TrivialAutoVarInitKind::Pattern: - cgm.errorNYI("trivial auto var init"); - break; - } - } + if (builtinID != Builtin::BI__builtin_alloca_uninitialized) + initializeAlloca(*this, allocaAddr, size, suitableAlignmentInBytes); // An alloca will always return a pointer to the alloca (stack) address // space. This address space need not be the same as the AST / Language @@ -194,6 +199,12 @@ RValue CIRGenFunction::emitBuiltinExpr(const GlobalDecl &gd, unsigned builtinID, // the AST level this is handled within CreateTempAlloca et al., but for the // builtin / dynamic alloca we have to handle it here. assert(!cir::MissingFeatures::addressSpace()); + cir::AddressSpace aas = getCIRAllocaAddressSpace(); + cir::AddressSpace eas = cir::toCIRAddressSpace( + e->getType()->getPointeeType().getAddressSpace()); + if (eas != aas) { + assert(false && "Non-default address space for alloca NYI"); + } // Bitcast the alloca to the expected type. return RValue::get( diff --git a/clang/lib/CIR/CodeGen/CIRGenExpr.cpp b/clang/lib/CIR/CodeGen/CIRGenExpr.cpp index 44626bbdd1dfa..f65d502a12318 100644 --- a/clang/lib/CIR/CodeGen/CIRGenExpr.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenExpr.cpp @@ -1190,7 +1190,19 @@ LValue CIRGenFunction::emitCastLValue(const CastExpr *e) { case CK_Dynamic: case CK_ToUnion: case CK_BaseToDerived: - case CK_AddressSpaceConversion: + case CK_AddressSpaceConversion: { + LValue lv = emitLValue(e->getSubExpr()); + QualType destTy = getContext().getPointerType(e->getType()); + cir::AddressSpace srcAS = + cir::toCIRAddressSpace(e->getSubExpr()->getType().getAddressSpace()); + cir::AddressSpace destAS = + cir::toCIRAddressSpace(e->getType().getAddressSpace()); + mlir::Value V = getTargetHooks().performAddrSpaceCast( + *this, lv.getPointer(), srcAS, destAS, convertType(destTy)); + return makeAddrLValue(Address(V, convertTypeForMem(e->getType()), + lv.getAddress().getAlignment()), + e->getType(), lv.getBaseInfo()); + } case CK_ObjCObjectLValueCast: case CK_VectorSplat: case CK_ConstructorConversion: @@ -2283,7 +2295,10 @@ Address CIRGenFunction::createTempAlloca(mlir::Type ty, CharUnits align, // be different from the type defined by the language. For example, // in C++ the auto variables are in the default address space. Therefore // cast alloca to the default address space when necessary. - assert(!cir::MissingFeatures::addressSpace()); + if (auto astAS = cir::toCIRAddressSpace(cgm.getLangTempAllocaAddressSpace()); + getCIRAllocaAddressSpace() != astAS) { + llvm_unreachable("Requires address space cast which is NYI"); + } return Address(v, ty, align); } diff --git a/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp b/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp index bd09d78cd0eb6..48ffaf9770e7c 100644 --- a/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenExprScalar.cpp @@ -87,6 +87,7 @@ class ScalarExprEmitter : public StmtVisitor<ScalarExprEmitter, mlir::Value> { //===--------------------------------------------------------------------===// // Utilities //===--------------------------------------------------------------------===// + mlir::Type convertType(QualType ty) { return cgf.convertType(ty); } mlir::Value emitComplexToScalarConversion(mlir::Location loc, mlir::Value value, CastKind kind, @@ -1862,6 +1863,27 @@ mlir::Value ScalarExprEmitter::VisitCastExpr(CastExpr *ce) { return cgf.getBuilder().createBitcast(cgf.getLoc(subExpr->getSourceRange()), src, dstTy); } + case CK_AddressSpaceConversion: { + Expr::EvalResult result; + if (subExpr->EvaluateAsRValue(result, cgf.getContext()) && + result.Val.isNullPointer()) { + // If E has side effect, it is emitted even if its final result is a + // null pointer. In that case, a DCE pass should be able to + // eliminate the useless instructions emitted during translating E. + if (result.HasSideEffects) + Visit(subExpr); + return cgf.cgm.emitNullConstant(destTy, + cgf.getLoc(subExpr->getExprLoc())); + } + // Since target may map different address spaces in AST to the same address + // space, an address space conversion may end up as a bitcast. + cir::AddressSpace srcAS = cir::toCIRAddressSpace( + subExpr->getType()->getPointeeType().getAddressSpace()); + cir::AddressSpace destAS = + cir::toCIRAddressSpace(destTy->getPointeeType().getAddressSpace()); + return cgf.cgm.getTargetCIRGenInfo().performAddrSpaceCast( + cgf, Visit(subExpr), srcAS, destAS, convertType(destTy)); + } case CK_AtomicToNonAtomic: { cgf.getCIRGenModule().errorNYI(subExpr->getSourceRange(), diff --git a/clang/lib/CIR/CodeGen/CIRGenFunction.h b/clang/lib/CIR/CodeGen/CIRGenFunction.h index 166435f9e7e9e..f1158c7488f14 100644 --- a/clang/lib/CIR/CodeGen/CIRGenFunction.h +++ b/clang/lib/CIR/CodeGen/CIRGenFunction.h @@ -160,6 +160,10 @@ class CIRGenFunction : public CIRGenTypeCache { const TargetInfo &getTarget() const { return cgm.getTarget(); } mlir::MLIRContext &getMLIRContext() { return cgm.getMLIRContext(); } + const TargetCIRGenInfo &getTargetHooks() const { + return cgm.getTargetCIRGenInfo(); + } + // --------------------- // Opaque value handling // --------------------- diff --git a/clang/lib/CIR/CodeGen/CIRGenModule.cpp b/clang/lib/CIR/CodeGen/CIRGenModule.cpp index eef23a0ebda7f..5ff7c6cf25f80 100644 --- a/clang/lib/CIR/CodeGen/CIRGenModule.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenModule.cpp @@ -1363,6 +1363,23 @@ CIRGenModule::getAddrOfConstantStringFromLiteral(const StringLiteral *s, return builder.getGlobalViewAttr(ptrTy, gv); } +// TODO(cir): this could be a common AST helper for both CIR and LLVM codegen. +LangAS CIRGenModule::getLangTempAllocaAddressSpace() const { + if (getLangOpts().OpenCL) + return LangAS::opencl_private; + + // For temporaries inside functions, CUDA treats them as normal variables. + // LangAS::cuda_device, on the other hand, is reserved for those variables + // explicitly marked with __device__. + if (getLangOpts().CUDAIsDevice) + return LangAS::Default; + + if (getLangOpts().SYCLIsDevice || + (getLangOpts().OpenMP && getLangOpts().OpenMPIsTargetDevice)) + llvm_unreachable("NYI"); + return LangAS::Default; +} + void CIRGenModule::emitExplicitCastExprType(const ExplicitCastExpr *e, CIRGenFunction *cgf) { if (cgf && e->getType()->isVariablyModifiedType()) diff --git a/clang/lib/CIR/CodeGen/CIRGenModule.h b/clang/lib/CIR/CodeGen/CIRGenModule.h index 073e8d96b773b..d8b6f90d37b39 100644 --- a/clang/lib/CIR/CodeGen/CIRGenModule.h +++ b/clang/lib/CIR/CodeGen/CIRGenModule.h @@ -270,6 +270,12 @@ class CIRGenModule : public CIRGenTypeCache { getAddrOfConstantStringFromLiteral(const StringLiteral *s, llvm::StringRef name = ".str"); + /// Returns the address space for temporary allocations in the language. This + /// ensures that the allocated variable's address space matches the + /// expectations of the AST, rather than using the target's allocation address + /// space, which may lead to type mismatches in other parts of the IR. + LangAS getLangTempAllocaAddressSpace() const; + /// Set attributes which are common to any form of a global definition (alias, /// Objective-C method, function, global variable). /// diff --git a/clang/lib/CIR/CodeGen/CIRGenTypes.cpp b/clang/lib/CIR/CodeGen/CIRGenTypes.cpp index e65896a9ff109..c35143238a0f1 100644 --- a/clang/lib/CIR/CodeGen/CIRGenTypes.cpp +++ b/clang/lib/CIR/CodeGen/CIRGenTypes.cpp @@ -405,7 +405,7 @@ mlir::Type CIRGenTypes::convertType(QualType type) { const ReferenceType *refTy = cast<ReferenceType>(ty); QualType elemTy = refTy->getPointeeType(); auto pointeeType = convertTypeForMem(elemTy); - resultType = builder.getPointerTo(pointeeType); + resultType = builder.getPointerTo(pointeeType, elemTy.getAddressSpace()); assert(resultType && "Cannot get pointer type?"); break; } diff --git a/clang/lib/CIR/CodeGen/TargetInfo.cpp b/clang/lib/CIR/CodeGen/TargetInfo.cpp index 62a8c59abe604..26caa4a576abb 100644 --- a/clang/lib/CIR/CodeGen/TargetInfo.cpp +++ b/clang/lib/CIR/CodeGen/TargetInfo.cpp @@ -1,5 +1,7 @@ #include "TargetInfo.h" #include "ABIInfo.h" +#include "CIRGenFunction.h" +#include "clang/CIR/Dialect/IR/CIRDialect.h" using namespace clang; using namespace clang::CIRGen; @@ -68,3 +70,14 @@ bool TargetCIRGenInfo::isNoProtoCallVariadic( // For everything else, we just prefer false unless we opt out. return false; } + +mlir::Value TargetCIRGenInfo::performAddrSpaceCast( + CIRGenFunction &cgf, mlir::Value src, cir::AddressSpace srcAS, + cir::AddressSpace destAS, mlir::Type destTy, bool isNonNull) const { + // Since target may map different address spaces in AST to the same address + // space, an address space conversion may end up as a bitcast. + if (cir::GlobalOp globalOp = src.getDefiningOp<cir::GlobalOp>()) + llvm_unreachable("Global ops addrspace cast NYI"); + // Try to preserve the source's name to make IR more readable. + return cgf.getBuilder().createAddrSpaceCast(src, destTy); +} \ No newline at end of file diff --git a/clang/lib/CIR/CodeGen/TargetInfo.h b/clang/lib/CIR/CodeGen/TargetInfo.h index 1c3ba0b9971b3..0ba12ef7bb1c8 100644 --- a/clang/lib/CIR/CodeGen/TargetInfo.h +++ b/clang/lib/CIR/CodeGen/TargetInfo.h @@ -45,6 +45,18 @@ class TargetCIRGenInfo { /// Returns ABI info helper for the target. const ABIInfo &getABIInfo() const { return *info; } + /// Perform address space cast of an expression of pointer type. + /// \param V is the value to be casted to another address space. + /// \param SrcAddr is the CIR address space of \p V. + /// \param DestAddr is the targeted CIR address space. + /// \param DestTy is the destination pointer type. + /// \param IsNonNull is the flag indicating \p V is known to be non null. + virtual mlir::Value performAddrSpaceCast(CIRGenFunction &cgf, mlir::Value v, + cir::AddressSpace srcAS, + cir::AddressSpace destAS, + mlir::Type destTy, + bool isNonNull = false) const; + /// Determine whether a call to an unprototyped functions under /// the given calling convention should use the variadic /// convention or the non-variadic convention. diff --git a/clang/test/CIR/address-space-conversion.cpp b/clang/test/CIR/address-space-conversion.cpp new file mode 100644 index 0000000000000..0f600e52d24da --- /dev/null +++ b/clang/test/CIR/address-space-conversion.cpp @@ -0,0 +1,68 @@ +// RUN: %clang_cc1 -std=c++17 -triple x86_64-unknown-linux-gnu -fclangir -emit-cir %s -o %t.cir +// RUN: FileCheck --input-file=%t.cir %s -check-prefix=CIR +// RUN: %clang_cc1 -std=c++17 -triple x86_64-unknown-linux-gnu -fclangir -emit-llvm %s -o %t.ll +// RUN: FileCheck --input-file=%t.ll %s -check-prefix=LLVM + +using pi1_t = int __attribute__((address_space(1))) *; +using pi2_t = int __attribute__((address_space(2))) *; + +using ri1_t = int __attribute__((address_space(1))) &; +using ri2_t = int __attribute__((address_space(2))) &; + +// CIR: cir.func dso_local @{{.*test_ptr.*}} +// LLVM: define dso_local void @{{.*test_ptr.*}} +void test_ptr() { + pi1_t ptr1; + pi2_t ptr2 = (pi2_t)ptr1; + // CIR: %[[#PTR1:]] = cir.load{{.*}} %{{[0-9]+}} : !cir.ptr<!cir.ptr<!s32 + // CIR-NEXT: %[[#CAST:]] = cir.cast(address_space, %[[#PTR1]] : !cir.ptr<!s32i, addrspace(target<1>)>), !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %[[#PTR1:]] = load ptr addrspace(1), ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: %[[#CAST:]] = addrspacecast ptr addrspace(1) %[[#PTR1]] to ptr addrspace(2) + // LLVM-NEXT: store ptr addrspace(2) %[[#CAST]], ptr %{{[0-9]+}}, align 8 +} + +// CIR: cir.func dso_local @{{.*test_ref.*}} +// LLVM: define dso_local void @{{.*test_ref.*}} +void test_ref() { + pi1_t ptr; + ri1_t ref1 = *ptr; + ri2_t ref2 = (ri2_t)ref1; + // CIR: %[[#DEREF:]] = cir.load deref{{.*}} %{{[0-9]+}} : !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>>, !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: cir.store{{.*}} %[[#DEREF]], %[[#ALLOCAREF1:]] : !cir.ptr<!s32i, addrspace(target<1>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>> + // CIR-NEXT: %[[#REF1:]] = cir.load{{.*}} %[[#ALLOCAREF1]] : !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>>, !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: %[[#CAST:]] = cir.cast(address_space, %[[#REF1]] : !cir.ptr<!s32i, addrspace(target<1>)>), !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %[[#DEREF:]] = load ptr addrspace(1), ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: store ptr addrspace(1) %[[#DEREF]], ptr %[[#ALLOCAREF1:]], align 8 + // LLVM-NEXT: %[[#REF1:]] = load ptr addrspace(1), ptr %[[#ALLOCAREF1]], align 8 + // LLVM-NEXT: %[[#CAST:]] = addrspacecast ptr addrspace(1) %[[#REF1]] to ptr addrspace(2) + // LLVM-NEXT: store ptr addrspace(2) %[[#CAST]], ptr %{{[0-9]+}}, align 8 +} + +// CIR: cir.func dso_local @{{.*test_nullptr.*}} +// LLVM: define dso_local void @{{.*test_nullptr.*}} +void test_nullptr() { + constexpr pi1_t null1 = nullptr; + pi2_t ptr = (pi2_t)null1; + // CIR: %[[#NULL1:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<1>)> + // CIR-NEXT: cir.store{{.*}} %[[#NULL1]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<1>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<1>)>> + // CIR-NEXT: %[[#NULL2:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#NULL2]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: store ptr addrspace(1) null, ptr %{{[0-9]+}}, align 8 + // LLVM-NEXT: store ptr addrspace(2) null, ptr %{{[0-9]+}}, align 8 +} + +void test_side_effect(pi1_t b) { + pi2_t p = (pi2_t)(*b++, (int*)0); + // CIR: %{{[0-9]+}} = cir.ptr_stride(%{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<1>)>, %{{[0-9]+}} : !s32i), !cir.ptr<!s32i, addrspace(target<1>)> + // CIR: %[[#CAST:]] = cir.const #cir.ptr<null> : !cir.ptr<!s32i, addrspace(target<2>)> + // CIR-NEXT: cir.store{{.*}} %[[#CAST]], %{{[0-9]+}} : !cir.ptr<!s32i, addrspace(target<2>)>, !cir.ptr<!cir.ptr<!s32i, addrspace(target<2>)>> + + // LLVM: %{{[0-9]+}} = getelementptr i32, ptr addrspace(1) %{{[0-9]+}}, i64 1 + // LLVM: store ptr addrspace(2) null, ptr %{{[0-9]+}}, align 8 + +} |
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch 2 times, most recently from 61674c7 to 765a44e Compare 
andykaylor reviewed Sep 29, 2025
bcardosolopes approved these changes Sep 29, 2025
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch 2 times, most recently from fc85a21 to df23476 Compare RiverDave force-pushed the users/riverdave/cir/addrspace-support-for-cir-ptr branch 2 times, most recently from 9cd2c54 to 533e322 Compare RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch 3 times, most recently from 2dc195f to a438d04 Compare | I updated this based on the recent feedback on #161028 I made a change tn the function:
|
Sounds legit, thanks! |
Base automatically changed from users/riverdave/cir/addrspace-support-for-cir-ptr to main October 4, 2025 01:40
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch from a438d04 to a6f709a Compare | cc @andykaylor. Before merging and later back-porting these recent changes, I'd like your approval first, thanks. |
andykaylor reviewed Oct 7, 2025
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch 3 times, most recently from 41f7f5c to 979c034 Compare andykaylor reviewed Oct 20, 2025
clang/lib/CIR/CodeGen/CIRGenExpr.cpp Outdated
| // cast alloca to the default address space when necessary. | ||
| assert(!cir::MissingFeatures::addressSpace()); | ||
| | ||
| LangAS allocaAS = cgm.getLangTempAllocaAddressSpace(); |
There was a problem hiding this comment.
I'm confused as to what's going on here, in part because there are so many variations of createTempAlloca and I'm having trouble seeing the correspondence between what we have in CIR and what happens in classic codegen. That's a mess that we might need to clean up later.
For now, I'm trying to understand why you're calling getLangTempAllocaAddressSpace() here. In classic codegen, as you note in the comment above, the destination address space always comes to this function via an explicit argument. Usually, I think, that comes from calling the function with a different signature that is a wrapper that adds LangAS::Default, so the line of code below seems right. But the alloca address space is coming from the alloca's type (via RawAddress).
There was a problem hiding this comment.
Thanks! I had utilized what we had in the incubator as reference and we didn't seem to be utilizing the alloca param as you just pointed out.
As a side note:
(I'll try to reason more on what we have in the incubator and whether if it make sense for it to be upstreamed. —perhaps that's the biggest thing I've learnt from upstreaming so far haha).
…erformAddrSpaceCast
… address more nits
RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch from e52c7ed to 7ce9725 Compare RiverDave force-pushed the users/riverdave/cir-addrspace-casting branch from 7ce9725 to 0b4a042 Compare Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
related: #160386
Add support for address space conversions in CIR.
createAddrSpaceCastmethods toCIRBaseBuilderTyto handle address space conversionsemitCastLValueandVisitCastExprperformAddrSpaceCastmethod toTargetCIRGenInfofor target-specific address space castinggetLangTempAllocaAddressSpacetoCIRGenModuleto get the language-specific address space for temporary allocationsaddress-space-conversion.cppto verify address space conversion functionality