<s> } if ( FunctionTargetID . isSramEccSupported ( ) && FunctionTargetID . getSramEccSetting ( ) != IsaInfo :: TargetIDSetting :: Any && FunctionTargetID . getSramEccSetting ( ) != getTargetStreamer ( ) -> getTargetID ( ) -> getSramEccSetting ( ) ) { OutContext . reportError ( { } , <STR_LIT> + Twine ( MF -> getName ( ) ) + <STR_LIT> ) ; return ; } if ( ! MFI . isEntryFunction ( ) ) return ; if ( ( STM . isMesaKernel ( F ) || isHsaAbiVersion2 ( getGlobalSTI ( ) ) ) && ( F . getCallingConv ( ) == CallingConv :: <ISA_LIT>_KERNEL || F . getCallingConv ( ) == CallingConv :: SPIR_KERNEL ) ) { amd_kernel_code_t KernelCode


<s> AsmPrinter :: emitFunctionEntryLabel ( ) ; return ; } const <ISA_LIT>MachineFunctionInfo * MFI = MF -> getInfo < <ISA_LIT>MachineFunctionInfo > ( ) ; const GCNSubtarget & STM = MF -> getSubtarget < GCNSubtarget > ( ) ; if ( MFI -> isEntryFunction ( ) && STM . isAmdHsaOrMesa ( MF -> getFunction (


<s> const Triple :: OSType OS = TM . getTargetTriple ( ) . getOS ( ) ; if ( OS == Triple :: AMDHSA || OS == Triple :: AMDPAL ) return ; MCSymbol * GVSym = getSymbol ( GV ) ; GVSym -> redefineIfPossible ( ) ; if ( GVSym -> isDefined ( ) || GVSym -> isVariable ( ) ) report_fatal_error ( <STR_LIT> + Twine ( GVSym -> getName ( ) ) + <STR_LIT> ) ; const DataLayout & DL = GV -> getParent ( ) -> getDataLayout ( ) ; uint64_t Size = DL . getTypeAllocSize ( GV -> getValueType ( ) ) ; Align Alignment = GV -> getAlign ( ) . getValueOr ( Align ( <NUM_LIT> ) ) ; emitVisibility ( GVSym , GV -> getVisibility ( ) , ! GV -> isDeclaration


<s> const MachineFrameInfo & MFI = MF . getFrameInfo ( ) ; MD -> setFunctionScratchSize ( MF , MFI . getStackSize ( ) ) ; MD -> setRsrc1 ( CallingConv :: <ISA_LIT>_CS , CurrentProgramInfo . getPGMRSrc1 ( CallingConv :: <ISA_LIT>_CS ) ) ; MD -> setRsrc2 ( CallingConv :: <ISA_LIT>_CS , CurrentProgramInfo . ComputePGMRSrc2 ) ; MD -> setFunctionLdsSize ( MF , CurrentProgramInfo . LDSSize ) ; MD -> setFunctionNumUsedVgprs ( MF , CurrentProgramInfo .


<s> MD -> setScratchSize ( CC , alignTo ( CurrentProgramInfo . ScratchSize , <NUM_LIT> ) ) ; if ( MF . getFunction ( ) . getCallingConv ( ) == CallingConv :: <ISA_LIT>_PS ) { MD -> setRsrc2 ( CC , S_00B02C_EXTRA_LDS_<ISA_LIT>ZE ( CurrentProgramInfo . LDSBlocks ) ) ; MD -> setSpiPsInputEna ( MFI -> getP<ISA_LIT>nputEnable ( ) ) ; MD -> setSpiPsInputAddr ( MFI -> getP<ISA_LIT>nputAddr ( ) ) ; } const GCNSubtarget & STM = MF . getSubtarget <


<s> if ( MF . getFunction ( ) . getCallingConv ( ) == CallingConv :: <ISA_LIT>_PS ) { OutStreamer -> emitInt32 ( R_00B02C_SPI_SHADER_PGM_RSRC2_PS ) ; OutStreamer -> emitInt32 ( S_00B02C_EXTRA_LDS_<ISA_LIT>ZE ( CurrentProgramInfo . LDSBlocks ) ) ; OutStreamer -> emitInt32 ( R_0286CC_SPI_PS_INPUT_ENA ) ; OutStreamer -> emitInt32 ( MFI -> getP<ISA_LIT>nputEnable ( ) ) ; OutStreamer -> emitInt32 ( R_0286D0_SPI_PS_INPUT_ADDR ) ; OutStreamer -> emitInt32 ( MFI -> getP<ISA_LIT>nputAddr ( ) ) ; } OutStreamer -> emitInt32 ( R_SPILLED_SGPRS ) ; OutStreamer -> emitInt32 ( MFI ->


<s> if ( TM . getTargetTriple ( ) . getOS ( ) == Triple :: AMDPAL ) getTargetStreamer ( ) -> getPALMetadata ( ) -> readFromIR ( M ) ; if ( isHsaAbiVersion3Or4 ( getGlobalSTI ( ) ) ) return ; if ( TM . getTargetTriple ( ) . getOS ( ) == Triple :: AMDHSA ) getTargetStreamer ( ) -> EmitDirectiveHSACodeObjectVersion ( <NUM_LIT> , <NUM_LIT> ) ; IsaVersion Version = getIsaVersion ( getGlobalSTI ( )


<s> memset ( & KernelDescriptor , <NUM_LIT> , sizeof ( KernelDescriptor ) ) ; assert ( isUInt < <NUM_LIT> > ( PI . ScratchSize ) ) ; assert ( isUInt < <NUM_LIT> > ( PI . getComputePGMRSrc1 ( ) ) ) ; assert ( isUInt < <NUM_LIT> > ( PI . ComputePGMRSrc2 ) ) ; KernelDescriptor . group_segment_fixed_size = PI . LDSSize ; KernelDescriptor . private_segment_fixed_size = PI . ScratchSize ; Align MaxKernArgAlign ; KernelDescriptor . kernarg_size = STM . getKernArgSegmentSize ( F , MaxKernArgAlign ) ; KernelDescriptor . compute_pgm_rsrc1 = PI . getComputePGMRSrc1 ( ) ; KernelDescriptor . compute_pgm_rsrc2 = PI . ComputePGMRSrc2 ; KernelDescriptor . kernel_code_properties = getAmdhsaKernelCodeProperties ( MF ) ; assert ( STM . hasGFX90AInsts ( ) || CurrentProgramInfo . ComputePGMRSrc3GFX90A == <NUM_LIT> ) ; if ( STM . hasGFX90AInsts ( ) ) KernelDescriptor . compute_pgm_rsrc3 =

CurrentProgramInfo . ComputePGMRSrc3GFX90A ;


<s> AU . addRequired < <ISA_LIT>ResourceUsageAnalysis > ( ) ; AU . addPreserved < <ISA_LIT>ResourceUsageAnalysis > (


<s> static amd_element_byte_size_t getElementByteSizeValue ( unsigned Size ) { switch ( Size ) { case <NUM_LIT>


<s> uint64_t <ISA_LIT>AsmPrinter :: getFunctionCodeSize ( const MachineFunction & MF ) const { const GCNSubtarget & STM = MF . getSubtarget < GCNSubtarget > ( ) ; const <ISA_LIT>InstrInfo * TII = STM . getInstrInfo ( ) ; uint64_t CodeSize = <NUM_LIT> ; for ( const MachineBasicBlock & MBB : MF ) { for ( const MachineInstr & MI : MBB ) { if ( MI . isDebugInstr ( ) ) continue ; CodeSize += TII -> getInstSizeInBytes ( MI ) ; } } return CodeSize


<s> StringRef <ISA_LIT>AsmPrinter :: getPassName ( ) const { return <STR_LIT>


<s> default : LLVM_FALLTHROUGH ; case CallingConv :: <ISA_LIT>_CS : return R_00B848_COMPUTE_PGM_RSRC1 ; case CallingConv :: <ISA_LIT>_LS : return R_00B528_SPI_SHADER_PGM_RSRC1_LS ; case CallingConv :: <ISA_LIT>_HS : return R_00B428_SPI_SHADER_PGM_RSRC1_HS ; case CallingConv :: <ISA_LIT>_ES : return


<s> case CallingConv :: <ISA_LIT>_CS : return R_00B848_COMPUTE_PGM_RSRC1 ; case CallingConv :: <ISA_LIT>_LS : return R_00B528_SPI_SHADER_PGM_RSRC1_LS ; case CallingConv :: <ISA_LIT>_HS : return R_00B428_SPI_SHADER_PGM_RSRC1_HS ; case CallingConv :: <ISA_LIT>_ES : return R_00B328_SPI_SHADER_PGM_RSRC1_ES ; case CallingConv :: <ISA_LIT>_GS : return R_00B228_SPI_SHADER_PGM_RSRC1_GS ; case CallingConv :: <ISA_LIT>_VS : return R_00B128_SPI_SHADER_PGM_RSRC1_VS ; case CallingConv :: <ISA_LIT>_PS : return


<s> ProgInfo . NumSGPR = Info . NumExplicitSGPR ; ProgInfo . ScratchSize = Info . PrivateSegmentSize ; ProgInfo . VCCUsed = Info . UsesVCC ; ProgInfo . FlatUsed = Info . UsesFlatScratch ; ProgInfo . DynamicCallStack = Info . HasDynamicallySizedStack || Info . HasRecursion ; const uint64_t MaxScratchPerWorkitem = GCNSubtarget :: MaxWaveScratchSize / STM . getWavefrontSize ( ) ; if ( ProgInfo . ScratchSize > MaxScratchPerWorkitem ) { DiagnosticInfoStackSize DiagStackSize ( MF . getFunction ( ) , ProgInfo . ScratchSize , MaxScratchPerWorkitem , DS_Error ) ; MF . getFunction ( ) . getContext ( ) . diagnose ( DiagStackSize ) ; } const <ISA_LIT>MachineFunctionInfo * MFI = MF . getInfo < <ISA_LIT>MachineFunctionInfo > ( ) ; unsigned ExtraSGPRs = IsaInfo :: getNumExtraSGPRs ( & STM , ProgInfo . VCCUsed , ProgInfo . FlatUsed ) ; if ( STM . getGeneration ( ) >= <ISA_LIT>Subtarget :: VOLCANIC_ISLANDS && ! STM . hasSGPRInitBug ( ) ) { unsigned MaxAddressableNumSGPRs = STM . getAddressableNumSGPRs ( ) ; if ( ProgInfo . NumSGPR > MaxAddressableNumSGPRs ) { LLVMContext & Ctx = MF . getFunction ( ) . getContext ( ) ; DiagnosticInfoResourceLimit Diag ( MF . getFunction ( ) , <STR_LIT> , ProgInfo . NumSGPR , MaxAddressableNumSGPRs , DS_Error , DK_ResourceLimit ) ; Ctx . diagnose ( Diag ) ; ProgInfo . NumSGPR = MaxAddressableNumSGPRs - <NUM_LIT> ; } } ProgInfo . NumSGPR += ExtraSGPRs ; const Function & F = MF . getFunction ( ) ; unsigned WaveDispatchNumSGPR = <NUM_LIT> , WaveDispatchNumVGPR = <NUM_LIT> ; if ( isShader ( F . getCallingConv ( ) ) ) { bool IsPixelShader = F . getCallingConv ( ) == CallingConv :: <ISA_LIT>_PS && ! STM . isAmdHsaOS ( ) ; uint32_t InputEna = <NUM_LIT> ; uint32_t InputAddr = <NUM_LIT> ; unsigned LastEna = <NUM_LIT> ; if ( IsPixelShader ) { InputEna = MFI -> getP<ISA_LIT>nputEnable ( ) ; InputAddr = MFI -> getP<ISA_LIT>nputAddr ( ) ; assert ( ( InputEna || InputAddr ) && <STR_LIT> <STR_LIT> ) ; LastEna = InputEna ? findLastSet ( InputEna ) + <NUM_LIT> : <NUM_LIT> ; } const DataLayout & DL = F . getParent ( ) -> getDataLayout ( ) ; unsigned PSArgCount = <NUM_LIT> ; unsigned IntermediateVGPR = <NUM_LIT> ; for ( auto & Arg : F . args ( ) ) { unsigned NumRegs = ( DL . getTypeSizeInBits ( Arg . getType ( ) ) + <NUM_LIT> ) / <NUM_LIT> ; if ( Arg . hasAttribute ( Attribute :: InReg ) ) { WaveDispatchNumSGPR += NumRegs ; } else { if ( IsPixelShader && PSArgCount < <NUM_LIT>


<s> } } ProgInfo . NumSGPR = std :: max ( ProgInfo . NumSGPR , WaveDispatchNumSGPR ) ; ProgInfo . NumArchVGPR = std :: max ( ProgInfo . NumVGPR , WaveDispatchNumVGPR ) ; ProgInfo . NumVGPR = Info . getTotalNumVGPRs ( STM , Info . NumAGPR , ProgInfo . NumArchVGPR ) ; } ProgInfo . NumSGPRsForWavesPerEU = std :: max ( std :: max ( ProgInfo . NumSGPR , <NUM_LIT> ) , STM . getMinNumSGPRs ( MFI -> getMaxWavesPerEU ( ) ) ) ; ProgInfo . NumVGPRsForWavesPerEU = std :: max ( std :: max ( ProgInfo . NumVGPR , <NUM_LIT> ) , STM . getMinNumVGPRs ( MFI -> getMaxWavesPerEU ( ) ) ) ; if ( STM . getGeneration ( ) <= <ISA_LIT>Subtarget :: SEA_ISLANDS || STM . hasSGPRInitBug ( ) ) { unsigned MaxAddressableNumSGPRs = STM . getAddressableNumSGPRs ( ) ; if ( ProgInfo . NumSGPR > MaxAddressableNumSGPRs ) { LLVMContext & Ctx = MF . getFunction ( ) . getContext ( ) ; DiagnosticInfoResourceLimit Diag ( MF . getFunction ( ) , <STR_LIT> , ProgInfo . NumSGPR , MaxAddressableNumSGPRs , DS_Error , DK_ResourceLimit ) ; Ctx . diagnose ( Diag ) ; ProgInfo . NumSGPR = MaxAddressableNumSGPRs ; ProgInfo . NumSGPRsForWavesPerEU = MaxAddressableNumSGPRs ; } } if ( STM . hasSGPRInitBug ( ) ) { ProgInfo . NumSGPR = <STR_LIT> :: FIXED_NUM_SGPRS_FOR_INIT_BUG ; ProgInfo . NumSGPRsForWavesPerEU = <STR_LIT> :: FIXED_NUM_SGPRS_FOR_INIT_BUG ; } if ( MFI -> getNumUserSGPRs ( ) > STM . getMaxNumUserSGPRs ( ) ) { LLVMContext & Ctx = MF . getFunction ( ) . getContext ( ) ; DiagnosticInfoResourceLimit Diag ( MF . getFunction ( ) , <STR_LIT> , MFI -> getNumUserSGPRs ( ) , STM . getMaxNumUserSGPRs ( ) , DS_Error ) ; Ctx . diagnose ( Diag ) ; } if ( MFI -> getLDSSize ( ) > static_cast < unsigned > ( STM . getLocalMemorySize ( ) ) ) { LLVMContext & Ctx = MF . getFunction ( ) . getContext ( ) ; DiagnosticInfoResourceLimit Diag ( MF . getFunction ( ) , <STR_LIT> , MFI -> getLDSSize ( ) , STM . getLocalMemorySize ( ) , DS_Error ) ; Ctx . diagnose ( Diag ) ; } ProgInfo . SGPRBlocks = IsaInfo :: getNumSGPRBlocks ( & STM , ProgInfo . NumSGPRsForWavesPerEU ) ; ProgInfo . VGPRBlocks = IsaInfo :: getNumVGPRBlocks ( & STM , ProgInfo . NumVGPRsForWavesPerEU ) ; const <ISA_LIT>ModeRegisterDefaults Mode = MFI -> getMode ( ) ; ProgInfo . FloatMode = getFPMode ( Mode ) ; ProgInfo . IEEEMode = Mode . IEEE ; ProgInfo . DX10Clamp = Mode . DX10Clamp ; unsigned LDSAlignShift ; if ( STM . getGeneration ( ) < <ISA_LIT>Subtarget :: SEA_ISLANDS ) { LDSAlignShift = <NUM_LIT> ; } else { LDSAlignShift = <NUM_LIT> ; } unsigned LDSSpillSize = MFI -> getLDSWaveSpillSize (

) * MFI -> getMaxFlatWorkGroupSize ( ) ;


<s> auto & TSTargetID = getTargetStreamer ( ) -> getTargetID ( ) ; if ( ( ! TSTargetID -> isXnackSupported ( ) || TSTargetID -> isXnackOnOrOff ( ) ) && ( ! TSTargetID -> isSramEccSupported ( ) || TSTargetID -> isSramEccOnOrOff ( ) ) ) break ; const GCNSubtarget & STM = TM . getSubtarget < GCNSubtarget > ( F ) ; const IsaInfo :: <ISA_LIT>TargetID & STMTargetID = STM . getTargetID ( ) ; if ( TSTargetID -> isXnackSupported ( ) ) if ( TSTargetID -> getXnackSetting ( ) == IsaInfo :: TargetIDSetting ::

Any ) TSTargetID -> setXnackSetting ( STMTargetID . getXnackSetting ( ) ) ;


<s> if ( ! AsmPrinter :: isBlockOnlyReachableByFallthrough ( MBB ) ) return false ; if ( MBB -> empty ( ) ) return true ; return ( MBB -> back ( ) . getOpcode ( )


<s> void LLVM_EXTERNAL_VI<ISA_LIT>BILITY LLVMInitialize<ISA_LIT>AsmPrinter ( ) { TargetRegistry :: RegisterAsmPrinter ( getThe<ISA_LIT>Target ( ) , llvm :: create<ISA_LIT>AsmPrinterPass ) ; TargetRegistry :: RegisterAsmPrinter ( getTheGCNTarget ( )


<s> break ; default : return true ; } } const MachineOperand & MO = MI -> getOperand ( OpNo ) ; if ( MO . isReg ( ) ) { <ISA_LIT>InstPrinter :: printRegOperand ( MO . getReg ( ) , O , * MF -> getSubtarget ( ) . getRegisterInfo ( ) ) ; return false ; } else if ( MO . isImm ( ) ) { int64_t Val = MO . getImm ( ) ; if ( <STR_LIT> ( Val ) ) { O << Val ; } else if ( isUInt < <NUM_LIT> > ( Val ) ) { O << format ( <STR_LIT> PRIx16 , static_cast < uint16_t > ( Val ) ) ; } else if ( isUInt < <NUM_LIT> > ( Val ) ) { O << format ( <STR_LIT> PRIx32 , static_cast < uint32_t > (


<s> return true ; } } const MachineOperand & MO = MI -> getOperand ( OpNo ) ; if ( MO . isReg ( ) ) { <ISA_LIT>InstPrinter :: printRegOperand ( MO . getReg ( ) , O , * MF -> getSubtarget ( ) . getRegisterInfo ( ) ) ; return false ; } else if ( MO . isImm ( ) ) { int64_t Val = MO . getImm ( ) ; if ( <STR_LIT> (


<s> unsigned decode ( unsigned Code ) const { return ( Code >> Shift )


<s> unsigned encode ( unsigned Val ) const { return ( Val & Mask


<s> bool isSupported ( const MCSubtargetInfo &


<s> Function * UpdateDPP = <STR_LIT> ( M , <STR_LIT> , Ty ) ; Function * PermLaneX16 = <STR_LIT> ( M , <STR_LIT> , { } ) ; Function * ReadLane = <STR_LIT> ( M , <STR_LIT> , { } ) ; for ( unsigned Idx = <NUM_LIT> ; Idx < <NUM_LIT> ; Idx ++ ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: ROW_SHR0 | <NUM_LIT> << Idx ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } if ( ST -> hasDPPBroadcasts ( ) ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST15 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST31 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } else { Value * const PermX = B . CreateCall ( PermLaneX16 , { V , V , B . getInt32 ( - <NUM_LIT> ) , B . getInt32 ( - <NUM_LIT> ) , B . getFalse ( ) , B . getFalse ( ) } ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , PermX , B . getInt32 ( DPP :: QUAD_PERM_ID ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; if ( ! ST -> isWave32


<s> Value * <ISA_LIT>AtomicOptimizer :: buildScan ( IRBuilder < > & B , AtomicRMWInst :: BinOp Op , Value * V , Value * const Identity ) const { Type * const Ty = V -> getType ( ) ; Module * M = B . GetInsertBlock ( ) -> getModule ( ) ; Function * UpdateDPP = <STR_LIT> ( M , <STR_LIT> , Ty ) ; Function * PermLaneX16 = <STR_LIT> ( M , <STR_LIT> , { } ) ; Function * ReadLane = <STR_LIT> ( M , <STR_LIT> , { } ) ; for ( unsigned Idx = <NUM_LIT> ; Idx < <NUM_LIT> ; Idx ++ ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: ROW_SHR0 | <NUM_LIT> << Idx ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } if ( ST -> hasDPPBroadcasts ( ) ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST15 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST31 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } else { Value * const PermX = B . CreateCall ( PermLaneX16 , { V , V , B . getInt32 ( - <NUM_LIT> ) , B . getInt32 ( - <NUM_LIT> ) , B . getFalse ( ) , B . getFalse ( ) } ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , PermX , B . getInt32 ( DPP :: QUAD_PERM_ID ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; if ( ! ST -> isWave32 ( ) ) { Value * const Lane31 = B . CreateCall ( ReadLane , { V , B . getInt32 ( <NUM_LIT> ) } ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , Lane31 , B . getInt32 ( DPP :: QUAD_PERM_ID ) , B . getInt32 (


<s> Value * Old = V ; V = B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: ROW_SHR0 + <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ; V = B . CreateCall ( WriteLane , { B . CreateCall ( ReadLane , { Old , B . getInt32 ( <NUM_LIT> ) } ) , B . getInt32 ( <NUM_LIT> ) , V } ) ; if ( ! ST -> isWave32 ( ) ) { V = B . CreateCall ( WriteLane , { B . CreateCall ( ReadLane , { Old , B . getInt32 ( <NUM_LIT> ) } ) ,


<s> DominatorTreeWrapperPass * const DTW = getAnalysisIfAvailable < DominatorTreeWrapperPass > ( ) ; DT = DTW ? & DTW -> getDomTree ( ) : nullptr ; const TargetPassConfig & TPC = getAnalysis < TargetPassConfig > ( ) ; const TargetMachine & TM = TPC . getTM < TargetMachine > ( ) ; ST = & TM . getSubtarget < GCNSubtarget > ( F ) ; IsPixelShader = F . getCallingConv ( ) == CallingConv :: <ISA_LIT>_PS ; visit (


<s> const bool ValDivergent = DA -> isDivergentUse ( & I . getOperandUse ( ValIdx ) ) ; if ( ValDivergent && ( ! ST -> hasDPP ( ) || DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) { return ; } const ReplacementInfo Info = { & I , Op , ValIdx , ValDivergent } ; ToReplace . push_back ( Info


<s> break ; } const unsigned PtrIdx = <NUM_LIT> ; const unsigned ValIdx = <NUM_LIT> ; if ( DA -> isDivergentUse ( & I . getOperandUse ( PtrIdx ) ) ) { return ; } const bool ValDivergent = DA -> isDivergentUse ( & I . getOperandUse ( ValIdx ) ) ; if ( ValDivergent && ( ! ST -> hasDPP ( ) || DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) { return ; } const ReplacementInfo Info = { & I , Op , ValIdx , ValDivergent } ; ToReplace . push_back ( Info


<s> switch ( I . getIntrinsicID ( ) ) { default : return ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Add ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Sub ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: And ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Or ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Xor ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Min ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: UMin ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Max ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: UMax ; break ; } const unsigned ValIdx = <NUM_LIT> ; const bool ValDivergent = DA -> isDivergentUse ( & I . getOperandUse ( ValIdx ) ) ; if ( ValDivergent && ( ! ST -> hasDPP ( ) || DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) { return ; } for ( unsigned Idx = <NUM_LIT> ; Idx < I . getNumOperands ( ) ; Idx ++ ) { if ( DA -> isDivergentUse ( & I . getOperandUse ( Idx ) ) ) { return ; } } const ReplacementInfo Info = { & I


<s> void <ISA_LIT>AtomicOptimizer :: setConvergent ( CallInst * const CI ) const { CI -> addAttribute ( AttributeList :: FunctionIndex


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = Instruction :: Add ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = Instruction :: Sub ; break ; } const unsigned ValIdx = <NUM_LIT> ; const bool ValDivergent = DA -> isDivergent ( I . getOperand ( ValIdx ) ) ; if ( ValDivergent && ( ! HasDPP || ( DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) ) { return ; } for ( unsigned Idx = <NUM_LIT> ; Idx < I . getNumOperands ( ) ; Idx ++ ) { if ( DA -> isDivergent ( I .


<s> static Value * buildMul ( IRBuilder < > & B , Value *


<s> static Value * buildMul ( IRBuilder < > & B , Value * LHS , Value * RHS ) { const ConstantInt * CI = dyn_cast < ConstantInt


<s> Type * const Ty = V -> getType ( ) ; Module * M = B . GetInsertBlock ( ) -> getModule ( ) ; Function * UpdateDPP = <STR_LIT> ( M , <STR_LIT> , Ty ) ; for ( unsigned Idx = <NUM_LIT> ; Idx < <NUM_LIT> ; Idx ++ ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: ROW_SHR0 | <NUM_LIT> << Idx ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } if ( ST -> hasDPPBroadcasts ( ) ) { V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST15 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: BCAST31 ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ) ; } else { assert ( ST -> hasPermLaneX16 ( ) ) ; Value * const PermX = B . CreateIntrinsic ( <STR_LIT> , { } , { V , V , B . getInt32 ( - <NUM_LIT> ) , B . getInt32 ( - <NUM_LIT> ) , B . getFalse ( ) , B . getFalse ( ) } ) ; V = buildNonAtomicBinOp ( B , Op , V , B . CreateCall ( UpdateDPP , { Identity , PermX , B . getInt32 ( DPP :: QUAD_PERM_ID ) ,


<s> Function * WriteLane = <STR_LIT> ( M , <STR_LIT> , { } ) ; Value * Old = V ; V = B . CreateCall ( UpdateDPP , { Identity , V , B . getInt32 ( DPP :: ROW_SHR0 + <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getInt32 ( <NUM_LIT> ) , B . getFalse ( ) } ) ; V = B . CreateCall ( WriteLane , { B . CreateCall ( ReadLane , { Old , B . getInt32 ( <NUM_LIT> ) } ) , B . getInt32 ( <NUM_LIT> ) , V } ) ; if ( ! ST -> isWave32


<s> CmpInst :: Predicate Pred ; switch ( Op ) { default : llvm_unreachable ( <STR_LIT> ) ; case AtomicRMWInst :: Add : return B . CreateBinOp ( Instruction :: Add , LHS , RHS ) ; case AtomicRMWInst :: Sub : return B . CreateBinOp ( Instruction :: Sub , LHS , RHS ) ; case AtomicRMWInst :: Max : Pred = CmpInst


<s> static APInt getIdentityValueForAtomicOp ( AtomicRMWInst :: BinOp Op , unsigned BitWidth ) { switch ( Op ) { default : llvm_unreachable ( <STR_LIT> ) ; case AtomicRMWInst :: Add : case AtomicRMWInst :: Sub : case AtomicRMWInst ::


<s> llvm_unreachable ( <STR_LIT> ) ; case AtomicRMWInst :: Add : case AtomicRMWInst :: Sub : case AtomicRMWInst :: UMax : return APInt :: getMinValue ( BitWidth ) ; case AtomicRMWInst :: UMin : return APInt :: getMaxValue ( BitWidth ) ; case AtomicRMWInst :: Max : return APInt :: getSignedMinValue ( BitWidth ) ; case AtomicRMWInst :: Min : return APInt :: getSignedMaxValue


<s> AtomicRMWInst :: BinOp Op = I . getOperation ( ) ; switch ( Op ) { default : return ; case AtomicRMWInst :: Add : case AtomicRMWInst :: Sub : case AtomicRMWInst :: Max : case AtomicRMWInst :: Min : case AtomicRMWInst :: UMax : case AtomicRMWInst :: UMin : break ; } const unsigned PtrIdx = <NUM_LIT> ; const unsigned ValIdx = <NUM_LIT> ; if ( DA -> isDivergent ( I . getOperand ( PtrIdx ) ) ) { return ; } const bool ValDivergent = DA -> isDivergent ( I . getOperand ( ValIdx ) ) ; if ( ValDivergent && ( ! HasDPP || ( DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) ) { return ; } const ReplacementInfo Info = { & I , Op , ValIdx , ValDivergent } ; ToReplace . push_back ( Info


<s> void <ISA_LIT>AtomicOptimizer :: visitAtomicRMWInst ( AtomicRMWInst & I ) { switch ( I . getPointerAddressSpace ( ) ) { default : return ; case <ISA_LIT>AS :: GLOBAL_ADDRESS : case <ISA_LIT>AS :: LOCAL_ADDRESS : break ; } AtomicRMWInst :: BinOp Op = I . getOperation ( ) ; switch ( Op ) { default : return ; case AtomicRMWInst :: Add : case AtomicRMWInst :: Sub : case AtomicRMWInst :: And : case AtomicRMWInst :: Or : case AtomicRMWInst :: Xor : case AtomicRMWInst :: Max : case AtomicRMWInst :: Min : case AtomicRMWInst :: UMax : case AtomicRMWInst :: UMin : break ; } const unsigned PtrIdx = <NUM_LIT> ; const unsigned ValIdx = <NUM_LIT> ; if ( DA -> isDivergent ( I . getOperand ( PtrIdx


<s> void <ISA_LIT>AtomicOptimizer :: visitAtomicRMWInst ( AtomicRMWInst & I ) { switch ( I . getPointerAddressSpace ( ) ) { default : return ; case <ISA_LIT>AS :: GLOBAL_ADDRESS : case <ISA_LIT>AS :: LOCAL_ADDRESS : break ; } AtomicRMWInst :: BinOp Op = I . getOperation ( ) ; switch ( Op ) { default : return ; case AtomicRMWInst :: Add : case AtomicRMWInst ::


<s> break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Sub ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: And ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Or ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Xor ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Min ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: UMin ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Max ; break ; case <STR_LIT>


<s> DT = DTW ? & DTW -> getDomTree ( ) : nullptr ; const TargetPassConfig & TPC = getAnalysis < TargetPassConfig > ( ) ; const TargetMachine & TM = TPC . getTM < TargetMachine > ( ) ; const GCNSubtarget & ST = TM . getSubtarget < GCNSubtarget > ( F ) ; HasDPP = ST . hasDPP ( ) ; visit ( F ) ; const bool Changed = ! ToReplace . empty (


<s> DL = & F . getParent ( ) -> getDataLayout ( ) ; DominatorTreeWrapperPass * const DTW = getAnalysisIfAvailable < DominatorTreeWrapperPass > ( ) ; DT = DTW ? & DTW -> getDomTree ( ) : nullptr ; const TargetPassConfig & TPC = getAnalysis < TargetPassConfig > ( ) ; const TargetMachine & TM = TPC . getTM < TargetMachine > ( ) ; const GCNSubtarget & ST = TM . getSubtarget < GCNSubtarget > ( F ) ; HasDPP = ST . hasDPP (


<s> Instruction * const Ctpop = B . CreateUnaryIntrinsic ( <STR_LIT> , Ballot ) ; Value * const CtpopCast = B . CreateIntCast ( Ctpop , Ty , false ) ; NewV = B . CreateMul ( V , CtpopCast ) ; LaneOffset = B . CreateMul ( V , MbcntCast ) ; } Value * const Cond = B . CreateICmpEQ ( MbcntCast , B . getIntN ( TyBitWidth , <NUM_LIT> ) ) ; BasicBlock * const EntryBB = I . getParent ( ) ; Instruction * const SingleLaneTerminator = SplitBlockAndInsertIfThen ( Cond , & I , false , nullptr , DT , nullptr ) ; B . SetInsertPoint ( SingleLaneTerminator ) ; Instruction * const NewI = I . clone ( ) ; B . Insert ( NewI ) ; NewI -> setOperand ( ValIdx , NewV ) ; B . SetInsertPoint ( & I ) ; PHINode * const PHI = B . CreatePHI ( Ty , <NUM_LIT> ) ; PHI -> addIncoming ( UndefValue :: get ( Ty ) , EntryBB ) ; PHI -> addIncoming ( NewI , SingleLaneTerminator -> getParent ( ) ) ; Value * BroadcastI = nullptr ; if ( TyBitWidth == <NUM_LIT> ) { Value * const ExtractLo = B . CreateTrunc ( PHI , B . getInt32Ty ( ) ) ; Value * const ExtractHi = B . CreateTrunc ( B . CreateLShr ( PHI , B . getInt64 ( <NUM_LIT> ) ) , B . getInt32Ty ( ) ) ; CallInst * const ReadFirstLaneLo = B . CreateIntrinsic ( <STR_LIT> , { } , ExtractLo ) ; CallInst * const ReadFirstLaneHi = B . CreateIntrinsic ( <STR_LIT> , { } , ExtractHi ) ; Value * const PartialInsert = B . CreateInsertElement ( UndefValue :: get ( VecTy ) , ReadFirstLaneLo , B . getInt32 ( <NUM_LIT> ) ) ; Value * const Insert = B . CreateInsertElement ( PartialInsert , ReadFirstLaneHi , B . getInt32 ( <NUM_LIT> ) ) ; BroadcastI = B . CreateBitCast ( Insert , Ty ) ; } else if ( TyBitWidth == <NUM_LIT> ) { BroadcastI = B . CreateIntrinsic ( <STR_LIT> , { } , PHI ) ; } else { llvm_unreachable ( <STR_LIT> ) ; } Value * const Result = B . CreateBinOp ( Op , BroadcastI , LaneOffset ) ; if ( IsPixelShader ) { B . SetInsertPoint ( PixelExitBB -> getFirstNonPHI ( ) ) ; PHINode * const PHI = B . CreatePHI ( Ty


<s> const unsigned RowMask [ Iters ] = { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; const unsigned BankMask [ Iters ] = { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; for ( unsigned Idx = <NUM_LIT> ; Idx < Iters ; Idx ++ ) { Value * const UpdateValue = Idx < <NUM_LIT> ? FirstDPP : NewV ; CallInst * const DPP = B . CreateIntrinsic ( <STR_LIT> , Ty , { Identity , UpdateValue , B . getInt32 ( DPPCtrl [ Idx ] ) , B . getInt32 ( RowMask [ Idx ] ) , B . getInt32 ( BankMask [ Idx ] ) , B . getFalse ( ) } ) ; NewV = B . CreateBinOp ( Op , NewV , DPP ) ; } LaneOffset = B . CreateIntrinsic ( <STR_LIT> , Ty , NewV ) ; NewV = B . CreateBinOp ( Op , SetInactive , NewV ) ; if ( TyBitWidth == <NUM_LIT> ) { Value * const ExtractLo = B . CreateTrunc ( NewV , B . getInt32Ty ( ) ) ; Value * const ExtractHi = B . CreateTrunc ( B . CreateLShr ( NewV , B . getInt64 ( <NUM_LIT> ) ) , B . getInt32Ty ( ) ) ; CallInst * const ReadLaneLo = B . CreateIntrinsic ( <STR_LIT> , { } , { ExtractLo , B . getInt32 ( <NUM_LIT> ) } ) ; CallInst * const ReadLaneHi = B . CreateIntrinsic ( <STR_LIT> , { } , { ExtractHi , B . getInt32 ( <NUM_LIT> ) } ) ; Value * const PartialInsert = B . CreateInsertElement ( UndefValue :: get ( VecTy ) , ReadLaneLo , B . getInt32 ( <NUM_LIT> ) ) ; Value * const Insert = B . CreateInsertElement ( PartialInsert , ReadLaneHi , B . getInt32 ( <NUM_LIT> ) ) ; NewV = B . CreateBitCast ( Insert , Ty ) ; } else if ( TyBitWidth == <NUM_LIT> ) { CallInst * const ReadLane = B . CreateIntrinsic ( <STR_LIT> , { } , { NewV , B . getInt32 ( <NUM_LIT> ) } ) ; NewV = ReadLane ; } else { llvm_unreachable ( <STR_LIT> ) ; } NewV = B . CreateIntrinsic ( <STR_LIT> , Ty , NewV ) ; } else { Instruction * const Ctpop = B . CreateUnaryIntrinsic ( <STR_LIT> , Ballot ) ; Value * const CtpopCast = B . CreateIntCast ( Ctpop , Ty , false ) ; NewV = B . CreateMul ( V , CtpopCast ) ; LaneOffset = B . CreateMul ( V , MbcntCast ) ; } Value * const Cond = B . CreateICmpEQ ( MbcntCast ,

B . getIntN ( TyBitWidth , <NUM_LIT> ) ) ;


<s> static Value * buildNonAtomicBinOp ( IRBuilder < > & B , AtomicRMWInst :: BinOp Op , Value * LHS , Value * RHS ) { CmpInst :: Predicate Pred ; switch ( Op ) { default : llvm_unreachable ( <STR_LIT> ) ; case AtomicRMWInst :: Add : return B . CreateBinOp ( Instruction :: Add , LHS , RHS ) ; case AtomicRMWInst :: Sub : return B . CreateBinOp ( Instruction :: Sub , LHS , RHS ) ; case AtomicRMWInst :: And : return B . CreateBinOp ( Instruction :: And , LHS , RHS ) ; case AtomicRMWInst :: Or : return B . CreateBinOp ( Instruction :: Or , LHS , RHS ) ; case AtomicRMWInst :: Xor : return B . CreateBinOp ( Instruction :: Xor , LHS , RHS ) ; case AtomicRMWInst ::


<s> AU . addPreserved < DominatorTreeWrapperPass > ( ) ; AU . addRequired < LegacyDivergenceAnalysis > ( ) ; AU . addRequired < TargetPassConfig


<s> llvm_unreachable ( <STR_LIT> ) ; case AtomicRMWInst :: Add : case AtomicRMWInst :: Sub : case AtomicRMWInst :: Or : case AtomicRMWInst :: Xor : case AtomicRMWInst :: UMax : return APInt :: getMinValue ( BitWidth ) ; case AtomicRMWInst :: And : case AtomicRMWInst :: UMin : return APInt :: getMaxValue ( BitWidth ) ; case AtomicRMWInst :: Max : return APInt :: getSignedMinValue ( BitWidth ) ; case AtomicRMWInst ::


<s> return false ; } DA = & getAnalysis < LegacyDivergenceAnalysis > ( ) ; DL = & F . getParent ( ) -> getDataLayout ( ) ; DominatorTreeWrapperPass * const DTW = getAnalysisIfAvailable < DominatorTreeWrapperPass > ( ) ; DT = DTW ? & DTW -> getDomTree ( ) : nullptr ; const TargetPassConfig & TPC = getAnalysis < TargetPassConfig > ( ) ; const TargetMachine & TM = TPC . getTM < TargetMachine > ( ) ; const GCNSubtarget & ST = TM . getSubtarget < GCNSubtarget > ( F ) ; HasDPP = ST . hasDPP ( ) ; IsPixelShader = F . getCallingConv ( ) == CallingConv :: <ISA_LIT>_PS ; visit ( F ) ; const bool Changed = ! ToReplace . empty ( ) ; for ( ReplacementInfo & Info : ToReplace ) { optimizeAtomic ( * Info . I , Info . Op , Info . ValIdx , Info . ValDivergent ) ; } ToReplace . clear (


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Min ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: UMin ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: Max ; break ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : Op = AtomicRMWInst :: UMax ; break ; } const unsigned ValIdx = <NUM_LIT> ; const bool ValDivergent = DA -> isDivergentUse ( & I . getOperandUse ( ValIdx ) ) ; if ( ValDivergent && ( ! HasDPP || ( DL -> getTypeSizeInBits ( I . getType ( ) ) != <NUM_LIT> ) ) ) { return ; } for ( unsigned Idx = <NUM_LIT> ; Idx < I . getNumOperands ( ) ; Idx ++ ) { if ( DA -> isDivergentUse ( & I . getOperandUse


<s> bool IsNonEntryFunc = ! <STR_LIT> ( F -> getCallingConv ( ) ) ; auto & InfoCache = static_cast < <ISA_LIT>InformationCache & > ( A . getInfoCache ( ) ) ; bool NeedsQueuePtr = false ; auto CheckAddrSpaceCasts = [ & ] ( Instruction & I ) { unsigned SrcAS = static_cast < AddrSpaceCastInst & > ( I ) . getSrcAddressSpace ( ) ; if ( castRequiresQueuePtr ( SrcAS ) ) { NeedsQueuePtr = true ; return false ; } return true ; } ; bool HasApertureRegs = InfoCache . hasApertureRegs ( * F ) ; if ( ! HasApertureRegs ) { bool UsedAssumedInformation = false ; A . checkForAllInstructions ( CheckAddrSpaceCasts , * this , { Instruction :: AddrSpaceCast } , UsedAssumedInformation ) ; } if ( NeedsQueuePtr ) return true ; if ( ! IsNonEntryFunc && HasApertureRegs


<s> static bool funcRequiresHostcallPtr ( const Function & F ) { return F . hasFnAttribute ( Attribute :: SanitizeAddress ) || F . hasFnAttribute ( Attribute :: SanitizeThread ) || F . hasFnAttribute ( Attribute :: SanitizeMemory ) || F . hasFnAttribute ( Attribute :: SanitizeHWAddress ) || F . hasFnAttribute


<s> return F . hasFnAttribute ( Attribute :: SanitizeAddress ) || F . hasFnAttribute ( Attribute :: SanitizeThread ) || F . hasFnAttribute ( Attribute :: SanitizeMemory ) || F . hasFnAttribute (

Attribute :: SanitizeHWAddress ) || F . hasFnAttribute ( Attribute :: SanitizeMemTag ) ;


<s> bool funcRetrievesHostcallPtr ( Attributor & A ) { auto Pos = llvm :: <STR_LIT> ( ) ; AAPointerInfo :: OffsetAndSize OAS (


<s> auto & InfoCache = static_cast < <ISA_LIT>InformationCache & > ( A . getInfoCache ( ) ) ; unsigned MinGroupSize , MaxGroupSize ; std :: tie ( MinGroupSize , MaxGroupSize ) = InfoCache . getFlatWorkGroupSizes ( * F ) ; intersectKnown ( ConstantRange ( APInt ( <NUM_LIT> , MinGroupSize ) , APInt ( <NUM_LIT> , MaxGroupSize + <NUM_LIT>


<s> static ImplicitArgumentMask intrinsicToAttrMask ( <STR_LIT> ID , bool & NonKernelOnly , bool & NeedsImplicit , bool HasApertureRegs , bool SupportsGetDoorBellID ) { unsigned CodeObjectVersion = <STR_LIT> ( ) ; switch ( ID ) { case <STR_LIT> : NonKernelOnly = true ; return WORKITEM_ID_X ; case <STR_LIT> : NonKernelOnly = true ; return WORKGROUP_ID_X ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Z ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Z ; case <STR_LIT> : return DISPATCH_PTR ; case <STR_LIT> : return DISPATCH_ID ; case <STR_LIT> : return IMPLICIT_ARG_PTR ; case <STR_LIT> : NeedsImplicit = ( CodeObjectVersion == <NUM_LIT> ) ; return QUEUE_PTR ; case <STR_LIT> : case <STR_LIT> : if ( HasApertureRegs ) return NOT_IMPLICIT_INPUT ; return CodeObjectVersion == <NUM_LIT> ? IMPLICIT_ARG_PTR : QUEUE_PTR ; case <STR_LIT> : if ( SupportsGetDoorBellID ) return CodeObjectVersion >= <NUM_LIT> ? NOT_IMPLICIT_INPUT


<s> DenseSet < const char * > Allowed ( { & AAAMDAttributes :: ID , & AAUniformWorkGroupSize :: ID , & AAAMDFlatWorkGroupSize :: ID , & AACallEdges :: ID , & AAPointerInfo :: ID } ) ; Attributor A ( Functions , InfoCache , CGUpdater , & Allowed ) ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) { A . getOrCreateAAFor < AAAMDAttributes > ( IRPosition :: function ( F ) ) ; A . getOrCreateAAFor < AAUniformWorkGroupSize > ( IRPosition :: function ( F ) ) ; if ( ! <STR_LIT> ( F . getCallingConv (


<s> const GCNSubtarget & ST = TM . getSubtarget < GCNSubtarget > ( F ) ; return ST . supportsGetDoorbellID (


<s> AAPointerInfo :: OffsetAndSize OAS ( Pos , <NUM_LIT> ) ; return funcRetrievesImplicitKernelArg ( A


<s> auto Pos = llvm :: <STR_LIT> ( ) ; AAPointerInfo :: OffsetAndSize OAS ( Pos , <NUM_LIT> ) ; return funcRetrievesImplicitKernelArg ( A ,


<s> for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) Functions . insert ( & F ) ; } CallGraphUpdater CGUpdater ; BumpPtrAllocator Allocator ; <ISA_LIT>InformationCache InfoCache ( M , AG , Allocator , nullptr , * TM ) ; DenseSet < const char * > Allowed ( { & AAAMDAttributes :: ID , & AAUniformWorkGroupSize :: ID , & AAAMDFlatWorkGroupSize :: ID , & AACallEdges :: ID , & AAPointerInfo :: ID } ) ; AttributorConfig AC ( CGUpdater ) ; AC . Allowed = & Allowed ; AC . IsModulePass = true ; AC . DefaultInitializeLiveInternals = false ; Attributor A ( Functions , InfoCache , AC ) ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) { A . getOrCreateAAFor < AAAMDAttributes > ( IRPosition :: function ( F ) ) ; A . getOrCreateAAFor < AAUniformWorkGroupSize > ( IRPosition :: function ( F ) ) ; if ( ! <STR_LIT> ( F . getCallingConv ( ) ) ) { A . getOrCreateAAFor < AAAMDFlatWorkGroupSize > ( IRPosition


<s> ChangeStatus updateImpl ( Attributor & A ) override { Function * F = getAssociatedFunction ( ) ; auto OrigAssumed = getAssumed ( ) ; const AACallEdges & AAEdges = A . getAAFor < AACallEdges > ( * this , this -> getIRPosition ( ) , DepClassTy :: REQUIRED ) ; if ( AAEdges . hasNonAsmUnknownCallee ( ) ) return indicatePessimisticFixpoint ( ) ; bool IsNonEntryFunc = ! <STR_LIT> ( F -> getCallingConv ( ) ) ; bool NeedsImplicit = false ; auto & InfoCache = static_cast < <ISA_LIT>InformationCache & > ( A . getInfoCache ( ) ) ; bool HasApertureRegs = InfoCache . hasApertureRegs ( * F ) ; bool SupportsGetDoorbellID = InfoCache . supportsGetDoorbellID ( * F ) ; for ( Function * Callee : AAEdges . getOptimisticEdges ( ) ) { <STR_LIT> IID = Callee -> getIntrinsicID ( ) ; if ( IID == <STR_LIT> ) { const AAAMDAttributes & AAAMD = A . getAAFor < AAAMDAttributes > ( * this , IRPosition :: function ( * Callee ) , DepClassTy :: REQUIRED ) ; * this &= AAAMD ; continue ; } bool NonKernelOnly = false ; ImplicitArgumentMask AttrMask = intrinsicToAttrMask ( IID , NonKernelOnly , NeedsImplicit , HasApertureRegs , SupportsGetDoorbellID ) ; if ( AttrMask != NOT_IMPLICIT_INPUT ) { if ( ( IsNonEntryFunc || ! NonKernelOnly ) ) removeAssumedBits ( AttrMask ) ; } } if ( NeedsImplicit ) removeAssumedBits ( IMPLICIT_ARG_PTR ) ; if ( isAssumed ( QUEUE_PTR ) && checkForQueuePtr ( A ) ) { if ( <STR_LIT> ( ) == <NUM_LIT> ) removeAssumedBits ( IMPLICIT_ARG_PTR ) ; else removeAssumedBits ( QUEUE_PTR ) ; } if ( funcRetrievesMultigridSyncArg ( A ) ) { assert ( ! isAssumed ( IMPLICIT_ARG_PTR ) && <STR_LIT> ) ; removeAssumedBits ( MULTIGRID_SYNC_ARG ) ; } if ( funcRetrievesHostcallPtr ( A ) ) { assert ( ! isAssumed ( IMPLICIT_ARG_PTR ) && <STR_LIT> ) ; removeAssumedBits ( HOSTCALL_PTR


<s> if ( IRP . getPositionKind ( ) == IRPosition :: IRP_FUNCTION ) return * new ( A . Allocator ) AAAMDWorkGroupSizeFunction ( IRP , A ) ; llvm_unreachable ( <STR_LIT>


<s> const DenseSet < StringRef > & getAttributes ( ) const override { return


<s> const std :: string getName ( ) const override { return <STR_LIT>


<s> bool CallingConvSupportsAllImplicits = ( CC != CallingConv :: <ISA_LIT>_Gfx ) ; if ( <STR_LIT> ( F -> getCallingConv ( ) ) ) { indicatePessimisticFixpoint ( ) ; return ; } for ( StringRef Attr : ImplicitAttrNames ) { if ( F -> hasFnAttribute ( Attr ) ) Attributes . insert ( Attr ) ; } if ( CallingConvSupportsAllImplicits && F -> hasAddressTaken ( nullptr , true , true , true ) ) { for ( StringRef AttrName : ImplicitAttrNames ) { Attributes . insert (


<s> ChangeStatus manifest ( Attributor & A ) override { SmallVector < Attribute , <NUM_LIT> > AttrList ; LLVMContext & Ctx = getAssociatedFunction ( ) -> getContext ( ) ; for ( StringRef AttrName : Attributes ) AttrList . push_back ( Attribute :: get ( Ctx


<s> if ( IID != <STR_LIT> ) { if ( ! IsNonEntryFunc && IID == <STR_LIT> ) { AddAttribute ( <STR_LIT> ) ; continue ; } bool NonKernelOnly = false ; StringRef AttrName = intrinsicToAttrName ( IID , NonKernelOnly , NeedsQueuePtr ) ; if ( ! AttrName . empty ( ) && ( IsNonEntryFunc || ! NonKernelOnly ) ) AddAttribute ( AttrName ) ; continue ; } HasCall = true ; const AAAMDAttributes & AAAMD = A . getAAFor < AAAMDAttributes > ( * this , IRPosition :: function ( * Callee ) , DepClassTy :: REQUIRED ) ; const DenseSet < StringRef > & CalleeAttributes = AAAMD . getAttributes ( ) ; for ( StringRef AttrName : ImplicitAttrNames ) if ( CalleeAttributes . count ( AttrName ) ) AddAttribute ( AttrName ) ; } HasCall |= AAEdges . hasUnknownCallee ( ) ; if ( ! IsNonEntryFunc && HasCall ) AddAttribute ( <STR_LIT> ) ; auto CheckAlloca = [ & ] ( Instruction & I ) { AddAttribute ( <STR_LIT> ) ; return false ; } ; bool UsedAssumedInformation = false ; A . checkForAllInstructions ( CheckAlloca , * this , { Instruction :: Alloca } , UsedAssumedInformation ) ; if ( NeedsQueuePtr || Attributes . count ( <STR_LIT> ) ) { AddAttribute ( <STR_LIT> ) ; return Change ; } auto CheckAddrSpaceCasts = [ & ] ( Instruction & I ) { unsigned SrcAS = static_cast < AddrSpaceCastInst & > ( I ) . getSrcAddressSpace ( ) ; if ( castRequiresQueuePtr ( SrcAS ) ) { NeedsQueuePtr = true ; return false


<s> for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) Functions . insert ( & F ) ; } CallGraphUpdater CGUpdater ; BumpPtrAllocator Allocator ; <ISA_LIT>InformationCache InfoCache ( M , AG , Allocator , nullptr , * TM ) ; DenseSet < const char * > Allowed ( { & AAAMDAttributes :: ID , & AAAMDWorkGroupSize :: ID , & AACallEdges :: ID } ) ; Attributor A ( Functions , InfoCache , CGUpdater , & Allowed ) ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) { A . getOrCreateAAFor < AAAMDAttributes > ( IRPosition :: function ( F ) ) ; A . getOrCreateAAFor < AAAMDWorkGroupSize > ( IRPosition :: function ( F ) ) ; } } ChangeStatus Change = A . run ( ) ; return Change == ChangeStatus :: CHANGED


<s> indicatePessimisticFixpoint ( ) ; return ; } if ( <STR_LIT> ( F -> getCallingConv ( ) ) ) { indicatePessimisticFixpoint ( ) ; return ; } for ( StringRef Attr : ImplicitAttrNames ) { if ( F -> hasFnAttribute ( Attr ) ) Attributes . insert ( Attr ) ; } if ( CallingConvSupportsAllImplicits && F -> hasAddressTaken ( nullptr , true , true , true ) ) { for ( StringRef AttrName : ImplicitAttrNames ) { Attributes . insert ( AttrName


<s> Function * F = getAssociatedFunction ( ) ; const bool NeedsImplicit = funcRequiresImplicitArgPtr ( * F ) ; if ( NeedsImplicit ) removeAssumedBits ( IMPLICIT_ARG_PTR ) ; for ( auto Attr : ImplicitAttrs ) { if ( NeedsImplicit && Attr . first == IMPLICIT_ARG_PTR ) continue ; if ( F -> hasFnAttribute ( Attr . second ) ) addKnownBits ( Attr . first ) ; } if ( F -> isDeclaration ( ) ) return ; if ( <STR_LIT> ( F ->


<s> DenseSet < const char * > Allowed ( { & AAAMDAttributes :: ID , & AAUniformWorkGroupSize :: ID , & AACallEdges :: ID } ) ; Attributor A ( Functions , InfoCache , CGUpdater , & Allowed ) ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) { A . getOrCreateAAFor < AAAMDAttributes > ( IRPosition :: function ( F ) ) ; A . getOrCreateAAFor < AAUniformWorkGroupSize > ( IRPosition :: function (


<s> static bool classof ( const AbstractAttribute *


<s> static bool classof ( const AbstractAttribute * AA ) { return ( AA -> getIdAddr ( )


<s> AAAMDFlatWorkGroupSize & AAAMDFlatWorkGroupSize :: createForPosition ( const IRPosition & IRP , Attributor & A ) { if ( IRP . getPositionKind ( ) == IRPosition :: IRP_FUNCTION ) return * new ( A . Allocator


<s> AAAMDFlatWorkGroupSize & AAAMDFlatWorkGroupSize :: createForPosition ( const IRPosition & IRP , Attributor & A ) { if ( IRP . getPositionKind ( ) == IRPosition :: IRP_FUNCTION ) return * new ( A . Allocator ) AAAMDFlatWorkGroupSize ( IRP , A ) ; llvm_unreachable ( <STR_LIT>


<s> bool doInitialization ( Module & ) override { auto * TPC = getAnalysisIfAvailable < TargetPassConfig > (


<s> raw_string_ostream OS ( Str ) ; OS << <STR_LIT> ; OS << getAssumed ( ) . getLower ( ) << ',' << getAssumed ( ) . getUpper ( ) - <NUM_LIT> ; OS << ']' ; return OS .


<s> for ( const Use & U : C -> operands ( ) ) { const auto * OpC = dyn_cast < Constant > ( U ) ; if ( ! OpC ) continue ; Result |= getConstantAccess ( OpC ) ; } return


<s> std :: pair < unsigned , unsigned > getFlatWorkGroupSizes ( const Function & F ) { const GCNSubtarget & ST = TM . getSubtarget <


<s> return { ST . getMinFlatWorkGroupSize ( ) , ST . getMaxFlatWorkGroupSize (


<s> const GCNSubtarget & ST = TM . getSubtarget <


<s> void initialize ( Attributor & A ) override { Function * F = getAssociatedFunction ( ) ; auto & InfoCache = static_cast < <ISA_LIT>InformationCache & > ( A . getInfoCache ( ) ) ; unsigned MinGroupSize , MaxGroupSize ; std :: tie ( MinGroupSize , MaxGroupSize ) = InfoCache . getFlatWorkGroupSizes


<s> Function * F = getAssociatedFunction ( ) ; auto & InfoCache = static_cast < <ISA_LIT>InformationCache & > ( A . getInfoCache ( ) ) ; unsigned MinGroupSize , MaxGroupSize ; std :: tie ( MinGroupSize , MaxGroupSize ) = InfoCache . getFlatWorkGroupSizes ( * F ) ; intersectKnown ( ConstantRange ( APInt ( <NUM_LIT> , MinGroupSize ) ,

APInt ( <NUM_LIT> , MaxGroupSize + <NUM_LIT> ) ) ) ;


<s> return WORKITEM_ID_X ; case <STR_LIT> : NonKernelOnly = true ; return WORKGROUP_ID_X ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Z ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Z ; case <STR_LIT> : return DISPATCH_PTR ; case <STR_LIT> : return DISPATCH_ID ; case <STR_LIT> : return IMPLICIT_ARG_PTR ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case


<s> case <STR_LIT> : NonKernelOnly = true ; return WORKITEM_ID_X ; case <STR_LIT> : NonKernelOnly = true ; return WORKGROUP_ID_X ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKITEM_ID_Z ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Y ; case <STR_LIT> : case <STR_LIT> : return WORKGROUP_ID_Z ; case <STR_LIT> : return DISPATCH_PTR ; case <STR_LIT> : return DISPATCH_ID ; case <STR_LIT> : return IMPLICIT_ARG_PTR ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT>


<s> bool isValidState ( ) const override { return true


<s> OS << getAssumed ( ) . getLower ( ) << ',' << getAssumed ( ) . getUpper ( ) - <NUM_LIT> ; AttrList . push_back ( Attribute :: get ( Ctx , <STR_LIT> , OS . str ( ) ) ) ; return IRAttributeManifest :: manifestAttrs ( A , getIRPosition ( )


<s> bool runOnModule ( Module & M ) override { SetVector < Function * > Functions ; AnalysisGetter AG ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) Functions . insert ( & F ) ; } CallGraphUpdater CGUpdater ; BumpPtrAllocator Allocator ; <ISA_LIT>InformationCache InfoCache ( M , AG , Allocator , nullptr , * TM ) ; DenseSet < const char * > Allowed ( { & AAAMDAttributes :: ID , & AAUniformWorkGroupSize :: ID , & AAAMDFlatWorkGroupSize :: ID , & AACallEdges :: ID } ) ; Attributor A ( Functions , InfoCache , CGUpdater , & Allowed ) ; for ( Function & F : M ) { if ( ! F . isIntrinsic ( ) ) { A . getOrCreateAAFor < AAAMDAttributes > ( IRPosition


<s> ChangeStatus updateImpl ( Attributor & A ) override { ChangeStatus Change = ChangeStatus :: UNCHANGED ; auto CheckCallSite = [ & ] ( AbstractCallSite CS ) { Function * Caller = CS . getInstruction ( ) -> getFunction ( ) ; LLVM_DEBUG ( dbgs ( ) << <STR_LIT> << Caller -> getName ( ) << <STR_LIT> << getAssociatedFunction ( ) -> getName ( ) << '\n' ) ; const auto & CallerInfo = A . getAAFor < AAAMDFlatWorkGroupSize > ( * this , IRPosition :: function ( * Caller ) ,


<s> Function * Caller = CS . getInstruction ( ) -> getFunction ( ) ; LLVM_DEBUG ( dbgs ( ) << <STR_LIT> << Caller -> getName ( ) << <STR_LIT> << getAssociatedFunction ( ) -> getName ( ) << '\n' ) ; const auto & CallerInfo = A . getAAFor < AAAMDFlatWorkGroupSize > ( * this , IRPosition :: function ( * Caller ) , DepClassTy :: REQUIRED ) ; Change |= clampStateAndIndicateChange ( this -> getState ( ) , CallerInfo . getState ( ) ) ; return true ; } ; bool AllCallSitesKnown = true ; if ( ! A . checkForAllCallSites ( CheckCallSite , * this , true , AllCallSitesKnown )

) return indicatePessimisticFixpoint ( ) ;


<s> unsigned SrcAS = CE -> getOperand ( <NUM_LIT> ) -> getType ( ) -> getPointerAddressSpace ( ) ; return castRequiresQueuePtr ( SrcAS ) ; } return


<s> bool allFP64FP16Denormals ( ) const { return FP64FP16InputDenormals &&


<s> static Waitcnt allZeroExceptVsCnt ( ) { return Waitcnt ( <NUM_LIT> , <NUM_LIT> ,


<s> return Waitcnt ( std :: min ( VmCnt , Other . VmCnt ) , std :: min ( ExpCnt , Other . ExpCnt ) , std :: min ( LgkmCnt , Other . LgkmCnt ) , std :: min (


<s> bool dominates ( const Waitcnt & Other ) const { return VmCnt <= Other . VmCnt && ExpCnt <= Other . ExpCnt && LgkmCnt <= Other . LgkmCnt


<s> uint32_t fpDenormModeDPValue ( ) const { if ( FP64FP16InputDenormals && FP64FP16OutputDenormals ) return FP_DENORM_FLUSH_NONE


<s> if ( FP32OutputDenormals ) return FP_DENORM_FLUSH_IN ; return FP_DENORM_FLUSH_IN_FLUSH_OUT


<s> <ISA_LIT>ModeRegisterDefaults Mode ; Mode . IEEE = ! <STR_LIT> ( CC ) ; return


<s> TargetIDSetting getSramEccSetting ( ) const { return


<s> TargetIDSetting getXnackSetting ( ) const { return


<s> return VmCnt != ~ <NUM_LIT> || ExpCnt != ~ <NUM_LIT> || LgkmCnt != ~ <NUM_LIT>


<s> return Literal >= - <NUM_LIT> && Literal


<s> bool isInlineCompatible ( <ISA_LIT>ModeRegisterDefaults CalleeMode ) const { if ( DX10Clamp != CalleeMode . DX10Clamp ) return false ; if ( IEEE != CalleeMode . IEEE ) return false ; return oneWayCompatible ( FP64FP16InputDenormals , CalleeMode . FP64FP16InputDenormals ) && oneWayCompatible ( FP64FP16OutputDenormals , CalleeMode . FP64FP16OutputDenormals ) && oneWayCompatible ( FP32InputDenormals , CalleeMode

. FP32InputDenormals ) && oneWayCompatible ( FP32OutputDenormals , CalleeMode . FP32OutputDenormals ) ;


<s> bool isSramEccOnOrAny ( ) const { return SramEccSetting == TargetIDSetting :: On || SramEccSetting == TargetIDSetting


<s> return XnackSetting == TargetIDSetting :: On || XnackSetting


<s> static bool oneWayCompatible ( bool CallerMode


<s> static bool oneWayCompatible ( bool CallerMode , bool CalleeMode


<s> void setXnackSetting ( TargetIDSetting NewXnackSetting ) { XnackSetting =


<s> MCSection * getHSADataGlobalAgentSection ( MCContext & Ctx ) { return Ctx . getELFSection ( <STR_LIT> , ELF :: SHT_PROGBITS , ELF :: SHF_ALLOC | ELF :: SHF_WRITE | ELF

:: SHF_<ISA_LIT>_HSA_GLOBAL | ELF :: SHF_<ISA_LIT>_HSA_AGENT ) ;


<s> return Ctx . getELFSection ( <STR_LIT> , ELF :: SHT_PROGBITS , ELF :: SHF_ALLOC | ELF :: SHF_<ISA_LIT>_HSA_READONLY | ELF


<s> return Ctx . getELFSection ( <STR_LIT> , ELF :: SHT_PROGBITS , ELF :: SHF_ALLOC | ELF :: SHF_WRITE | ELF :: SHF_EXECINSTR | ELF :: SHF_<ISA_LIT>_HSA_AGENT | ELF


<s> unsigned ShaderType = ShaderType :: COMPUTE ; if ( A . isStringAttribute ( ) ) { StringRef Str = A . getValueAsString (


<s> IsaVersion ISA = getIsaVersion ( Features ) ; memset ( & Header , <NUM_LIT> , sizeof ( Header ) ) ; Header . amd_kernel_code_version_major = <NUM_LIT> ; Header . amd_kernel_code_version_minor = <NUM_LIT> ; Header . amd_machine_kind = <NUM_LIT> ; Header . amd_machine_version_major = ISA . Major ; Header . amd_machine_version_minor =


<s> bool isCI ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( )


<s> bool isGlobalSegment ( const GlobalValue * GV ) { return GV -> getType ( ) -> getAddressSpace ( ) ==


<s> bool isGroupSegment ( const GlobalValue * GV ) { return GV -> getType ( ) -> getAddressSpace


<s> return GV -> getType ( ) -> getAddressSpace ( ) == <ISA_LIT>AS


<s> bool isReadOnlySegment ( const GlobalValue


<s> bool is<ISA_LIT> ( const MCSubtargetInfo & STI


<s> static Waitcnt allZero ( const IsaVersion & Version


<s> unsigned getAddressableNumVGPRs ( const MCSubtargetInfo * STI ) { return <NUM_LIT>


<s> const bool IsCompute = <STR_LIT> ( CC ) ; <ISA_LIT>ModeRegisterDefaults Mode ; Mode . DX10Clamp = true ; Mode . IEEE = IsCompute ; Mode . FP32Denormals = false ; Mode . FP64FP16Denormals = true ; return Mode


<s> return isGFX10 ( STI ) ? getGfx10PlusBufferFormatInfo (

Format ) : getGfx9BufferFormatInfo ( Format ) ;


<s> if ( is<ISA_LIT> ( STI ) || isCI ( STI ) || isVI ( STI ) ) return ID_SYMBOLIC_FIRST_GFX9_ ; else if ( isGFX9 ( STI ) ) return ID_SYMBOLIC_FIRST_GFX10_ ; else return


<s> unsigned getMaxWavesPerEU ( const MCSubtargetInfo


<s> if ( Version . Major >= <NUM_LIT> ) return ExtraSGPRs ; if ( Version . Major < <NUM_LIT> ) { if ( FlatScrUsed ) ExtraSGPRs = <NUM_LIT> ; } else { if ( XNACKUsed ) ExtraSGPRs


<s> if ( VCCUsed ) ExtraSGPRs = <NUM_LIT> ; IsaVersion Version = getIsaVersion ( STI -> getCPU ( ) ) ; if ( Version . Major >= <NUM_LIT> ) return ExtraSGPRs ; if ( Version . Major < <NUM_LIT> ) { if ( FlatScrUsed ) ExtraSGPRs = <NUM_LIT> ; }


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT>


<s> return STI -> getFeatureBits ( ) . test


<s> unsigned getVGPREncodingGranule ( const MCSubtargetInfo * STI , Optional <


<s> return isGCN3Encoding ( ST ) || isGFX10 (


<s> static bool hasSMEMByteOffset ( const MCSubtargetInfo & ST ) { return isGCN3Encoding ( ST ) || isGFX10 (


<s> static bool hasSMRDSignedImmOffset ( const MCSubtargetInfo & ST ) { return isGFX9 ( ST ) || isGFX10


<s> bool isInlineCompatible ( <ISA_LIT>ModeRegisterDefaults CalleeMode ) const { return * this


<s> static bool isLegalSMRDEncodedUnsignedOffset ( const MCSubtargetInfo & ST , int64_t EncodedOffset ) { return hasSMEMByteOffset ( ST ) ? isUInt < <NUM_LIT> > ( EncodedOffset ) : isUInt <


<s> static bool isLegalSMRDEncodedUnsignedOffset ( const MCSubtargetInfo & ST , int64_t EncodedOffset ) { return hasSMEMByteOffset ( ST ) ? isUInt < <NUM_LIT> > ( EncodedOffset

) : isUInt < <NUM_LIT> > ( EncodedOffset ) ;


<s> return ID_SYMBOLIC_FIRST_ <= Id && Id < getLastSymbolicHwreg ( STI ) && IdSymbolic


<s> bool isValidHwreg ( int64_t Id , const MCSubtargetInfo & STI


<s> return ( OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ) && OpId != OP_GS_NOP ; case ID_GS_DONE : return OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ; case


<s> if ( ! Strict ) return <NUM_LIT> <= StreamId && isUInt < STREAM_ID_WIDTH_ > ( StreamId ) ; switch ( MsgId ) { case ID_GS : return STREAM_ID_FIRST_ <= StreamId && StreamId < STREAM_ID_LAST_ ; case ID_GS_DONE : return ( OpId == OP_GS_NOP ) ? ( StreamId == STREAM_ID_NONE_ ) : ( STREAM_ID_FIRST_ <= StreamId && StreamId <


<s> return TT . getOS ( ) == Triple :: AMDPAL


<s> return TT . getOS ( ) == Triple :: AMDPAL || TT . getArch


<s> <ISA_LIT>ModeRegisterDefaults ( ) : IEEE ( true ) , DX10Clamp ( true ) , FP32Denormals ( true ) , FP64FP16Denormals


<s> int64_t getSMRDEncodedOffset ( const MCSubtargetInfo & ST , int64_t ByteOffset ) { if ( hasSMEMByteOffset ( ST ) ) return ByteOffset ; return ByteOffset >>


<s> bool isLegalSMRDImmOffset ( const MCSubtargetInfo & ST , int64_t ByteOffset ) { int64_t EncodedOffset = getSMRDEncodedOffset ( ST , ByteOffset ) ; return ( hasSMEMByteOffset ( ST ) ) ? isUInt < <NUM_LIT> > ( EncodedOffset


<s> bool shouldEmitConstantsToTextSection ( const Triple & TT ) { return TT . getOS ( )


<s> if ( ! IEEEAttr . empty ( ) ) IEEE = IEEEAttr == <STR_LIT> ; StringRef DX10ClampAttr = F . getFnAttribute ( <STR_LIT> ) . getValueAsString


<s> return unpackBits ( Waitcnt , getExpcntBitShift ( ) , getExpcntBitWidth


<s> unsigned decodeLgkmcnt ( const IsaInfo :: IsaVersion & Version , unsigned Waitcnt ) { return unpackBits ( Waitcnt , getLgkmcntBitShift ( ) , getLgkmcntBitWidth (


<s> unsigned VmcntHi = unpackBits ( Waitcnt , getVmcntBitShiftHi ( ) , getVmcntBitWidthHi ( ) ) ; VmcntHi <<= getVmcntBitWidthLo


<s> unsigned encodeLgkmcnt ( const IsaInfo :: IsaVersion & Version , unsigned Waitcnt , unsigned


<s> Waitcnt = encodeVmcnt ( Version , Waitcnt , Vmcnt ) ; Waitcnt = encodeExpcnt ( Version , Waitcnt , Expcnt ) ; Waitcnt = encodeLgkmcnt ( Version , Waitcnt , Lgkmcnt ) ; return Waitcnt


<s> unsigned getAddressableNumVGPRs ( const FeatureBitset &


<s> <ISA_LIT>AS get<ISA_LIT>AS ( const Module


<s> <ISA_LIT>AS get<ISA_LIT>AS ( const Module & M


<s> unsigned getEUsPerCU ( const FeatureBitset & Features ) { return <NUM_LIT>


<s> unsigned getEUsPerCU ( const FeatureBitset &


<s> unsigned getLgkmcntBitMask ( const IsaInfo :: IsaVersion & Version ) { return ( <NUM_LIT> << getLgkmcntBitWidth ( ) )


<s> return ( <NUM_LIT> << getLgkmcntBitWidth ( ) ) -


<s> unsigned getLocalMemorySize ( const FeatureBitset & Features ) { if ( Features . test ( FeatureLocalMemorySize32768 ) ) return <NUM_LIT> ; if ( Features . test ( FeatureLocalMemorySize65536 ) ) return <NUM_LIT> ; return <NUM_LIT>


<s> unsigned getMaxFlatWorkGroupSize ( const


<s> unsigned MaxNumSGPRs = alignDown ( getTotalNumSGPRs ( Features ) / WavesPerEU , getSGPRAllocGranule ( Features ) ) ; unsigned AddressableNumSGPRs = getAddressableNumSGPRs ( Features ) ; if ( Version . Major >= <NUM_LIT> && !

Addressable ) AddressableNumSGPRs = <NUM_LIT> ;


<s> unsigned MaxNumVGPRs = alignDown ( getTotalNumVGPRs ( Features ) / WavesPerEU , getVGPRAllocGranule ( Features ) ) ; unsigned AddressableNumVGPRs = getAddressableNumVGPRs (


<s> assert ( WavesPerEU != <NUM_LIT> ) ; unsigned MaxNumVGPRs = alignDown ( getTotalNumVGPRs ( Features )

/ WavesPerEU , getVGPRAllocGranule ( Features ) ) ;


<s> return getWavesPerWorkGroup ( Features


<s> return alignTo ( getMaxWavesPerCU ( Features , FlatWorkGroupSize ) , getEUsPerCU ( Features )


<s> if ( N == <NUM_LIT> ) return <NUM_LIT> ; N = <NUM_LIT> / N ; return std :: min ( N , <NUM_LIT>


<s> unsigned getMinFlatWorkGroupSize ( const


<s> unsigned MinNumSGPRs = alignDown ( getTotalNumSGPRs ( Features ) / ( WavesPerEU + <NUM_LIT> ) , getSGPRAllocGranule ( Features ) ) + <NUM_LIT> ; return std :: min ( MinNumSGPRs , getAddressableNumSGPRs ( Features


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : return <NUM_LIT>


<s> int64_t getSMRDEncodedOffset ( const MCSubtargetInfo & ST , int64_t ByteOffset ) { if ( is<ISA_LIT> ( ST ) || isCI ( ST ) ) return ByteOffset >> <NUM_LIT> ; return


<s> if ( Version . Major >= <NUM_LIT> ) return <NUM_LIT> ; return <NUM_LIT>


<s> if ( Version . Major >= <NUM_LIT> ) return <NUM_LIT> ; return


<s> unsigned getTotalNumVGPRs ( const FeatureBitset &


<s> unsigned VmcntLo = ( <NUM_LIT> << getVmcntBitWidthLo ( ) ) - <NUM_LIT> ; if ( Version . Major < <NUM_LIT> ) return VmcntLo ; unsigned VmcntHi = ( ( <NUM_LIT> << getVmcntBitWidthHi ( ) ) - <NUM_LIT> ) << getVmcntBitWidthLo ( ) ; return VmcntLo


<s> unsigned Expcnt = getBitMask ( getExpcntBitShift ( ) , getExpcntBitWidth ( ) ) ; unsigned Lgkmcnt = getBitMask ( getLgkmcntBitShift ( ) ,


<s> if ( Features . test ( FeatureWavefrontSize32 ) )


<s> if ( Features . test ( FeatureWavefrontSize32 ) ) return <NUM_LIT> ; return


<s> unsigned getWavesPerWorkGroup ( const FeatureBitset & Features , unsigned


<s> switch ( CC ) { case CallingConv :: <ISA_LIT>_KERNEL : case CallingConv :: SPIR_KERNEL : case CallingConv :: <ISA_LIT>_VS : case CallingConv :: <ISA_LIT>_GS : case CallingConv :: <ISA_LIT>_PS : case CallingConv :: <ISA_LIT>_CS : case CallingConv


getAddressSpace ( ) == AS . GLOBAL_ADDRESS ;


<s> return GV -> getType ( ) -> getAddressSpace ( ) ==


<s> bool isGroupSegment ( const GlobalValue * GV , <ISA_LIT>AS


<s> bool isLegalSMRDImmOffset ( const MCSubtargetInfo & ST , int64_t ByteOffset ) { int64_t EncodedOffset = getSMRDEncodedOffset ( ST , ByteOffset ) ; return is<ISA_LIT> ( ST ) || isCI ( ST ) ? isUInt


<s> bool isReadOnlySegment ( const GlobalValue * GV , <ISA_LIT>AS AS ) { return GV -> getType ( ) -> getAddressSpace ( )


<s> case CallingConv :: <ISA_LIT>_HS : case CallingConv :: <ISA_LIT>_GS : case CallingConv :: <ISA_LIT>_PS : case CallingConv :: <ISA_LIT>_CS : return


<s> bool is<ISA_LIT>SrcOperand ( const MCInstrDesc & Desc , unsigned OpNo ) { assert ( OpNo < Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; return OpType >= <STR_LIT>


<s> assert ( OpNo < Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; return OpType >= <STR_LIT> &&


<s> if ( ! Ptr || isa < UndefValue > ( Ptr ) || isa < Argument > ( Ptr ) || isa < Constant > ( Ptr ) || isa < GlobalValue > ( Ptr ) ) return true ; const Instruction * I = dyn_cast < Instruction


<s> unsigned mc2PseudoReg ( unsigned Reg ) { switch ( Reg ) { case <STR_LIT> : case <STR_LIT> : return FLAT_SCR ; case <STR_LIT> : case <STR_LIT> : return <STR_LIT> ; case


<s> unsigned getMinNumSGPRs ( const MCSubtargetInfo * STI , unsigned WavesPerEU ) { assert ( WavesPerEU != <NUM_LIT> ) ; IsaVersion Version = getIsaVersion ( STI -> getCPU ( ) ) ; if ( Version . Major >= <NUM_LIT> ) return <NUM_LIT> ; if ( WavesPerEU >= getMaxWavesPerEU ( ) ) return <NUM_LIT> ; unsigned MinNumSGPRs = getTotalNumSGPRs ( STI ) / ( WavesPerEU + <NUM_LIT> ) ; if ( STI -> getFeatureBits ( ) . test ( FeatureTrapHandler ) ) MinNumSGPRs -= std :: min ( MinNumSGPRs , (


<s> bool isValidMsgId ( int64_t MsgId , const MCSubtargetInfo & STI , bool Strict ) { return Strict ? isValidMsgId ( MsgId ) && ( MsgId != ID_GS_ALLOC_REQ || isGFX9 ( STI ) || isGFX10 ( STI ) ) : <NUM_LIT> <= MsgId && isUInt < ID_WIDTH_ >


<s> const bool IsCompute = <STR_LIT> ( CC ) ; <ISA_LIT>ModeRegisterDefaults Mode


<s> if ( Features . test ( FeatureISAVersion7_0_2 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion7_0_3 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion7_0_4 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_1 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_2 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_3 ) ) return { <NUM_LIT> , <NUM_LIT> ,


<s> bool hasCodeObjectV3 ( const FeatureBitset & Features ) { return Features . test (


<s> case <NUM_LIT> : return <STR_LIT> ; case <NUM_LIT> : return <STR_LIT> ; default : llvm_unreachable (


<s> case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : return <NUM_LIT> ; case


<s> Stream << TargetTriple . getArchName ( ) << '-' << TargetTriple . getVendorName ( ) << '-' << TargetTriple . getOSName ( ) << '-' << TargetTriple . getEnvironmentName ( ) << '-' << <STR_LIT> << ISAVersion . Major << ISAVersion . Minor << ISAVersion . Stepping ; Stream . flush


<s> void streamIsaVersion ( const MCSubtargetInfo * STI , raw_ostream & Stream ) { auto TargetTriple = STI -> getTargetTriple ( ) ; auto ISAVersion = IsaInfo :: getIsaVersion ( STI -> getFeatureBits ( ) ) ; Stream << TargetTriple . getArchName ( ) << '-' << TargetTriple . getVendorName ( ) << '-' << TargetTriple . getOSName ( ) << '-' << TargetTriple . getEnvironmentName ( ) << '-' << <STR_LIT> << ISAVersion

. Major << ISAVersion . Minor << ISAVersion . Stepping ;


<s> unsigned getMaxWavesPerEU ( const MCSubtargetInfo * STI ) { if ( ! isGFX10 ( * STI )


<s> return hasGFX10_3Insts ( * STI ) ? <NUM_LIT>


<s> STI -> getFeatureBits ( ) . test ( FeatureWavefrontSize32 ) ; return IsWave32 ? <NUM_LIT>


<s> return isHsaAbiVersion3 ( STI ) || isHsaAbiVersion4 ( STI ) ||


<s> if ( Optional < uint8_t > HsaAbiVer = getHsaAbiVersion ( STI ) ) return * HsaAbiVer

== ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V5 ;


<s> static Waitcnt allZero ( ) { return Waitcnt ( <NUM_LIT> , <NUM_LIT> , <NUM_LIT>


<s> if ( ! STI . getFeatureBits ( ) . test ( FeatureSupportsXNACK ) ) XnackSetting = TargetIDSetting :: Unsupported ; if ( ! STI . getFeatureBits ( ) . test ( FeatureSupportsSRAMECC ) )

SramEccSetting = TargetIDSetting :: Unsupported ;


<s> return Waitcnt ( std :: min ( VmCnt , Other . VmCnt ) , std :: min ( ExpCnt , Other . ExpCnt ) , std ::


<s> void decodeDfmtNfmt ( unsigned Format , unsigned & Dfmt , unsigned & Nfmt ) { Dfmt = ( Format >> DFMT_SHIFT ) & DFMT_MASK ; Nfmt = ( Format >> NFMT_SHIFT )


<s> Id = ( Val & ID_MASK_ ) >> ID_SHIFT_ ; Offset = ( Val & OFFSET_MASK_ )


<s> return unpackBits ( Waitcnt , getLgkmcntBitShift ( )

, getLgkmcntBitWidth ( Version . Major ) ) ;


<s> OpId = ( Val & OP_MASK_ ) >> OP_SHIFT_ ; StreamId = ( Val & STREAM_ID_MASK_ ) >>


<s> return VmCnt <= Other . VmCnt && ExpCnt <= Other

. ExpCnt && LgkmCnt <= Other . LgkmCnt ;


<s> bool dominates ( const Waitcnt & Other ) const { return VmCnt <= Other . VmCnt && ExpCnt <= Other


<s> int64_t encodeDfmtNfmt ( unsigned Dfmt , unsigned Nfmt


<s> return packBits ( Lgkmcnt , Waitcnt , getLgkmcntBitShift ( )


<s> unsigned encodeLgkmcnt ( const IsaVersion & Version , unsigned Waitcnt , unsigned Lgkmcnt ) { return packBits ( Lgkmcnt , Waitcnt , getLgkmcntBitShift ( )


<s> unsigned AddrComponents = ( BaseOpcode -> Coordinates ? Dim -> NumCoords : <NUM_LIT> ) + ( BaseOpcode -> LodOrClampOrMip ? <NUM_LIT> : <NUM_LIT> ) ; if ( IsA16 ) AddrWords += divideCeil ( AddrComponents , <NUM_LIT> ) ; else AddrWords += AddrComponents ; if ( BaseOpcode -> Gradients ) { if ( ( IsA16 && ! IsG16Supported ) || BaseOpcode -> G16 ) AddrWords += alignTo < <NUM_LIT> > ( Dim -> NumGradients / <NUM_LIT> ) ; else AddrWords += Dim ->


. test ( FeatureTgSplit ) ? <NUM_LIT> : <NUM_LIT> ) ;


<s> if ( isGFX10Plus ( STI ) ) return UFMT_DEFAULT ; return


<s> int64_t getDfmt ( const StringRef Name ) { for ( int Id = DFMT_MIN ; Id <= DFMT_MAX ; ++ Id ) { if ( Name == DfmtSymbolic [ Id ]


<s> assert ( Id <= DFMT_MAX ) ; return DfmtSymbolic [


<s> assert ( Id <= DFMT_MAX ) ; return DfmtSymbolic [ Id


<s> if ( isGFX10Plus ( * STI ) && STI -> getFeatureBits ( ) . test ( FeatureCuMode ) ) return <NUM_LIT> ; return


<s> return isGFX10Plus ( STI ) ? getGfx10PlusBufferFormatInfo ( Format

) : getGfx9BufferFormatInfo ( Format ) ;


<s> return getIntegerAttribute ( F , <STR_LIT> , F . getCallingConv ( ) ==


<s> bool getHasDepthExport ( const Function &


<s> return ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 ; case <NUM_LIT> : return ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V3 ; case <NUM_LIT> : return ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V4 ; default : report_fatal_error ( Twine ( <STR_LIT> ) +


<s> int64_t getHwregId ( const StringRef Name ) { for ( int Id = ID_SYMBOLIC_FIRST_ ; Id < ID_SYMBOLIC_LAST_ ; ++ Id ) { if ( IdSymbolic [ Id ] && Name == IdSymbolic [


<s> else if ( isGFX9 ( STI ) ) return ID_SYMBOLIC_FIRST_GFX10_ ; else if ( isGFX10 ( STI ) && ! isGFX10_BEncoding ( STI ) ) return ID_SYMBOLIC_FIRST_GFX1030_ ; else return


<s> static unsigned getLastSymbolicHwreg ( const MCSubtargetInfo & STI ) { if ( is<ISA_LIT> ( STI ) || isCI ( STI ) || isVI ( STI ) ) return ID_SYMBOLIC_FIRST_GFX9_ ; else if ( isGFX9 ( STI ) ) return ID_SYMBOLIC_FIRST_GFX10_ ; else if ( isGFX10 ( STI ) && ! isGFX10_BEncoding

( STI ) ) return ID_SYMBOLIC_FIRST_GFX1030_ ;


<s> return ( VersionMajor >= <NUM_LIT> ) ?


<s> if ( ! isGFX10Plus ( * STI ) ) return <NUM_LIT> ; return hasGFX10_3Insts ( * STI


<s> if ( STI -> getTargetTriple ( ) . getArch ( ) != Triple :: amdgcn ) return <NUM_LIT> ; unsigned N = getWavesPerWorkGroup ( STI


<s> if ( STI -> getTargetTriple ( ) . getArch ( ) != Triple :: amdgcn ) return <NUM_LIT> ; unsigned N = getWavesPerWorkGroup ( STI , FlatWorkGroupSize ) ; if ( N == <NUM_LIT> ) return <NUM_LIT> ; N = <NUM_LIT> / N ; return std :: min ( N , <NUM_LIT>


<s> const MIMGBaseOpcodeInfo * getMIMGBaseOpcode ( unsigned Opc ) { const MIMGInfo * Info = getMIMGInfo


<s> if ( Version . Major >= <NUM_LIT> ) return <NUM_LIT> ; if ( WavesPerEU >= getMaxWavesPerEU ( STI ) ) return <NUM_LIT> ; unsigned MinNumSGPRs = getTotalNumSGPRs ( STI ) / ( WavesPerEU + <NUM_LIT> ) ; if ( STI -> getFeatureBits ( ) . test ( FeatureTrapHandler ) ) MinNumSGPRs -= std :: min ( MinNumSGPRs , ( unsigned ) TRAP_NUM_SGPRS ) ; MinNumSGPRs = alignDown ( MinNumSGPRs , getSGPRAllocGranule ( STI ) ) + <NUM_LIT> ; return std :: min ( MinNumSGPRs


<s> unsigned MinNumVGPRs = alignDown ( getTotalNumVGPRs ( STI ) / ( WavesPerEU + <NUM_LIT> ) , getVGPRAllocGranule ( STI )


<s> unsigned MinNumVGPRs = alignDown ( getTotalNumVGPRs ( STI ) / ( WavesPerEU + <NUM_LIT> ) , getVGPRAllocGranule ( STI ) ) + <NUM_LIT> ; return std :: min ( MinNumVGPRs


<s> assert ( <NUM_LIT> <= MsgId && MsgId < ID_GAPS_LAST_ ) ; return IdSymbolic [ MsgId ]


<s> return ( MsgId == ID_SYSMSG ) ? OpSysSymbolic [ OpId


<s> const MTBUFInfo * Info = getMTBUFOpcodeHelper ( Opc ) ; return Info ? Info


<s> bool getMTBUFHasSrsrc ( unsigned Opc ) { const MTBUFInfo * Info = getMTBUFOpcodeHelper ( Opc ) ; return Info ? Info -> has_srsrc : false


<s> bool getMTBUFHasSrsrc ( unsigned Opc ) { const MTBUFInfo * Info = getMTBUFOpcodeHelper ( Opc ) ; return Info ? Info -> has_srsrc


<s> int getMTBUFOpcode ( unsigned BaseOpc , unsigned Elements ) { const MTBUFInfo * Info = getMTBUFInfoFromBaseOpcodeAndElements ( BaseOpc , Elements ) ; return Info ? Info ->


<s> int getMUBUFElements ( unsigned Opc ) { const MUBUFInfo * Info = getMUBUFOpcodeHelper


<s> const MUBUFInfo * Info = getMUBUFOpcodeHelper ( Opc ) ; return Info ? Info -> IsBufferInv :


<s> const MUBUFInfo * Info = getMUBUFInfoFromBaseOpcodeAndElements ( BaseOpc , Elements


<s> int getMUBUFOpcode ( unsigned BaseOpc , unsigned Elements ) { const MUBUFInfo * Info = getMUBUFInfoFromBaseOpcodeAndElements ( BaseOpc , Elements ) ; return Info ? Info -> Opcode


<s> return getNfmtLookupTable ( STI ) [ Id


<s> if ( <STR_LIT> ( ST ) ) return Signed ? <NUM_LIT>


<s> if ( <STR_LIT> ( ST ) ) return Signed ? <NUM_LIT> : <NUM_LIT> ; return Signed ? <NUM_LIT> : <NUM_LIT>


<s> NumVGPRs = alignTo ( std :: max ( <NUM_LIT> , NumVGPRs ) , getVGPREncodingGranule ( STI , EnableWavefrontSize32 ) ) ; return NumVGPRs / getVGPREncodingGranule ( STI


<s> NumVGPRs = alignTo ( std :: max ( <NUM_LIT> , NumVGPRs ) , getVGPREncodingGranule ( STI ,


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT>


<s> const SMInfo * Info = getSMEMOpcodeHelper


<s> if ( ! IsBuffer && hasSMRDSignedImmOffset ( ST ) ) { assert ( hasSMEMByteOffset ( ST ) ) ; return isInt < <NUM_LIT> > ( ByteOffset ) ? Optional < int64_t > ( ByteOffset ) : None ; } if ( ! isDwordAligned ( ByteOffset ) && ! hasSMEMByteOffset ( ST ) ) return None ; int64_t EncodedOffset = convertSMRDOffsetUnits ( ST , ByteOffset ) ; return isLegalSMRDEncodedUnsignedOffset ( ST , EncodedOffset ) ? Optional < int64_t


<s> if ( FeatureString . endswith ( <STR_LIT> ) ) return TargetIDSetting ::


<s> static TargetIDSetting getTargetIDSettingFromFeatureString ( StringRef FeatureString ) { if ( FeatureString . endswith ( <STR_LIT> ) ) return TargetIDSetting :: Off ; if ( FeatureString . endswith ( <STR_LIT> ) ) return


<s> if ( Val . MaxIndex == <NUM_LIT> && Name == Val . Name ) return Val . Tgt ; if ( Val . MaxIndex > <NUM_LIT> && Name . startswith ( Val . Name ) ) { StringRef Suffix = Name . drop_front ( Val . Name . size ( ) ) ; unsigned


<s> for ( const ExpTgt & Val : ExpTgtInfo ) { if ( Val . Tgt <= Id && Id <= Val . Tgt + Val . MaxIndex ) { Index = ( Val . MaxIndex == <NUM_LIT> ) ? - <NUM_LIT> : ( Id - Val


<s> Index = ( Val . MaxIndex == <NUM_LIT> ) ? - <NUM_LIT> : ( Id - Val . Tgt ) ; Name = Val . Name ; return true


<s> if ( ! isGFX10Plus ( * STI ) ) return <NUM_LIT> ; return STI -> getFeatureBits ( ) . test ( FeatureWavefrontSize32 ) ? <NUM_LIT>


<s> StringRef getUnifiedFormatName ( unsigned Id ) { return isValidUnifiedFormat ( Id ) ? UfmtSymbolic


<s> if ( hasGFX10_3Insts ( * STI ) ) return IsWave32 ? <NUM_LIT> : <NUM_LIT> ; return IsWave32 ?


<s> if ( STI -> getFeatureBits ( ) . test ( FeatureGFX90AInsts ) ) return <NUM_LIT> ; bool IsWave32 = EnableWavefrontSize32 ? * EnableWavefrontSize32 : STI -> getFeatureBits ( ) . test ( FeatureWavefrontSize32 ) ; return IsWave32 ? <NUM_LIT>


<s> const VOPInfo * Info = getVOP1OpcodeHelper ( Opc ) ; return Info ? Info ->


<s> bool getVOP2IsSingle ( unsigned Opc ) { const VOPInfo * Info = getVOP2OpcodeHelper


<s> const VOPInfo * Info = getVOP2OpcodeHelper (


<s> unsigned Expcnt = getBitMask ( getExpcntBitShift ( ) , getExpcntBitWidth ( ) ) ; unsigned Lgkmcnt = getBitMask ( getLgkmcntBitShift ( ) , getLgkmcntBitWidth ( Version . Major ) ) ; unsigned Waitcnt = VmcntLo | Expcnt | Lgkmcnt ; if ( Version . Major < <NUM_LIT> ) return Waitcnt ; unsigned VmcntHi = getBitMask ( getVmcntBitShiftHi (


<s> unsigned Lgkmcnt = getBitMask ( getLgkmcntBitShift ( ) , getLgkmcntBitWidth ( Version . Major ) ) ; unsigned Waitcnt = VmcntLo | Expcnt | Lgkmcnt ; if ( Version . Major < <NUM_LIT> ) return Waitcnt ; unsigned VmcntHi = getBitMask ( getVmcntBitShiftHi ( ) , getVmcntBitWidthHi


<s> unsigned getWavesPerEUForWorkGroup ( const MCSubtargetInfo * STI , unsigned FlatWorkGroupSize ) { return divideCeil ( getWavesPerWorkGroup ( STI , FlatWorkGroupSize ) , getEUsPerCU


<s> return divideCeil ( getWavesPerWorkGroup ( STI , FlatWorkGroupSize ) , getEUsPerCU


<s> unsigned getWavesPerWorkGroup ( const MCSubtargetInfo *


<s> unsigned getWavesPerWorkGroup ( const MCSubtargetInfo * STI , unsigned FlatWorkGroupSize ) { return divideCeil ( FlatWorkGroupSize , getWavefrontSize ( STI )


<s> bool hasArchitectedFlatScratch ( const MCSubtargetInfo & STI


<s> return STI . getFeatureBits ( ) [ <STR_LIT>


<s> bool hasGFX10_3Insts ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( ) [ <STR_LIT>


<s> return STI . getFeatureBits ( ) [ <STR_LIT> ] && ! STI . getFeatureBits ( )


<s> static bool hasSMEMByteOffset ( const MCSubtargetInfo & ST ) { return isGCN3Encoding ( ST ) || isGFX10Plus (


<s> static bool hasSMEMByteOffset ( const MCSubtargetInfo & ST ) { return isGCN3Encoding ( ST ) || isGFX10Plus


<s> static bool hasSMRDSignedImmOffset ( const MCSubtargetInfo & ST ) { return isGFX9Plus (


<s> bool isArgPassedInSGPR ( const Argument * A ) { const Function * F = A -> getParent ( ) ; CallingConv :: ID CC = F -> getCallingConv ( ) ; switch ( CC ) { case CallingConv :: <ISA_LIT>_KERNEL : case CallingConv :: SPIR_KERNEL : return true ; case CallingConv :: <ISA_LIT>_VS : case CallingConv :: <ISA_LIT>_LS : case CallingConv :: <ISA_LIT>_HS : case CallingConv :: <ISA_LIT>_ES : case CallingConv :: <ISA_LIT>_GS : case CallingConv


<s> case CallingConv :: <ISA_LIT>_Gfx : return F -> getAttributes ( ) . hasParamAttr ( A -> getArgNo ( ) , Attribute :: InReg ) || F -> getAttributes ( ) . hasParamAttr ( A ->


<s> bool isCompute ( CallingConv :: ID cc ) { return ! isGraphics ( cc ) || cc


<s> static bool isDwordAligned ( uint64_t ByteOffset )


<s> int16_t Hi16 = static_cast < int16_t > ( Literal >> <NUM_LIT> ) ; if ( ! ( Literal & <NUM_LIT> ) ) return true ; return Lo16 == Hi16


<s> bool isGFX10 ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( ) [


<s> bool isGFX10_AEncoding ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( ) [


<s> bool isGFX10_BEncoding ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( ) [


<s> bool isGFX90A ( const MCSubtargetInfo & STI ) { return STI . getFeatureBits ( ) [


<s> bool isGFX9Plus ( const MCSubtargetInfo & STI ) { return isGFX9 ( STI ) || isGFX10Plus ( STI


<s> bool isGraphics ( CallingConv :: ID cc


<s> bool isGroupSegment ( const GlobalValue * GV


<s> bool isGroupSegment ( const GlobalValue * GV ) { return GV -> getAddressSpace ( ) == <ISA_LIT>AS ::


<s> bool isHsaAbiVersion2 ( const MCSubtargetInfo * STI ) { if ( Optional < uint8_t > HsaAbiVer = getHsaAbiVersion ( STI

) ) return * HsaAbiVer == ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 ;


<s> return isHsaAbiVersion3 ( STI ) || isHsaAbiVersion4 (


<s> bool isInlinableIntLiteralV216 ( int32_t Literal ) { int16_t Lo16 = static_cast < int16_t > ( Literal ) ; if ( isInt < <NUM_LIT> > ( Literal ) || isUInt < <NUM_LIT> > ( Literal ) ) return isInlinableIntLiteral ( Lo16 ) ; int16_t Hi16 = static_cast < int16_t > ( Literal >> <NUM_LIT>


<s> bool isInlinableLiteral16 ( int16_t Literal , bool HasInv2Pi ) { if ( ! HasInv2Pi ) return false ; if ( isInlinableIntLiteral ( Literal


<s> if ( isInlinableIntLiteral ( Literal ) ) return true ; uint64_t Val = static_cast < uint64_t > ( Literal ) ; return ( Val == DoubleToBits ( <NUM_LIT> ) ) || ( Val == DoubleToBits ( <NUM_LIT> ) ) || ( Val == DoubleToBits ( - <NUM_LIT> ) ) || ( Val == DoubleToBits ( <NUM_LIT> ) ) || ( Val == DoubleToBits ( - <NUM_LIT> ) ) || ( Val == DoubleToBits ( <NUM_LIT> ) ) || ( Val == DoubleToBits ( - <NUM_LIT> ) ) || (

Val == DoubleToBits ( <NUM_LIT> ) ) || ( Val == DoubleToBits ( - <NUM_LIT> ) ) || ( Val == <NUM_LIT> && HasInv2Pi ) ;


<s> assert ( HasInv2Pi ) ; if ( isInt < <NUM_LIT> > ( Literal ) || isUInt < <NUM_LIT> > ( Literal ) ) { int16_t Trunc = static_cast < int16_t >


<s> bool isLegalSMRDEncodedSignedOffset ( const MCSubtargetInfo & ST , int64_t EncodedOffset


<s> bool isLegalSMRDEncodedUnsignedOffset ( const MCSubtargetInfo & ST , int64_t EncodedOffset ) { return hasSMEMByteOffset ( ST ) ? isUInt < <NUM_LIT> > ( EncodedOffset ) :

isUInt < <NUM_LIT> > ( EncodedOffset ) ;


<s> bool isSupportedTgtId ( unsigned Id , const MCSubtargetInfo & STI


<s> decodeDfmtNfmt ( Id , Dfmt , Nfmt ) ; return isValidNfmt ( Nfmt


<s> bool isValidFormatEncoding ( unsigned Val , const


<s> return isGFX10Plus ( STI ) ? ( Val <= UFMT_MAX ) : (


<s> bool isValidHwreg ( int64_t Id ) { return <NUM_LIT> <= Id && isUInt < ID_WIDTH_


<s> bool isValidHwreg ( int64_t Id ) { return <NUM_LIT> <= Id && isUInt


<s> bool isValidHwregOffset ( int64_t Offset ) { return <NUM_LIT> <= Offset && isUInt <


<s> return <NUM_LIT> <= Offset && isUInt


<s> bool isValidMsgOp ( int64_t MsgId , int64_t OpId , const MCSubtargetInfo & STI , bool Strict ) { assert ( isValidMsgId ( MsgId , STI , Strict ) ) ; if ( ! Strict ) return <NUM_LIT> <= OpId && isUInt < OP_WIDTH_ > ( OpId ) ; switch ( MsgId ) { case ID_GS : return ( OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ) && OpId != OP_GS_NOP ; case ID_GS_DONE : return OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ; case ID_SYSMSG


<s> return ( OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ) && OpId != OP_GS_NOP ; case ID_GS_DONE : return OP_GS_FIRST_ <= OpId && OpId < OP_GS_LAST_ ; case ID_SYSMSG : return OP_SYS_FIRST_ <= OpId && OpId < OP_SYS_LAST_ ; default : return OpId


<s> return MsgId == ID_GS || MsgId == ID_GS_DONE


<s> bool msgSupportsStream ( int64_t MsgId


<s> bool msgSupportsStream ( int64_t MsgId , int64_t OpId ) { return ( MsgId == ID_GS || MsgId == ID_GS_DONE ) && OpId


<s> if ( XnackSupported ) { XnackSetting = * XnackRequested ? TargetIDSetting :: On : TargetIDSetting :: Off ; } else { if ( * XnackRequested ) { errs ( ) << <STR_LIT> <STR_LIT> ; } else { errs ( ) << <STR_LIT> <STR_LIT> ; } } } if ( SramEccRequested ) { if ( SramEccSupported ) { SramEccSetting = * SramEccRequested ? TargetIDSetting :: On : TargetIDSetting :: Off ; } else { if ( * SramEccRequested


<s> SmallVector < StringRef , <NUM_LIT> > TargetIDSplit ; TargetID . split ( TargetIDSplit , ':' ) ; for ( const auto & FeatureString : TargetIDSplit ) { if ( FeatureString . startswith ( <STR_LIT> ) ) XnackSetting = getTargetIDSettingFromFeatureString ( FeatureString ) ; if ( FeatureString . startswith ( <STR_LIT> )

) SramEccSetting = getTargetIDSettingFromFeatureString ( FeatureString ) ;


<s> void <ISA_LIT>TargetID :: setTargetIDFromTargetIDStream ( StringRef TargetID ) { SmallVector < StringRef , <NUM_LIT> > TargetIDSplit ; TargetID . split (


<s> <STR_LIT> ( const Function & F ) { * this = getDefaultForCallingConv ( F . getCallingConv ( ) ) ; StringRef IEEEAttr = F . getFnAttribute ( <STR_LIT> ) . getValueAsString ( ) ; if ( ! IEEEAttr . empty ( ) ) IEEE = IEEEAttr == <STR_LIT> ; StringRef DX10ClampAttr = F . getFnAttribute ( <STR_LIT> ) . getValueAsString ( ) ; if ( ! DX10ClampAttr . empty ( ) ) DX10Clamp = DX10ClampAttr == <STR_LIT> ; StringRef DenormF32Attr = F . getFnAttribute ( <STR_LIT> ) . getValueAsString ( ) ; if ( ! DenormF32Attr . empty ( ) ) { DenormalMode DenormMode = parseDenormalFPAttribute ( DenormF32Attr ) ; FP32InputDenormals = DenormMode . Input == DenormalMode :: IEEE ; FP32OutputDenormals = DenormMode . Output == DenormalMode :: IEEE ; } StringRef DenormAttr = F . getFnAttribute ( <STR_LIT> ) . getValueAsString ( ) ; if ( ! DenormAttr . empty ( ) ) { DenormalMode DenormMode = parseDenormalFPAttribute


<s> if ( Imm <= MaxImm + <NUM_LIT> ) { Overflow = Imm - MaxImm ; Imm = MaxImm ; } else { uint32_t High = ( Imm + Alignment . value ( ) ) & ~ <NUM_LIT> ; uint32_t Low = ( Imm + Alignment . value ( ) ) & <NUM_LIT> ; Imm = Low ; Overflow = High - Alignment . value ( ) ; } } if ( Overflow > <NUM_LIT> && Subtarget -> getGeneration ( ) <= <ISA_LIT>Subtarget :: SEA_ISLANDS ) return false ; ImmOffset = Imm ; SOffset =


<s> std :: string Processor = <STR_LIT> ; if ( Version . Major >= <NUM_LIT> ) Processor = STI . getCPU ( ) . str ( ) ; else Processor = ( Twine ( <STR_LIT> ) + Twine ( Version . Major ) + Twine ( Version . Minor ) + Twine ( Version . Stepping ) ) . str ( ) ; std :: string Features = <STR_LIT> ; if ( Optional < uint8_t > HsaAbiVersion = getHsaAbiVersion ( & STI ) ) { switch ( * HsaAbiVersion ) { case ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 : if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { if ( ! isXnackOnOrAny ( ) ) report_fatal_error ( <STR_LIT> + Twine ( Processor ) + <STR_LIT> ) ; } else if ( Processor == <STR_LIT> ) { } else if ( Processor


<s> switch ( OpInfo . OperandType ) { case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT>


<s> if ( Features . test ( FeatureISAVersion8_1_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion9_0_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion9_0_1 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( ! Features . test ( FeatureGCN ) || Features . test ( FeatureSouthernIslands ) ) return {


<s> bool isEntryFunctionCC ( CallingConv


<s> bool isEntryFunctionCC ( CallingConv ::


<s> <ISA_LIT>AS get<ISA_LIT>AS ( Triple T ) { <ISA_LIT>AS AS


<s> unsigned AddressableNumSGPRs = getAddressableNumSGPRs ( Features ) ; if ( Version . Major >= <NUM_LIT> && ! Addressable ) AddressableNumSGPRs = <NUM_LIT> ; unsigned MaxNumSGPRs = getTotalNumSGPRs ( Features ) / WavesPerEU ; if ( Features . test ( FeatureTrapHandler ) ) MaxNumSGPRs -= std :: min ( MaxNumSGPRs


<s> Header . amd_machine_version_stepping = ISA . Stepping ; Header . kernel_code_entry_byte_offset = sizeof ( Header ) ; Header . wavefront_size = <NUM_LIT> ; Header . call_convention = - <NUM_LIT> ; Header . kernarg_segment_alignment = <NUM_LIT> ; Header . group_segment_alignment = <NUM_LIT> ; Header . private_segment_alignment


<s> IsaInfo :: IsaVersion ISA = IsaInfo :: getIsaVersion ( Features ) ; memset ( & Header , <NUM_LIT> , sizeof ( Header ) ) ; Header . amd_kernel_code_version_major = <NUM_LIT> ; Header . amd_kernel_code_version_minor = <NUM_LIT> ; Header . amd_machine_kind = <NUM_LIT> ; Header . amd_machine_version_major = ISA . Major ; Header . amd_machine_version_minor = ISA . Minor ; Header . amd_machine_version_stepping = ISA . Stepping ; Header . kernel_code_entry_byte_offset = sizeof ( Header ) ; Header . wavefront_size = <NUM_LIT> ; Header . call_convention = - <NUM_LIT> ; Header . kernarg_segment_alignment = <NUM_LIT>


<s> bool isIntrinsicSourceOfDivergence ( unsigned IntrID ) { return lookupSourceOfDivergenceByIntr (


<s> IsaVersion getIsaVersion ( const FeatureBitset & Features ) { if ( Features . test ( FeatureISAVersion7_0_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion7_0_1 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_0 ) ) return {


<s> switch ( OpInfo . OperandType ) { case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT>


<s> return getIntegerAttribute ( F , <STR_LIT> ,


<s> unsigned getShaderType ( const Function


<s> unsigned decodeExpcnt ( IsaVersion Version , unsigned Waitcnt ) { return unpackBits ( Waitcnt , getExpcntBitShift ( ) , getExpcntBitWidth ( )


<s> unsigned decodeExpcnt ( IsaVersion Version , unsigned Waitcnt ) { return unpackBits ( Waitcnt , getExpcntBitShift (


<s> unsigned decodeVmcnt ( IsaVersion Version , unsigned Waitcnt


<s> Vmcnt = decodeVmcnt ( Version , Waitcnt ) ; Expcnt = decodeExpcnt ( Version , Waitcnt ) ; Lgkmcnt = decodeLgkmcnt ( Version ,


<s> void decodeWaitcnt ( IsaVersion Version , unsigned Waitcnt , unsigned & Vmcnt , unsigned & Expcnt , unsigned & Lgkmcnt ) { Vmcnt = decodeVmcnt ( Version


<s> unsigned encodeExpcnt ( IsaVersion Version , unsigned Waitcnt , unsigned Expcnt ) { return packBits ( Expcnt , Waitcnt , getExpcntBitShift (


<s> unsigned encodeExpcnt ( IsaVersion Version , unsigned Waitcnt , unsigned


<s> unsigned encodeLgkmcnt ( IsaVersion Version ,


<s> unsigned encodeLgkmcnt ( IsaVersion Version , unsigned Waitcnt , unsigned Lgkmcnt ) { return packBits ( Lgkmcnt , Waitcnt , getLgkmcntBitShift ( ) , getLgkmcntBitWidth (


<s> unsigned encodeVmcnt ( IsaVersion Version , unsigned Waitcnt , unsigned Vmcnt ) { return packBits ( Vmcnt , Waitcnt , getVmcntBitShift


<s> unsigned Waitcnt = getWaitcntBitMask ( Version ) ; Waitcnt = encodeVmcnt ( Version , Waitcnt , Vmcnt ) ; Waitcnt = encodeExpcnt ( Version , Waitcnt , Expcnt ) ; Waitcnt = encodeLgkmcnt ( Version , Waitcnt , Lgkmcnt ) ; return


<s> <ISA_LIT>ModeRegisterDefaults Mode ; Mode . DX10Clamp = true ; Mode . IEEE = <STR_LIT> ( CC )


<s> unsigned getExpcntBitMask ( IsaVersion


<s> return ( <NUM_LIT> << getExpcntBitWidth ( ) )


<s> if ( Features . test ( FeatureISAVersion8_0_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_1 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_2 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_3 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_4 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_1_0 ) ) return


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT>


<s> unsigned getRegOperandSize ( const MCRegisterInfo * MRI , const


<s> unsigned RCID = Desc . OpInfo [ OpNo ] . RegClass ; return getRegBitWidth ( MRI -> getRegClass (


<s> unsigned getVmcntBitShift ( ) { return


<s> unsigned getVmcntBitWidth ( ) { return


<s> unsigned Lgkmcnt = getBitMask ( getLgkmcntBitShift ( ) , getLgkmcntBitWidth ( ) ) ; return Vmcnt | Expcnt


<s> IsaVersion ISA = getIsaVersion ( Features ) ; memset ( & Header , <NUM_LIT> , sizeof ( Header ) ) ; Header . amd_kernel_code_version_major = <NUM_LIT> ; Header . amd_kernel_code_version_minor = <NUM_LIT> ; Header . amd_machine_kind = <NUM_LIT> ; Header . amd_machine_version_major =


<s> Header . amd_machine_version_stepping = ISA . Stepping ; Header . kernel_code_entry_byte_offset = sizeof ( Header ) ; Header . wavefront_size = <NUM_LIT> ; Header . call_convention = - <NUM_LIT> ; Header . kernarg_segment_alignment = <NUM_LIT> ; Header . group_segment_alignment = <NUM_LIT>


<s> bool is<ISA_LIT>SrcFPOperand ( const MCInstrDesc & Desc , unsigned OpNo ) { assert ( OpNo <= Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; switch ( OpType ) { case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return true ; default : return false


<s> bool is<ISA_LIT>SrcInlinableOperand ( const MCInstrDesc & Desc , unsigned OpNo ) { assert ( OpNo <= Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; return OpType >= <STR_LIT> && OpType <=


<s> bool is<ISA_LIT>SrcOperand ( const MCInstrDesc & Desc , unsigned OpNo ) { assert ( OpNo <= Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; return OpType >= <STR_LIT> && OpType


<s> if ( Features . test ( FeatureISAVersion8_0_1 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_2 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_0_3 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion8_1_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureVolcanicIslands ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion9_0_0 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureISAVersion9_0_2 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( Features . test ( FeatureGFX9 ) ) return { <NUM_LIT> , <NUM_LIT> , <NUM_LIT> } ; if ( ! Features . test ( FeatureGCN ) || Features . test ( FeatureSouthernIslands ) )


<s> case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT>


<s> if ( ! STI . getFeatureBits ( ) . test ( FeatureSupportsSRAMECC ) ) SramEccSetting =


<s> Optional < uint8_t > getHsaAbiVersion ( const MCSubtargetInfo * STI ) { if ( STI && STI -> getTargetTriple ( ) . getOS ( ) != Triple :: AMDHSA ) return None ; switch ( AmdhsaCodeObjectVersion ) { case <NUM_LIT> : return ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 ; case <NUM_LIT> : return ELF ::


<s> if ( ! isGFX10Plus ( * STI ) ) return <NUM_LIT> ; return hasGFX10_3Insts ( * STI ) ? <NUM_LIT>


<s> bool isArgPassedInSGPR ( const Argument * A ) { const Function * F = A -> getParent ( ) ; CallingConv :: ID CC = F -> getCallingConv ( ) ; switch ( CC ) { case CallingConv :: <ISA_LIT>_KERNEL : case CallingConv :: SPIR_KERNEL : return true ; case CallingConv :: <ISA_LIT>_VS : case CallingConv ::


<s> if ( const auto && HsaAbiVer = getHsaAbiVersion ( STI )

) return HsaAbiVer . getValue ( ) == ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 ;


<s> if ( const auto && HsaAbiVer = getHsaAbiVersion ( STI ) ) return HsaAbiVer . getValue

( ) == ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V3 ;


<s> Stream << TargetTriple . getArchName ( ) << '-' << TargetTriple . getVendorName ( ) << '-' << TargetTriple . getOSName ( ) << '-' << TargetTriple . getEnvironmentName ( ) << '-' << <STR_LIT> << Version . Major << Version . Minor << Version . Stepping ; if ( hasXNACK ( * STI ) ) Stream


<s> Stream << TargetTriple . getArchName ( ) << '-' << TargetTriple . getVendorName ( ) << '-' << TargetTriple . getOSName ( ) << '-' << TargetTriple . getEnvironmentName ( ) << '-' << <STR_LIT> << Version . Major << Version . Minor << Version . Stepping ; if ( hasXNACK ( * STI ) ) Stream << <STR_LIT> ; if ( hasSRAMECC ( * STI ) ) Stream << <STR_LIT> ; Stream . flush (


<s> return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return <NUM_LIT> ; case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case


<s> assert ( OpNo < Desc . NumOperands ) ; unsigned OpType = Desc . OpInfo [ OpNo ] . OperandType ; switch ( OpType ) { case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return true


<s> auto TargetTriple = STI . getTargetTriple ( ) ; auto Version = getIsaVersion ( STI . getCPU ( ) ) ; StreamRep << TargetTriple . getArchName ( ) << '-' << TargetTriple . getVendorName ( ) << '-' << TargetTriple . getOSName ( ) << '-' << TargetTriple . getEnvironmentName ( ) << '-' ; std :: string Processor = <STR_LIT> ; if ( Version . Major >= <NUM_LIT> ) Processor = STI . getCPU ( ) . str ( ) ; else Processor = ( Twine ( <STR_LIT> ) + Twine ( Version . Major ) + Twine ( Version . Minor ) + Twine ( Version . Stepping ) ) . str ( ) ; std :: string Features = <STR_LIT> ; if ( Optional < uint8_t > HsaAbiVersion = getHsaAbiVersion ( & STI ) ) { switch ( * HsaAbiVersion ) { case ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V2 : if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { if ( ! isXnackOnOrAny ( ) ) report_fatal_error ( <STR_LIT> + Processor + <STR_LIT> ) ; } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { } else if ( Processor == <STR_LIT> ) { if ( ! isXnackOnOrAny ( ) ) report_fatal_error ( <STR_LIT> + Processor + <STR_LIT> ) ; } else if ( Processor == <STR_LIT> ) { if ( isXnackOnOrAny ( ) ) Processor = <STR_LIT> ; } else if ( Processor == <STR_LIT> ) { if ( isXnackOnOrAny ( ) ) Processor = <STR_LIT> ; } else if ( Processor == <STR_LIT> ) { if ( isXnackOnOrAny ( ) ) Processor = <STR_LIT> ; } else if ( Processor == <STR_LIT> ) { if ( isXnackOnOrAny ( ) ) Processor = <STR_LIT> ; } else if ( Processor == <STR_LIT> ) { if ( isXnackOnOrAny ( ) ) report_fatal_error ( <STR_LIT> + Processor + <STR_LIT> ) ; } else { report_fatal_error ( <STR_LIT> + Processor ) ; } break ; case ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V3 : if ( isXnackOnOrAny ( ) ) Features += <STR_LIT> ; if ( isSramEccOnOrAny ( ) ) Features += <STR_LIT> ; break ; case ELF :: ELFABIVER<ISA_LIT>ON_AMDGPU_HSA_V4 : if ( getSramEccSetting ( ) == TargetIDSetting :: Off ) Features += <STR_LIT> ; else if ( getSramEccSetting ( ) == TargetIDSetting :: On ) Features += <STR_LIT> ; if ( getXnackSetting ( ) == TargetIDSetting :: Off


<s> int64_t encodeHwreg ( int64_t Id , int64_t Offset , int64_t Width


<s> int64_t encodeHwreg ( int64_t Id , int64_t Offset , int64_t Width ) { return ( Id << ID_SHIFT_ ) | ( Offset << OFFSET_SHIFT_ ) | ( ( Width - <NUM_LIT>


<s> bool decodeDepCtr ( unsigned Code , int & Id , StringRef & Name , unsigned &

Val , bool & IsDefault , const MCSubtargetInfo & STI ) {


<s> static int encodeCustomOperandVal ( const CustomOperandVal & Op , int64_t InputVal ) { if ( InputVal < <NUM_LIT> || InputVal > Op . Max


<s> if ( InputVal < <NUM_LIT> || InputVal > Op . Max ) return OPR_VAL_INVALID ; return Op . encode ( InputVal


<s> static int Default = - <NUM_LIT> ; if ( Default == - <NUM_LIT> ) Default = getDefaultCustomOperandEncoding (

DepCtrInfo , DEP_CTR_<ISA_LIT>ZE , STI ) ;


<s> for ( int Idx = <NUM_LIT> ; Idx < Size ; ++ Idx ) { const auto & Op = Opr [ Idx ] ; if ( ! Op . isSupported ( STI ) ) continue ; UsedOprMask |= Op . getMask ( ) ; unsigned Val = Op . decode ( Code ) ; if ( ! Op . isValid ( Val ) ) return false ; HasNonDefaultVal |= ( Val != Op . Default ) ; } return ( Code & ~ UsedOprMask ) ==


<s> return isSymbolicCustomOperandEncoding ( DepCtrInfo , DEP_CTR_<ISA_LIT>ZE ,


<s> switch ( OpType ) { case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : case <STR_LIT> : return true


<s> unsigned decodeExpcnt ( const IsaVersion & Version , unsigned Waitcnt ) { return unpackBits ( Waitcnt , getExpcntBitShift ( Version . Major ) , getExpcntBitWidth ( Version


<s> return unpackBits ( Waitcnt , getLgkmcntBitShift ( Version . Major ) , getLgkmcntBitWidth ( Version .


<s> unsigned VmcntHi = unpackBits ( Waitcnt , getVmcntBitShiftHi ( Version . Major ) , getVmcntBitWidthHi ( Version .


<s> return packBits ( Expcnt , Waitcnt , getExpcntBitShift ( Version . Major ) , getExpcntBitWidth ( Version


<s> uint64_t encodeMsg ( uint64_t MsgId , uint64_t OpId , uint64_t StreamId ) { return MsgId | ( OpId << OP_SHIFT_


<s> Waitcnt = packBits ( Vmcnt , Waitcnt , getVmcntBitShiftLo ( Version . Major ) , getVmcntBitWidthLo ( Version . Major ) ) ; return packBits ( Vmcnt >> getVmcntBitWidthLo ( Version . Major ) , Waitcnt , getVmcntBitShiftHi ( Version . Major ) , getVmcntBitWidthHi