<s> assert ( NodeAttrs :: type ( Attrs ) == NodeAttrs :: Ref ) ; assert ( ! ( NodeAttrs :: flags ( Attrs ) & NodeAttrs ::


<s> assert ( ! ( NodeAttrs :: flags ( Attrs ) & NodeAttrs :: PhiRef ) ) ; ( void )


<s> auto Aliased = [ this , & MaxRefs ] ( RegisterRef RR , std :: vector < unsigned > & Closure ) -> bool { for ( unsigned I : Closure ) if ( PRI . alias ( RR , MaxRefs [ I ] ) ) return true ; return false ; } ; NodeList Preds ; const MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( MachineBasicBlock * PB : MBB -> predecessors ( ) ) Preds . push_back ( findBlock ( PB ) ) ; while ( ! MaxRefs . empty ( ) ) { std :: vector < unsigned > ClosureIdx = { <NUM_LIT> } ; for ( unsigned i = <NUM_LIT> ; i != MaxRefs . size ( ) ; ++ i ) if ( Aliased ( MaxRefs [ i ] , ClosureIdx ) ) ClosureIdx . push_back ( i ) ; unsigned CS = ClosureIdx . size ( ) ; NodeAddr < PhiNode * > PA = newPhi ( BA ) ; for ( unsigned X = <NUM_LIT> ; X != CS ; ++ X ) { RegisterRef RR = MaxRefs [ ClosureIdx [ X ] ] ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } for ( NodeAddr < BlockNode * > PBA : Preds ) { for ( unsigned X = <NUM_LIT> ; X != CS ; ++ X ) { RegisterRef RR = MaxRefs [ ClosureIdx [ X ]


<s> const Function & F = * MF . getFunction ( ) ; const Constant * PF = F . hasPersonalityFn ( ) ? F . getPersonalityFn ( ) : nullptr ; const TargetLowering & TLI = * MF . getSubtarget ( ) . getTargetLowering ( ) ; if ( RegisterId R = TLI . getExceptionPointerRegister ( PF ) ) LR . insert ( RegisterRef ( R ) ) ; if ( RegisterId R = TLI . getExceptionSelectorRegister ( PF ) ) LR . insert ( RegisterRef ( R ) ) ; return LR


<s> RegisterSet LR ; const Function & F = * MF . getFunction ( ) ; const Constant * PF = F . hasPersonalityFn ( ) ? F . getPersonalityFn ( ) : nullptr ; const TargetLowering & TLI = * MF . getSubtarget ( ) . getTargetLowering ( ) ; if ( RegisterId R = TLI . getExceptionPointerRegister ( PF ) ) LR . insert ( RegisterRef ( R ) ) ; if ( RegisterId R = TLI . getExceptionSelectorRegister ( PF ) ) LR . insert ( RegisterRef


<s> if ( In . isCall ( ) || In . isReturn ( ) || In . isInlineAsm ( ) ) return true ; if ( In . isBranch ( ) ) for ( const MachineOperand & O : In . operands ( ) ) if ( O . isGlobal ( ) || O . isSymbol ( ) ) return true ; const MCInstrDesc & D = In . getDesc ( ) ; if ( ! D . getImplicitDefs ( ) && ! D . getImplicitUses ( ) ) return false ; const MachineOperand & Op = In . getOperand ( OpNum ) ; if ( Op . getSubReg ( ) != <NUM_LIT>


<s> if ( Sub != <NUM_LIT> ) Reg = TRI . getSubReg ( Reg , Sub ) ; return RegisterRef ( Reg


<s> RegisterRef DataFlowGraph :: makeRegRef ( unsigned Reg , unsigned Sub ) const { assert ( PhysicalRegisterInfo :: isRegMaskId ( Reg ) || TargetRegisterInfo :: isPhysicalRegister ( Reg ) ) ; assert ( Reg != <NUM_LIT> ) ; if ( Sub != <NUM_LIT> ) Reg = TRI . getSubReg ( Reg ,


<s> bool IsCall = In . isCall ( ) , IsReturn = In . isReturn ( ) ; bool IsPredicated = TII . isPredicated ( & In ) ; unsigned NumOps = In . getNumOperands ( ) ; RegisterSet DoneDefs ; for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || ! Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; if ( ! IsCall && ! ImpDefs . count ( RR ) ) continue ; if ( DoneDefs . count ( RR ) ) continue ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isUse ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; bool Implicit = Op . isImplicit ( ) ; bool TakeImplicit = IsReturn || IsCall || IsPredicated ; if ( Implicit && ! TakeImplicit && ! ImpUses . count ( RR ) ) continue ; uint16_t Flags = NodeAttrs


<s> unsigned NumOps = In . getNumOperands ( ) ; RegisterSet DoneDefs ; for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || ! Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; if ( ! IsCall && ! ImpDefs . count ( RR ) ) continue ; if ( DoneDefs . count ( RR ) ) continue ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isUse ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op


<s> bool TargetOperandInfo :: isPreserving ( const MachineInstr & In ,


<s> char * P = static_cast < char * > ( T ) ; Blocks . push_back ( P ) ; assert ( ( Blocks . size ( ) < ( <NUM_LIT> << ( <NUM_LIT> * sizeof ( NodeId ) - BitsPerIndex ) ) ) && <STR_LIT> ) ; ActiveEnd = P


<s> Res . push_back ( RA ) ; N = RA . Addr -> getSibling ( ) ; } return Res ; } ; NodeList ReachedDefs = getAllNodes ( DA . Addr -> getReachedDef ( ) ) ; NodeList ReachedUses = getAllNodes ( DA . Addr -> getReachedUse ( ) ) ; if ( RD == <NUM_LIT> ) { for ( NodeAddr < RefNode * > I : ReachedDefs ) I . Addr -> setSibling ( <NUM_LIT> ) ; for ( NodeAddr < RefNode * > I : ReachedUses ) I . Addr -> setSibling ( <NUM_LIT> ) ; } for ( NodeAddr < DefNode * > I : ReachedDefs ) I . Addr -> setReachingDef ( RD ) ; for ( NodeAddr < UseNode * > I : ReachedUses ) I . Addr -> setReachingDef ( RD ) ; NodeId Sib = DA . Addr -> getSibling ( ) ; if ( RD ==


<s> return RR ; } ; RegisterSet MaxDF ; for ( RegisterRef I : HasDF -> second ) MaxDF . insert ( MaxCoverIn ( I , HasDF -> second ) ) ; std :: vector < RegisterRef > MaxRefs ; for ( RegisterRef I : MaxDF ) MaxRefs . push_back ( MaxCoverIn ( I , AllRefs ) ) ; llvm :: sort ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; auto NewEnd = std :: unique ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; MaxRefs . erase ( NewEnd , MaxRefs . end ( ) ) ; auto Aliased = [ this , & MaxRefs ] ( RegisterRef RR , std :: vector < unsigned > & Closure ) -> bool { for ( unsigned I : Closure ) if ( PRI . alias ( RR , MaxRefs [ I ] ) ) return true ; return false ; } ; NodeList Preds ; const MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( MachineBasicBlock * PB : MBB -> predecessors ( ) ) Preds . push_back ( findBlock ( PB ) ) ; while ( ! MaxRefs . empty ( ) ) { std :: vector < unsigned > ClosureIdx = { <NUM_LIT> } ; for ( unsigned i = <NUM_LIT> ; i != MaxRefs . size ( ) ; ++ i ) if ( Aliased ( MaxRefs [ i ] , ClosureIdx ) ) ClosureIdx . push_back ( i ) ; unsigned CS = ClosureIdx . size ( ) ; NodeAddr < PhiNode * > PA = newPhi ( BA ) ; for ( unsigned X = <NUM_LIT> ; X != CS ; ++ X ) { RegisterRef RR = MaxRefs [ ClosureIdx [ X ] ] ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } for ( NodeAddr < BlockNode * > PBA : Preds ) { for ( unsigned X = <NUM_LIT> ; X != CS ; ++ X ) { RegisterRef RR = MaxRefs [ ClosureIdx [ X ] ] ; NodeAddr < PhiUseNode * > PUA = newPhiUse ( PA , RR , PBA ) ; PA . Addr -> addMember ( PUA , * this ) ; } } auto Begin = MaxRefs . begin


<s> Func = newFunc ( & MF ) ; if ( MF . empty ( ) ) return ; for ( MachineBasicBlock & B : MF ) { NodeAddr < BlockNode * > BA = newBlock ( Func , & B ) ; BlockNodes . insert ( std :: make_pair ( & B , BA ) ) ; for ( MachineInstr & I : B ) { if ( I . isDebugValue ( ) ) continue ; buildStmt ( BA , I ) ; } } NodeAddr < BlockNode * > EA = Func . Addr -> getEntryBlock ( * this ) ; NodeList Blocks = Func . Addr -> members ( * this ) ; BlockRefsMap RefM ; buildBlockRefs ( EA , RefM ) ; MachineRegisterInfo & MRI = MF . getRegInfo ( ) ; MachineBasicBlock & EntryB = * EA . Addr -> getCode ( ) ; assert ( EntryB . pred_empty ( ) && <STR_LIT> ) ; for ( auto I = MRI . livein_begin ( ) , E = MRI . livein_end ( ) ; I != E ; ++ I ) LiveIns . insert ( RegisterRef ( I -> first ) ) ; if ( MRI . tracksLiveness ( ) ) { for ( auto I : EntryB . liveins ( ) ) LiveIns . insert ( RegisterRef ( I . PhysReg , I . LaneMask ) ) ; } for ( auto I = LiveIns . rr_begin ( ) , E = LiveIns . rr_end ( ) ; I != E ; ++ I ) { RegisterRef RR = * I ; NodeAddr < PhiNode * > PA = newPhi ( EA ) ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR ,


<s> void DataFlowGraph :: buildBlockRefs ( NodeAddr < BlockNode * > BA , BlockRefsMap & RefM ) { RegisterSet & Refs = RefM [ BA . Id ] ; MachineDomTreeNode * N = MDT . getNode ( BA . Addr -> getCode ( ) ) ; assert ( N ) ; for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock ( ) ; NodeAddr < BlockNode * > SBA = findBlock ( SB ) ; buildBlockRefs ( SBA , RefM ) ; const RegisterSet & RefsS = RefM [ SBA . Id ] ; Refs . insert ( RefsS . begin ( ) , RefsS . end (


<s> auto DFLoc = MDF . find ( BB ) ; if ( DFLoc == MDF . end ( ) || DFLoc -> second . empty ( ) ) return ; RegisterSet Defs ; for ( NodeAddr < InstrNode * > IA : BA . Addr -> members ( * this ) ) for ( NodeAddr < RefNode * > RA : IA . Addr -> members_if ( IsDef , * this ) ) Defs . insert ( RA . Addr -> getRegRef ( * this ) ) ; const MachineDominanceFrontier :: DomSetType & DF = DFLoc -> second ; SetVector < MachineBasicBlock * > IDF ( DF . begin ( ) , DF . end ( ) ) ; for ( unsigned i = <NUM_LIT> ; i < IDF . size ( ) ; ++ i ) { auto F = MDF . find ( IDF [ i ] ) ; if ( F != MDF . end ( ) ) IDF . insert ( F -> second . begin ( ) , F -> second . end ( ) ) ; } RegisterSet & Refs = RefM [ BA . Id ] ; for ( auto DB : IDF ) { NodeAddr < BlockNode * > DBA = findBlock ( DB ) ; const RegisterSet & RefsD = RefM [ DBA .


<s> auto BA = newBlock ( Func , & B ) ; BlockNodes . insert ( std :: make_pair ( & B , BA ) ) ; for ( auto & I : B ) { if ( I . isDebugValue ( ) ) continue ; buildStmt ( BA , I ) ; } } NodeAddr < BlockNode * > EA = Func . Addr -> getEntryBlock ( * this ) ; BlockRefsMap RefM ; buildBlockRefs ( EA , RefM ) ; MachineRegisterInfo & MRI = MF


<s> for ( auto I = MRI . livein_begin ( ) , E = MRI . livein_end ( ) ; I != E ; ++ I ) { NodeAddr < PhiNode * > PA = newPhi ( EA ) ; RegisterRef RR = { I -> first , <NUM_LIT> } ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } BlockRefsMap PhiM ; auto Blocks = Func . Addr -> members ( * this ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) recordDefsForDF ( PhiM , RefM , BA ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) buildPhis ( PhiM , RefM , BA ) ; DefStackMap DM ; linkBlockRefs ( DM , EA ) ; if ( ! ( Options & BuildOptions :: KeepDeadPhis )


<s> void DataFlowGraph :: buildPhis ( BlockRefsMap & PhiM , BlockRefsMap & RefM , NodeAddr < BlockNode * > BA ) { auto HasDF = PhiM . find ( BA . Id ) ; if ( HasDF == PhiM . end ( ) || HasDF -> second . empty ( ) ) return ; auto MaxCoverIn = [ this ] ( RegisterRef RR , RegisterSet & RRs ) -> RegisterRef { for ( auto I : RRs ) if ( I != RR && RAI . covers ( I , RR ) ) RR = I ; return RR ; } ; RegisterSet MaxDF ; for ( auto I : HasDF -> second ) MaxDF . insert ( MaxCoverIn ( I , HasDF -> second ) ) ; std :: vector < RegisterRef > MaxRefs ; auto & RefB = RefM [ BA . Id ] ; for ( auto I : MaxDF ) MaxRefs . push_back ( MaxCoverIn ( I , RefB ) ) ; std :: sort ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; auto NewEnd = std :: unique ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; MaxRefs . erase ( NewEnd , MaxRefs . end ( ) ) ; auto Aliased = [ this , & MaxRefs ] ( RegisterRef RR , std :: vector < unsigned > & Closure ) -> bool { for ( auto I : Closure ) if ( RAI . alias ( RR , MaxRefs [ I ] ) ) return true ; return false ; } ; std :: vector < NodeId > PredList ; const MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( auto PB : MBB -> predecessors ( ) ) { auto B = findBlock ( PB ) ; PredList . push_back ( B . Id ) ; } while ( ! MaxRefs . empty


<s> MachineDomTreeNode * N = MDT . getNode ( BA . Addr -> getCode ( ) ) ; for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock ( ) ; auto SBA = findBlock ( SB ) ; linkBlockRefs ( DefM , SBA ) ; } auto IsUseForBA = [ BA ] ( NodeAddr < NodeBase * > NA ) -> bool { if ( NA . Addr -> getKind ( ) != NodeAttrs :: Use ) return false ; assert ( NA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ; NodeAddr < PhiUseNode * > PUA = NA ; return PUA . Addr -> getPredecessor ( ) == BA . Id ; } ; MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( auto SB : MBB -> successors ( ) ) { auto SBA = findBlock ( SB ) ; for ( NodeAddr < InstrNode * > IA : SBA . Addr -> members_if ( IsPhi , * this ) ) { for ( auto U : IA . Addr -> members_if ( IsUseForBA , * this ) ) { NodeAddr < PhiUseNode * > PUA = U ; RegisterRef RR = PUA . Addr


<s> pushDefs ( IA , DefM ) ; } MachineDomTreeNode * N = MDT . getNode ( BA . Addr -> getCode ( ) ) ; for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock ( ) ; auto SBA = findBlock ( SB ) ; linkBlockRefs ( DefM , SBA ) ; } auto IsUseForBA = [ BA ] ( NodeAddr < NodeBase * > NA ) -> bool { if ( NA . Addr -> getKind ( ) != NodeAttrs :: Use ) return false ; assert ( NA . Addr -> getFlags ( )


<s> MachineBasicBlock * BB = BA . Addr -> getCode ( ) ; assert ( BB ) ; auto DFLoc = MDF . find ( BB ) ; if ( DFLoc == MDF . end ( ) || DFLoc -> second . empty ( ) ) return ; RegisterSet Defs ; for ( auto I : BA . Addr -> members ( * this ) ) { assert ( I . Addr -> getType ( ) == NodeAttrs :: Code ) ; assert ( I . Addr -> getKind ( ) == NodeAttrs :: Phi || I . Addr -> getKind ( ) == NodeAttrs :: Stmt ) ; NodeAddr < InstrNode * > IA = I ; for ( NodeAddr < RefNode * > RA : IA . Addr -> members_if ( IsDef , * this ) ) Defs . insert ( RA . Addr -> getRegRef ( ) ) ; } const MachineDominanceFrontier :: DomSetType & DF = DFLoc -> second ; SetVector < MachineBasicBlock * > IDF ( DF . begin ( ) , DF . end ( ) ) ; for ( unsigned i = <NUM_LIT> ; i < IDF . size ( ) ; ++ i ) { auto F = MDF . find ( IDF [ i ] ) ; if ( F != MDF . end ( ) ) IDF . insert ( F -> second . begin ( ) , F -> second .


<s> if ( AR . Reg == BR . Reg ) { LaneBitmask M = AR . Mask & BR . Mask ; return M . any ( ) ? RegisterRef ( AR . Reg , M ) : RegisterRef ( ) ; } RegisterRef NAR = PRI . normalize ( AR ) ; RegisterRef NBR = PRI . normalize ( BR


<s> RegisterRef RR = { I -> first , <NUM_LIT> } ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } BlockRefsMap PhiM ; auto Blocks = Func . Addr -> members ( * this ) ; for ( NodeAddr < BlockNode * > BA : Blocks

) recordDefsForDF ( PhiM , RefM , BA ) ;


<s> RegisterRef RR = { I -> first , <NUM_LIT> } ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } BlockRefsMap PhiM ; auto Blocks = Func . Addr -> members ( * this ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) recordDefsForDF (


<s> if ( const uint16_t * ImpU = In . getDesc ( ) . getImplicitUses ( ) ) while ( uint16_t R = * ImpU ++ ) ImpUses . insert ( { R , <NUM_LIT> } ) ; bool NeedsImplicit = isCall ( In ) || In . isInlineAsm ( ) || In . isReturn ( ) ; bool IsPredicated = TII . isPredicated ( In ) ; unsigned NumOps = In . getNumOperands ( ) ; RegisterSet DoneDefs ; for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || ! Op . isImplicit ( ) ) continue ; RegisterRef RR = { Op . getReg ( ) , Op . getSubReg ( ) } ; if ( ! NeedsImplicit && ! ImpDefs . count ( RR ) ) continue ; if ( DoneDefs . count ( RR ) ) continue ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) Flags |= NodeAttrs :: Preserving ; if ( TOI . isClobbering ( In , OpN


<s> if ( In . isCall ( ) || In . isReturn ( ) || In . isInlineAsm ( ) ) return true ; if ( In . isBranch ( ) ) for ( auto & O : In . operands ( ) ) if ( O . isGlobal ( ) || O . isSymbol ( ) ) return true ; const MCInstrDesc & D = In . getDesc ( ) ; if ( ! D . getImplicitDefs ( ) && ! D . getImplicitUses ( ) ) return false ; const MachineOperand & Op = In


<s> if ( In . isBranch ( ) ) for ( auto & O : In . operands ( ) ) if ( O . isGlobal ( ) || O . isSymbol ( ) ) return true ; const MCInstrDesc & D = In . getDesc ( ) ; if ( ! D . getImplicitDefs ( ) && ! D . getImplicitUses


<s> buildBlockRefs ( EA , RefM ) ; MachineRegisterInfo & MRI = MF . getRegInfo ( ) ; MachineBasicBlock & EntryB = * EA . Addr -> getCode ( ) ; assert ( EntryB . pred_empty ( ) && <STR_LIT> ) ; for ( auto I = MRI . livein_begin ( ) , E = MRI . livein_end ( ) ; I != E ; ++ I ) LiveIns . insert ( RegisterRef ( I -> first ) ) ; for ( auto I : EntryB . liveins ( ) ) LiveIns . insert ( RegisterRef ( I . PhysReg , I . LaneMask ) ) ; for ( std :: pair < RegisterId , LaneBitmask > P : LiveIns ) { NodeAddr < PhiNode * > PA = newPhi ( EA ) ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; RegisterRef RR ( P . first , P . second ) ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , * this ) ; } RegisterSet EHRegs = getLandingPadLiveIns ( ) ; if ( ! EHRegs . empty ( ) ) { for ( NodeAddr < BlockNode * > BA : Blocks ) { const MachineBasicBlock & B = * BA . Addr -> getCode ( ) ; if ( ! B . isEHPad ( ) ) continue ; NodeList Preds ; for ( MachineBasicBlock * PB : B . predecessors ( ) ) Preds . push_back ( findBlock ( PB ) ) ; for ( RegisterRef RR : EHRegs ) { NodeAddr < PhiNode * > PA = newPhi ( BA ) ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef (


<s> NodeAddr < StmtNode * > SA = newStmt ( BA , & In ) ; auto isCall = [ ] ( const MachineInstr & In ) -> bool { if ( In . isCall ( ) ) return true ; if ( In . isBranch ( ) ) for ( const MachineOperand & Op : In . operands ( ) ) if ( Op . isGlobal ( ) || Op . isSymbol ( ) ) return true ; return false ; } ; auto isDefUndef = [ this ] ( const MachineInstr & In , RegisterRef DR ) -> bool { for ( const MachineOperand & UseOp : In . operands ( ) ) { if ( ! UseOp . isReg ( ) || ! UseOp . isUse ( ) || UseOp . isUndef ( ) ) continue ; RegisterRef UR = makeRegRef ( UseOp . getReg ( ) , UseOp . getSubReg ( ) ) ; if ( PRI . alias ( DR , UR ) ) return false ; } return true ; } ; RegisterSet ImpUses , ImpDefs ; if ( const uint16_t * ImpD = In . getDesc ( ) . getImplicitDefs ( ) ) while ( uint16_t R = * ImpD ++ ) ImpDefs . insert ( RegisterRef ( R ) ) ; if ( const uint16_t * ImpU = In . getDesc ( ) . getImplicitUses ( ) ) while ( uint16_t R = * ImpU ++ ) ImpUses . insert ( RegisterRef ( R ) ) ; bool IsCall = isCall ( In ) ; bool NeedsImplicit = IsCall || In . isInlineAsm ( ) || In . isReturn ( ) ; bool IsPredicated = TII . isPredicated ( In ) ; unsigned NumOps = In . getNumOperands ( ) ; RegisterSet DoneDefs ; for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; RegisterRef RR = makeRegRef ( Op . getReg ( ) , Op . getSubReg ( ) ) ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) { Flags |= NodeAttrs :: Preserving ; if ( isDefUndef ( In , RR ) ) Flags |= NodeAttrs :: Undef ; } if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; if ( IsCall && Op . isDead ( ) )


<s> RegisterSet AS ; assert ( TargetRegisterInfo :: isPhysicalRegister ( Reg ) ) ; for ( MCRegAliasIterator AI ( Reg , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . insert ( RegisterRef ( * AI ) ) ; return


<s> assert ( TargetRegisterInfo :: isPhysicalRegister ( Reg ) ) ; for ( MCRegAliasIterator AI ( Reg , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . insert ( RegisterRef ( * AI ) ) ; return


<s> RegisterRef DataFlowGraph :: makeRegRef ( unsigned Reg , unsigned Sub ) const { assert ( TargetRegisterInfo :: isPhysicalRegister ( Reg ) ) ; if ( Sub != <NUM_LIT> ) Reg = TRI . getSubReg ( Reg , Sub ) ; return RegisterRef ( Reg


<s> SuperReg = * SR ; } uint32_t Sub = TRI . getSubRegIndex ( SuperReg , RR . Reg ) ; const TargetRegisterClass & RC = * TRI . getMinimalPhysRegClass ( RR


<s> while ( true ) { MCSuperRegIterator SR ( SuperReg , & TRI , false ) ; if ( ! SR . isValid ( ) ) break ; SuperReg = * SR ; } uint32_t Sub = TRI . getSubRegIndex ( SuperReg , RR . Reg


<s> dbgs ( ) << <STR_LIT> << Print < RegisterRef > ( RR , * this ) << <STR_LIT> << * MI << <STR_LIT> << MI -> getParent ( ) -> getNumber ( ) << '\n' ; llvm_unreachable ( nullptr ) ; } DefM [ RR . Reg ] . push ( DA ) ; for ( RegisterRef A : getAliasSet ( RR . Reg ) ) { assert ( A != RR ) ; DefM [ A . Reg ] . push ( DA ) ; } for ( NodeAddr < NodeBase * > T : Rel ) Visited


<s> LaneBitmask M = AR . Mask & BR . Mask ; return M . any ( ) ? RegisterRef ( AR . Reg , M ) : RegisterRef ( ) ; } RegisterRef NAR = normalizeRef ( AR


<s> RegisterRef DataFlowGraph :: restrictRef ( RegisterRef AR , RegisterRef BR ) const { if ( AR . Reg == BR . Reg ) { LaneBitmask M = AR . Mask & BR . Mask ; return M . any ( ) ? RegisterRef ( AR . Reg , M ) : RegisterRef ( ) ; } RegisterRef NAR = normalizeRef


<s> assert ( TargetRegisterInfo :: isPhysicalRegister ( RA . Reg ) ) ; assert ( TargetRegisterInfo :: isPhysicalRegister ( RB . Reg ) ) ; MCRegUnitMaskIterator UMA ( RA . Reg , & TRI ) ; MCRegUnitMaskIterator UMB ( RB . Reg , & TRI ) ; while ( UMA . isValid ( ) && UMB . isValid ( ) ) { std :: pair < uint32_t , LaneBitmask > PA = * UMA ; std :: pair < uint32_t , LaneBitmask > PB = * UMB ; if ( PA . first == PB . first ) { if ( PA . second . none ( ) || PB . second . none ( ) ) return true ; LaneBitmask LA = PA . second & RA . Mask ; LaneBitmask LB = PB . second & RB . Mask ; if ( LA . any ( ) && LB . any ( ) ) { unsigned Root = * MCRegUnitRootIterator ( PA . first , & TRI ) ; uint32_t SubA = TRI . getSubRegIndex ( RA . Reg , Root ) ; uint32_t SubB = TRI . getSubRegIndex ( RB . Reg , Root ) ; const TargetRegisterClass & RC = * TRI . getMinimalPhysRegClass ( Root ) ; LaneBitmask MaskA = TRI . reverseComposeSubRegIndexLaneMask ( SubA , LA ) ; LaneBitmask MaskB = TRI . reverseComposeSubRegIndexLaneMask ( SubB


<s> for ( MachineBasicBlock & B : MF ) { NodeAddr < BlockNode * > BA = newBlock ( Func , & B ) ; BlockNodes . insert ( std :: make_pair ( & B , BA ) ) ; for ( MachineInstr & I : B ) { if ( I . isDebugValue ( ) ) continue ; buildStmt ( BA , I ) ; } } NodeAddr < BlockNode * > EA = Func . Addr -> getEntryBlock ( * this ) ; NodeList Blocks = Func . Addr -> members ( * this ) ; BlockRefsMap RefM ; buildBlockRefs ( EA , RefM ) ; MachineRegisterInfo & MRI = MF . getRegInfo ( ) ; for ( auto I = MRI . livein_begin ( ) , E = MRI . livein_end ( ) ; I != E ; ++ I ) { NodeAddr < PhiNode * > PA = newPhi ( EA ) ; RegisterRef RR = RegisterRef ( I


<s> NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps ; ++ OpN ) { MachineOperand & Op = In . getOperand ( OpN ) ; if ( ! Op . isReg ( ) || ! Op . isDef ( ) || ! Op . isImplicit ( ) ) continue ; RegisterRef RR = makeRegRef ( Op . getReg ( ) , Op . getSubReg ( ) ) ; if ( ! NeedsImplicit && ! ImpDefs . count ( RR ) ) continue ; if ( DoneDefs . count ( RR ) ) continue ; uint16_t Flags = NodeAttrs :: None ; if ( TOI . isPreserving ( In , OpN ) ) { Flags |= NodeAttrs :: Preserving ; if ( isDefUndef ( In , RR ) ) Flags |= NodeAttrs :: Undef ; } if ( TOI . isClobbering ( In , OpN ) ) Flags |= NodeAttrs :: Clobbering ; if ( TOI . isFixedReg ( In , OpN ) ) Flags |= NodeAttrs :: Fixed ; if ( IsCall && Op . isDead ( ) ) Flags |= NodeAttrs :: Dead ; NodeAddr < DefNode * > DA = newDef ( SA , Op , Flags ) ; SA . Addr -> addMember ( DA , * this ) ; DoneDefs . insert ( RR ) ; } for ( unsigned OpN = <NUM_LIT> ; OpN < NumOps


<s> return RegisterRef ( T . begin ( ) -> first , T . begin ( ) ->


<s> RegisterRef NR = normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F != Masks . end ( ) ) { if ( ( F -> second & NR . Mask ) . any ( ) ) return true ; } if ( CheckUnits ) { for ( MCRegUnitIterator U ( RR . Reg , & TRI ) ; U . isValid ( ) ; ++ U ) if ( ExpAliasUnits . test ( * U ) ) return true ; } return


<s> bool RegisterAggr :: hasCoverOf ( RegisterRef RR ) const { RegisterRef NR = normalize ( RR ) ; if ( NR . Mask . none ( ) ) return


<s> for ( std :: pair < RegisterId , LaneBitmask > P : RG . Masks ) insert ( RegisterRef ( P . first , P . second ) ) ; return * this


<s> bool Cover = Defs . insert ( QR ) . hasCoverOf ( RR ) ; if ( Alias ) { if ( Cover ) break ; continue ; } NodeAddr < DefNode * > RDA = * I ; if ( TAP . Id == <NUM_LIT> ) { TAP = TA ; } else { TAP . Addr -> setFlags ( TAP . Addr -> getFlags ( ) | NodeAttrs


<s> RegisterRef RR = TA . Addr -> getRegRef ( * this ) ; NodeAddr < T > TAP ; RegisterAggr Defs ( TRI ) ; for ( auto I = DS . top ( ) , E = DS . bottom ( ) ; I != E ; I . down ( ) ) { RegisterRef QR = I -> Addr -> getRegRef ( * this ) ; bool Alias = Defs . hasAliasOf ( QR ) ; bool Cover = Defs . insert ( QR ) . hasCoverOf ( RR ) ; if ( Alias ) { if ( Cover ) break ; continue ; } NodeAddr < DefNode * > RDA = * I ; if ( TAP . Id == <NUM_LIT> ) { TAP =


<s> while ( true ) { MCSuperRegIterator SR ( SuperReg , & TRI , false ) ; if ( ! SR . isValid ( ) ) break ; SuperReg = * SR ; } const TargetRegisterClass & RC = * TRI . getMinimalPhysRegClass ( RR . Reg ) ; LaneBitmask Common = RR . Mask & RC . LaneMask ; uint32_t Sub = TRI . getSubRegIndex ( SuperReg , RR . Reg ) ; LaneBitmask SuperMask = TRI . composeSubRegIndexLaneMask ( Sub , Common ) ; return RegisterRef ( SuperReg , SuperMask


<s>
OS
<<
'{'
;
for
(
auto
I
:
Masks
)
OS
<<
' '
<<
PrintReg
(
I
.
first
,
&
TRI
)


<s> auto ML = getLastMember ( G ) ; if ( ML . Id != <NUM_LIT> ) { ML . Addr -> append ( NA ) ; } else { Code . FirstM = NA . Id ; NodeId Self = G .


<s> if ( Code . LastM == MA . Id ) Code .


<s> assert ( M . Addr -> getType ( ) == NodeAttrs :: Code ) ; if ( M . Addr -> getKind ( ) == NodeAttrs :: Stmt ) { Code . FirstM = PA . Id ; PA . Addr -> setNext ( M . Id ) ; } else { assert ( M . Addr -> getKind ( ) == NodeAttrs :: Phi ) ; NodeAddr < NodeBase * > MN = M ; do { M = MN


<s> unsigned OA = TRI . getSubRegIdxOffset ( RA . Sub ) ; unsigned SB = TRI . getSubRegIdxSize ( RB . Sub ) ; unsigned OB = TRI . getSubRegIdxOffset ( RB . Sub ) ; if ( OA <= OB && OA + SA > OB ) return true ; if ( OB <= OA && OB + SB > OA ) return true ; return false ; } assert ( PhysA && PhysB ) ; ( void ) PhysA , ( void ) PhysB ; unsigned A = RA . Sub ? TRI . getSubReg ( RA . Reg , RA . Sub ) : RA . Reg ; unsigned B = RB . Sub ? TRI . getSubReg ( RB . Reg , RB . Sub ) : RB . Reg ; for ( MCRegAliasIterator I ( A , & TRI , true ) ; I . isValid ( ) ; ++ I ) if


<s> NodeId Nx = Next ; if ( Next != NA . Id ) { Next = NA


<s> assert ( N ) ; for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock ( ) ; auto SBA = Func . Addr -> findBlock ( SB , * this ) ; buildBlockRefs ( SBA , RefM ) ; const auto & SRs = RefM [ SBA . Id ] ; Refs . insert ( SRs . begin ( ) , SRs . end ( ) ) ; } for ( NodeAddr < InstrNode * > IA : BA . Addr -> members ( * this ) ) for ( NodeAddr < RefNode * > RA

: IA . Addr -> members ( * this ) ) Refs . insert ( RA . Addr -> getRegRef ( ) ) ;


<s> auto & Refs = RefM [ BA . Id ] ; MachineDomTreeNode * N = MDT . getNode ( BA . Addr -> getCode ( ) ) ; assert ( N ) ; for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock


<s> std :: sort ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; auto NewEnd = std :: unique ( MaxRefs . begin ( ) , MaxRefs . end ( ) ) ; MaxRefs . erase ( NewEnd , MaxRefs . end ( ) ) ; auto Aliased = [ this , & MaxRefs ] ( RegisterRef RR , std :: vector < unsigned > & Closure ) -> bool { for ( auto I : Closure ) if ( RAI . alias ( RR , MaxRefs [ I ] ) ) return true ; return false ; } ; std :: vector < NodeId > PredList ; const MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( auto PB : MBB -> predecessors ( ) ) { auto B = Func . Addr -> findBlock ( PB , * this ) ; PredList . push_back ( B . Id ) ; } while ( ! MaxRefs . empty ( ) ) { std :: vector < unsigned > ClosureIdx = { <NUM_LIT> } ; for ( unsigned i = <NUM_LIT> ; i != MaxRefs . size ( ) ; ++ i ) if ( Aliased ( MaxRefs [ i ] , ClosureIdx ) ) ClosureIdx . push_back ( i ) ; unsigned CS = ClosureIdx . size ( ) ; NodeAddr < PhiNode * > PA = newPhi ( BA ) ; for ( unsigned X = <NUM_LIT> ; X != CS ; ++ X ) { RegisterRef RR = MaxRefs [ ClosureIdx [ X ] ] ; uint16_t PhiFlags = NodeAttrs :: PhiRef | NodeAttrs :: Preserving ; NodeAddr < DefNode * > DA = newDef ( PA , RR , PhiFlags ) ; PA . Addr -> addMember ( DA , *


<s> Blocks . clear ( ) ; ActiveEnd = nullptr


<s> return KB == Phi || KB == Stmt ; case Phi : case Stmt : return type ( B ) == Ref ; } return


<s> bool RegisterAliasInfo :: covers ( const RegisterSet & RRs , RegisterRef RR ) const { if ( RRs . count ( RR ) ) return true ; if ( TargetRegisterInfo :: isVirtualRegister ( RR . Reg ) ) { if ( RR . Sub != <NUM_LIT> ) return RRs . count ( { RR . Reg , <NUM_LIT> } ) ; return false ; } unsigned Reg = RR . Sub == <NUM_LIT> ? RR . Reg : TRI . getSubReg ( RR . Reg , RR . Sub ) ; for ( MCSuperRegIterator SR ( Reg , & TRI ) ; SR . isValid ( ) ; ++ SR ) if ( RRs . count ( { *


<s> Iterator & down ( ) { Pos = DS . nextDown ( Pos ) ; return


<s> Iterator & down ( ) { Pos = DS . nextDown ( Pos ) ;


<s> for ( MCRegAliasIterator AI ( R , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . push_back ( RegisterRef (


<s> if ( TargetRegisterInfo :: isVirtualRegister ( RR . Reg ) ) return AS ; assert ( TargetRegisterInfo :: isPhysicalRegister ( RR . Reg ) ) ; unsigned R = RR . Reg ; if ( RR . Sub ) R = TRI . getSubReg ( RR . Reg , RR . Sub ) ; for ( MCRegAliasIterator AI ( R , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . push_back ( RegisterRef ( { * AI , <NUM_LIT> } ) ) ; return


<s> const MachineDominanceFrontier & getDF ( ) const { return MDF


<s> const MachineDominatorTree & getDT (


<s> NodeAddr < BlockNode * > FuncNode :: getEntryBlock ( const DataFlowGraph & G ) { MachineBasicBlock * EntryB = & getCode ( ) -> front ( ) ; return findBlock ( EntryB , G )


<s> MachineBasicBlock * EntryB = & getCode ( ) -> front ( ) ; return findBlock ( EntryB , G


<s> NodeAddr < NodeBase * > CodeNode :: getFirstMember


<s> NodeAddr < FuncNode * > getFunc ( ) const { return Func


<s> NodeAddr < NodeBase * > CodeNode :: getLastMember ( const DataFlowGraph & G ) const { if ( Code . LastM == <NUM_LIT> )


<s> auto NA = G . addr < NodeBase * > ( getNext ( ) ) ; while ( NA . Addr != this ) { if ( NA . Addr -> getType ( ) == NodeAttrs :: Ref ) { NodeAddr < RefNode * > RA = NA ; if ( RA . Addr -> getRegRef ( ) == RR && P ( NA ) ) return NA ; if ( NextOnly ) break ; NA = G . addr < NodeBase * > ( NA . Addr -> getNext ( ) ) ; } else { assert ( NA . Addr -> getType ( ) == NodeAttrs ::


<s> assert ( ! ( getFlags ( ) & NodeAttrs :: PhiRef


<s> NodeId getReachedDef ( ) const { return Ref . Def .


<s> NodeId getReachingDef ( ) const { return Ref . RD


<s> NodeList Refs ; NodeId Start = RA . Id ; do { Refs . push_back ( RA ) ; RA = getNextRelated ( IA , RA ) ; } while ( RA . Id != <NUM_LIT>


<s> const TargetInstrInfo & getTII ( ) const { return


<s> const TargetInstrInfo & getTII ( ) const


<s> uint16_t getType ( ) const { return NodeAttrs :: type


<s> uint16_t getType ( ) const { return NodeAttrs :: type (


<s> memset ( this , <NUM_LIT> , sizeof * this


<s> bool TargetOperandInfo :: isClobbering ( const MachineInstr & In ,


<s> static bool IsCode ( const NodeAddr < NodeBase * > BA ) { return BA . Addr -> getType ( ) == NodeAttrs :: Code && BA


<s> static bool IsCode ( const NodeAddr < NodeBase * > BA ) { return BA . Addr -> getType ( ) == NodeAttrs :: Code && BA . Addr -> getKind (


<s> if ( Op . getSubReg ( ) != <NUM_LIT> ) return false ; unsigned Reg = Op . getReg ( ) ; const MCPhysReg * ImpR = Op . isDef ( ) ?

D . getImplicitDefs ( ) : D . getImplicitUses ( ) ;


<s> const MCPhysReg * ImpR = Op . isDef ( ) ? D . getImplicitDefs ( ) : D . getImplicitUses ( ) ; if ( ! ImpR ) return false ; while ( * ImpR ) if ( * ImpR ++ ==


<s> return BA . Addr -> getType ( ) == NodeAttrs :: Code && BA . Addr ->


<s> bool TargetOperandInfo :: isPreserving ( const MachineInstr & In , unsigned OpNum ) const { return TII . isPredicated (


<s> return BA . Addr -> getType ( ) == NodeAttrs :: Ref &&


<s> return BA . Addr -> getType ( ) == NodeAttrs :: Ref && BA . Addr -> getKind ( )


<s> MachineBasicBlock * MBB = BA . Addr -> getCode ( ) ; for ( auto SB : MBB -> successors ( ) ) { auto SBA = Func . Addr -> findBlock ( SB , * this ) ; for ( NodeAddr < InstrNode * > IA : SBA . Addr -> members_if ( IsPhi , * this ) ) { for ( auto U : IA . Addr -> members_if ( IsUseForBA , * this ) ) { NodeAddr < PhiUseNode * > PUA = U ; RegisterRef RR = PUA . Addr -> getRegRef ( ) ; linkRefUp < UseNode * > ( IA , PUA , DefM [ RR


<s> for ( auto I : * N ) { MachineBasicBlock * SB = I -> getBlock ( ) ; auto SBA = Func . Addr -> findBlock ( SB , * this ) ; linkBlockRefs ( DefM , SBA ) ; } auto IsUseForBA = [ BA ] ( NodeAddr < NodeBase * > NA ) -> bool { if ( NA . Addr -> getKind ( ) != NodeAttrs :: Use ) return false ; assert ( NA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ; NodeAddr < PhiUseNode * > PUA = NA ; return PUA . Addr -> getPredecessor ( ) == BA . Id ; } ; MachineBasicBlock * MBB = BA . Addr -> getCode


<s> } NodeAddr < DefNode * > RDA = * I ; if ( TAP . Id == <NUM_LIT> ) { TAP = TA ; } else { TAP . Addr -> setFlags ( TAP . Addr -> getFlags ( ) | NodeAttrs :: Shadow ) ; TAP = getNextShadow ( IA , TAP , true ) ; } TAP . Addr -> linkToDef ( TAP . Id , RDA ) ; if ( PrecUp


<s> void UseNode :: linkToDef ( NodeId Self , NodeAddr < DefNode * > DA ) { Ref . RD = DA . Id ; Ref . Sib = DA . Addr -> getReachedUse ( ) ; DA . Addr ->


<s> void UseNode :: linkToDef ( NodeId Self , NodeAddr < DefNode * > DA ) { Ref . RD = DA . Id ; Ref . Sib = DA . Addr -> getReachedUse ( ) ; DA . Addr -> setReachedUse


<s> NodeAddr < RefNode * > NA ; NodeId Start = RA . Id ; while ( true ) { NA = getNextRelated ( IA , RA ) ; if ( NA . Id == <NUM_LIT> || NA . Id == Start ) break ; if ( P (


<s> uint32_t makeId ( uint32_t Block , uint32_t Index ) const { return ( ( Block << BitsPerIndex ) | Index


<s> uint32_t makeId ( uint32_t Block , uint32_t Index ) const { return ( ( Block << BitsPerIndex ) | Index )


<s> NodeList CodeNode :: members ( const DataFlowGraph & G ) const { static auto True = [ ] ( NodeAddr < NodeBase * > ) ->


<s> NodeList CodeNode :: members ( const DataFlowGraph & G ) const { static auto True = [ ] ( NodeAddr < NodeBase * > ) -> bool { return true ; } ; return members_if ( True


<s> NodeList CodeNode :: members_if ( Predicate P , const DataFlowGraph & G ) const { NodeList MM ; auto M = getFirstMember ( G ) ; if ( M . Id == <NUM_LIT> ) return


<s> NodeAddr < NodeBase * > NodeAllocator :: New ( ) { if ( needNewBlock ( ) ) startNewBlock ( ) ; uint32_t ActiveB = Blocks . size ( ) - <NUM_LIT> ; uint32_t Index = ( ActiveEnd - Blocks [ ActiveB ] ) / NodeMemSize ; NodeAddr < NodeBase * > NA = { reinterpret_cast < NodeBase * > ( ActiveEnd )


<s> BA . Addr -> setCode ( BB ) ; Owner . Addr -> addMember ( BA ,


<s> NodeAddr < BlockNode * > DataFlowGraph :: newBlock ( NodeAddr < FuncNode * > Owner , MachineBasicBlock * BB ) { NodeAddr < BlockNode * > BA = newNode ( NodeAttrs :: Code | NodeAttrs :: Block ) ; BA . Addr -> setCode ( BB ) ; Owner . Addr -> addMember ( BA , * this ) ; return


<s> NodeAddr < DefNode * > DA = newNode ( NodeAttrs :: Ref | NodeAttrs :: Def | Flags ) ; assert ( Flags & NodeAttrs :: PhiRef ) ; DA . Addr -> setRegRef ( RR ) ; return DA


<s> assert ( Flags & NodeAttrs :: PhiRef ) ; DA . Addr -> setRegRef ( RR ) ; return DA


<s> NodeAddr < FuncNode * > DataFlowGraph :: newFunc ( MachineFunction * MF ) { NodeAddr < FuncNode * > FA = newNode ( NodeAttrs :: Code | NodeAttrs :: Func ) ; FA . Addr -> setCode


<s> P . Addr -> init ( ) ; P . Addr -> setAttrs ( Attrs


<s> P . Addr -> init ( ) ; P . Addr -> setAttrs ( Attrs ) ; return


<s> assert ( Flags & NodeAttrs :: PhiRef ) ; PUA . Addr -> setRegRef ( RR ) ; PUA . Addr -> setPredecessor ( PredB . Id ) ; return PUA


<s> NodeAddr < StmtNode * > SA = newNode ( NodeAttrs :: Code | NodeAttrs :: Stmt ) ; SA . Addr -> setCode (


<s> NodeAddr < StmtNode * > DataFlowGraph :: newStmt ( NodeAddr < BlockNode * > Owner , MachineInstr * MI ) { NodeAddr < StmtNode * > SA = newNode ( NodeAttrs :: Code | NodeAttrs :: Stmt ) ; SA . Addr -> setCode (


<s> NodeAddr < UseNode * > UA = newNode ( NodeAttrs :: Ref | NodeAttrs :: Use | Flags ) ; UA . Addr -> setRegRef ( & Op


<s> NodeAllocator ( uint32_t NPB = <NUM_LIT> ) : NodesPerBlock ( NPB ) , BitsPerIndex ( Log2_32 ( NPB ) ) ,

IndexMask ( ( <NUM_LIT> << BitsPerIndex ) - <NUM_LIT> ) , ActiveEnd ( nullptr ) {


<s> OS << Print < NodeId > ( RA . Id , G ) << '<' << Print < RegisterRef > ( RA . Addr -> getRegRef ( )


<s> T ptr ( NodeId N ) const { return static_cast < T > ( ptr


<s> void push ( NodeAddr < DefNode * > DA ) { Stack . push_back (


<s> RegisterRef RR = PDA . Addr -> getRegRef ( ) ; if ( ! Defined . insert ( RR ) . second ) { auto * MI = NodeAddr < StmtNode * > ( IA ) . Addr -> getCode ( ) ; dbgs ( ) << <STR_LIT> << Print < RegisterRef > ( RR , * this ) << <STR_LIT> << * MI << <STR_LIT> << MI -> getParent ( ) -> getNumber ( ) << '\n' ; llvm_unreachable ( nullptr ) ; } DefM [ RR ] . push ( DA ) ; for ( auto A : RAI . getAliasSet ( RR ) ) { assert ( A != RR ) ; DefM [ A ] .


<s> NodeList Rel = getRelatedRefs ( IA , DA ) ; NodeAddr < DefNode * > PDA = Rel . front ( ) ; RegisterRef RR = PDA . Addr -> getRegRef ( ) ; if ( ! Defined . insert ( RR ) . second ) { auto * MI = NodeAddr < StmtNode * > ( IA ) . Addr -> getCode ( ) ; dbgs ( ) << <STR_LIT> << Print < RegisterRef > ( RR , * this ) << <STR_LIT> << * MI << <STR_LIT> << MI -> getParent ( ) -> getNumber ( ) << '\n' ; llvm_unreachable ( nullptr ) ; } DefM [ RR ] .


<s> for ( auto I = DefM . begin ( ) , E = DefM . end ( ) ; I != E ; ++ I ) I -> second . clear_block ( B ) ; for ( auto I = DefM . begin ( ) , E = DefM . end ( ) , NextI = I ; I != E ; I = NextI ) { NextI = std :: next ( I ) ; if ( I -> second . empty ( ) ) DefM . erase (


<s> void removeFromOwner ( NodeAddr < RefNode * > RA ) { NodeAddr < InstrNode * > IA = RA . Addr -> getOwner ( * this ) ; IA . Addr -> removeMember ( RA ,


<s> Memory . clear ( ) ; Func = NodeAddr < FuncNode *


<s> setAttrs ( NodeAttrs :: set_flags ( getAttrs (


<s> void setPredecessor ( NodeId B ) { assert ( getFlags ( ) & NodeAttrs


<s> void setReachingDef ( NodeId RD ) { Ref . RD = RD


<s> void RefNode :: setRegRef ( MachineOperand * Op ) { assert ( NodeAttrs :: type ( Attrs ) == NodeAttrs :: Ref ) ; assert ( ! ( NodeAttrs :: flags ( Attrs ) & NodeAttrs :: PhiRef ) ) ; Ref . Op =


<s> static uint16_t set_kind ( uint16_t A , uint16_t K ) { return ( A & ~ KindMask ) |


<s> void * T = MemPool . Allocate ( NodesPerBlock * NodeMemSize , NodeMemSize ) ; char * P = static_cast < char *


<s> Blocks . push_back ( P ) ; assert ( ( Blocks . size ( ) < ( ( size_t ) <NUM_LIT> << (

<NUM_LIT> * sizeof ( NodeId ) - BitsPerIndex ) ) ) && <STR_LIT> ) ;


<s> static uint16_t type ( uint16_t T ) { return T


<s> if ( RemoveFromOwner ) removeFromOwner (


<s> if ( RemoveFromOwner ) removeFromOwner


<s> auto TA = addr < DefNode * > ( RDA . Addr -> getReachedDef ( ) ) ; if ( TA . Id == DA . Id ) { RDA . Addr -> setReachedDef ( Sib ) ; } else { while ( TA . Id != <NUM_LIT> ) { NodeId S = TA . Addr -> getSibling ( ) ; if ( S == DA . Id ) { TA . Addr -> setSibling ( Sib ) ; break ; } TA = addr < DefNode * > ( S ) ; } } if ( ! ReachedDefs . empty ( ) ) { auto Last = NodeAddr < DefNode * > ( ReachedDefs . back ( ) ) ; Last . Addr -> setSibling ( RDA . Addr -> getReachedDef ( ) ) ; RDA . Addr -> setReachedDef ( ReachedDefs . front ( ) . Id ) ; } if ( ! ReachedUses . empty


<s> Iterator & up ( ) { Pos = DS . nextUp ( Pos )


<s> NextD = DA . Addr -> getSibling ( ) ; RegisterRef DR = DA . Addr -> getRegRef ( DFG ) ; if ( DefRRs . hasCoverOf ( DR ) || ! DFG . alias ( RefRR , DR ) ) continue ; NodeSet T ; if ( DFG . IsPreservingDef ( DA ) ) { T = getAllReachedUses ( RefRR , DA , DefRRs ) ; } else { RegisterAggr NewDefRRs = DefRRs ; NewDefRRs . insert ( DR ) ; T = getAllReachedUses ( RefRR , DA , NewDefRRs ) ; } Uses . insert ( T . begin ( ) , T . end ( ) ) ; } return Uses


<s> NodeId Start = RefA . Id ; auto SNA = DFG . addr < RefNode * > ( Start ) ; if ( NodeId RD = SNA . Addr -> getReachingDef ( ) ) DefQ . insert ( RD ) ; for ( unsigned i = <NUM_LIT> ; i < DefQ . size ( ) ; ++ i ) { auto TA = DFG . addr < DefNode * > ( DefQ [ i ] ) ; if ( TA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) continue ; RegisterRef RR = TA . Addr -> getRegRef ( DFG ) ; if ( ! DFG . IsPreservingDef ( TA ) ) if ( RegisterAggr :: isCoverOf ( RR , RefRR , TRI ) ) continue ; for ( auto S : DFG . getRelatedRefs ( TA . Addr -> getOwner ( DFG ) , TA ) ) if ( NodeId RD = NodeAddr < RefNode * > ( S ) . Addr -> getReachingDef ( ) ) DefQ . insert ( RD ) ; } SetVector < NodeId > Defs ; for ( NodeId N : DefQ ) { auto TA = DFG . addr < DefNode * > ( N ) ; bool IsPhi = TA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ; if ( ! IsPhi && ! DFG . alias ( RefRR , TA . Addr -> getRegRef ( DFG ) ) ) continue ; Defs . insert ( TA . Id ) ; Owners . insert ( TA . Addr -> getOwner ( DFG ) . Id ) ; } auto Block = [ this ] ( NodeAddr < InstrNode * > IA ) -> MachineBasicBlock * { if ( IA . Addr -> getKind ( ) == NodeAttrs :: Stmt ) return NodeAddr < StmtNode * > ( IA ) . Addr -> getCode ( ) -> getParent ( ) ; assert ( IA . Addr -> getKind ( ) == NodeAttrs :: Phi


<s> } NodeList RDs = getAllReachingDefs ( RefRR , RefA , true , DefRRs ) ; if ( RDs . empty ( ) ) return Defs ; NodeSet TmpDefs = Defs ; for ( NodeAddr < NodeBase * > R : RDs ) TmpDefs . insert ( R . Id ) ; NodeSet Result = Defs ; for ( NodeAddr < DefNode * > DA : RDs ) { Result . insert ( DA . Id ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) continue ; NodeAddr < PhiNode * > PA = DA . Addr -> getOwner ( DFG ) ; if ( Visited . count ( PA . Id ) ) continue ; Visited . insert ( PA . Id ) ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { const auto & T = getAllReachingDefsRec ( RefRR ,


<s> if ( ! S . isValid ( ) ) { LV . set ( I . PhysReg ) ; continue ; } do { LaneBitmask M = TRI . getSubRegIndexLaneMask ( S . getSubRegIndex ( ) ) ; if ( ( M & I . LaneMask ) . any ( ) ) LV . set ( S . getSubReg ( ) ) ; ++ S ; } while ( S . isValid ( ) ) ; } } ; BitVector LiveIn ( TRI . getNumRegs ( ) ) , Live ( TRI . getNumRegs ( ) ) ; CopyLiveIns ( B , LiveIn ) ; for ( auto SI : B -> successors ( ) ) CopyLiveIns ( SI , Live ) ; for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = & * I ; if ( MI -> isDebugValue ( ) ) continue ; MI -> clearKillInfo (


<s> for ( auto I = B . livein_begin ( ) , E = B . livein_end ( ) ; I != E ; ++ I ) T . push_back ( I -> PhysReg ) ; for ( auto I : T ) B . removeLiveIn ( I ) ; auto & LiveIns = LiveMap [ & B


<s> dbgs ( ) << <STR_LIT> ; dbgs ( ) << <STR_LIT> << Print < RefMap > ( LiveIn , DFG ) << '\n' ; dbgs ( ) << <STR_LIT> << Print < RegisterAggr > ( LiveMap [ B ] , DFG ) << '\n' ; } RefMap LiveInCopy = LiveIn ; LiveIn . clear ( ) ; for ( const std :: pair < RegisterId , NodeRefSet > & LE : LiveInCopy ) { RegisterRef LRef ( LE . first ) ; NodeRefSet & NewDefs = LiveIn [ LRef . Reg ] ; const NodeRefSet & OldDefs = LE . second ; for ( NodeRef OR : OldDefs ) { auto DA = DFG . addr < DefNode * > ( OR . first ) ; NodeAddr < InstrNode * > IA = DA . Addr -> getOwner ( DFG ) ; NodeAddr < BlockNode * > BA = IA . Addr -> getOwner ( DFG ) ; if ( B != BA . Addr -> getCode ( ) ) { NewDefs . insert ( OR ) ; continue ; } RegisterAggr RRs ( TRI ) ; LRef . Mask = OR . second ; if ( ! DFG . IsPreservingDef ( DA ) ) { assert ( ! ( IA . Addr -> getFlags ( ) & NodeAttrs :: Phi ) ) ; if ( RRs . insert ( DA . Addr -> getRegRef ( DFG ) ) . hasCoverOf ( LRef ) ) continue ; } for ( NodeAddr < DefNode * > TA : getAllReachingDefs ( DA ) ) { NodeAddr < InstrNode * > ITA = TA . Addr -> getOwner ( DFG ) ; NodeAddr < BlockNode * > BTA = ITA . Addr -> getOwner ( DFG ) ; if ( BTA . Addr -> getCode ( ) != B ) { RegisterAggr L ( TRI ) ; L . insert ( LRef ) . clear ( RRs ) ; assert ( ! L . empty ( ) ) ; NewDefs . insert ( { TA . Id , L . begin ( ) -> second } ) ; break ; } if ( ! ( TA . Addr -> getFlags ( ) & NodeAttrs :: Preserving ) ) RRs . insert ( TA . Addr -> getRegRef ( DFG ) ) ; if ( RRs . hasCoverOf ( LRef ) ) break ; } } } emptify ( LiveIn ) ; if ( Trace ) { dbgs ( ) << <STR_LIT> ; dbgs ( ) << <STR_LIT> << Print < RefMap > ( LiveIn , DFG ) << '\n' ; dbgs ( ) << <STR_LIT> << Print < RegisterAggr > ( LiveMap [ B ]


<s> SetVector < MachineBasicBlock * > IDFB ( F1 -> second . begin ( ) , F1 -> second . end ( ) ) ; for ( unsigned i = <NUM_LIT> ; i < IDFB . size ( ) ; ++ i ) { auto F2 = MDF . find ( IDFB [ i ] ) ; if ( F2 != MDF . end ( ) ) IDFB . insert ( F2 -> second . begin ( ) , F2 -> second . end ( ) ) ; } IDFB . insert ( & B ) ; IDF [ & B ] . insert ( IDFB . begin ( ) , IDFB . end ( ) ) ; } for ( auto I : IDF ) for ( auto S : I . second ) IIDF [ S ] . insert ( I . first ) ; computePhiInfo ( ) ; NodeAddr < FuncNode * > FA = DFG . getFunc ( ) ; auto Blocks = FA . Addr -> members ( DFG ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) { MachineBasicBlock * MB = BA . Addr -> getCode ( ) ; auto & LON = PhiLON [ MB ] ; for ( auto P : BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ) for ( auto S : RealUseMap [ P . Id ] ) LON [ S . first ] . insert ( S . second . begin ( ) , S . second . end ( ) ) ; } if ( Trace ) { dbgs ( ) << <STR_LIT> ; for ( auto I : PhiLON ) dbgs ( ) << <STR_LIT> << I . first -> getNumber ( ) << <STR_LIT> << Print < RefMap > ( I . second , DFG ) << '\n' ; } for ( NodeAddr < BlockNode * > BA : Blocks ) { auto Phis = BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ; for ( NodeAddr < PhiNode * > PA : Phis ) { auto & RUs = RealUseMap [ PA . Id ] ; if ( RUs . empty ( ) ) continue ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { NodeAddr < PhiUseNode * > UA = U ; if ( UA . Addr -> getReachingDef ( ) == <NUM_LIT> ) continue ; auto PrA = DFG . addr < BlockNode * > ( UA . Addr -> getPredecessor ( ) ) ; auto & LOX = PhiLOX [ PrA . Addr -> getCode ( ) ] ; for ( auto R : RUs ) { RegisterRef RR = R . first ; if ( ! isRestricted ( PA , UA ,

RR ) ) RR = getRestrictedRegRef ( UA ) ;


<s>
}
if
(
!
RealUses
.
empty
(
)
)
PhiUQ
.
push_back
(
PhiA
.
Id
)
;
for
(
auto
I
:
PhiRefs
)
{
if
(
!
DFG
.
IsRef
<
NodeAttrs
::
Use
>
(
I
)
)
continue
;
NodeAddr
<
UseNode
*
>
UA
=
I
;
auto
&
UpMap
=
PhiUp
[
UA
.
Id
]
;
RegisterSet
DefRRs
;
for
(
NodeAddr
<
DefNode
*
>
DA
:
getAllReachingDefs
(
UA
)
)
{
if
(
DA
.
Addr
->
getFlags
(
)
&
NodeAttrs
::
PhiRef
)
UpMap
[
DA
.
Addr
->
getOwner
(
DFG
)
.
Id
]
=
DefRRs
;
else
DefRRs
.
insert
(
DA
.
Addr
->
getRegRef
(
)
)
;
}
}
}
if
(
Trace
)
{
dbgs
(
)
<<
<STR_LIT>
;
for
(
auto
I
:
PhiUp
)
{
dbgs
(
)
<<
<STR_LIT>
<<
Print
<
NodeId
>
(
I
.
first
,
DFG
)
<<
<STR_LIT>
;
for
(
auto
R
:
I
.
second
)
dbgs
(
)
<<
' '
<<
Print
<
NodeId
>
(
R
.
first
,
DFG
)
<<
Print
<
RegisterSet
>
(
R
.
second
,
DFG
)
;
dbgs
(
)
<<
<STR_LIT>
;
}
}
for
(
unsigned
i
=
<NUM_LIT>
;
i
<
PhiUQ
.
size
(
)
;
++
i
)
{
auto
PA
=
DFG
.
addr
<
PhiNode
*
>
(
PhiUQ
[
i
]
)
;
auto
&
RealUses
=
RealUseMap
[
PA
.
Id
]
;
for
(
auto
U
:
PA
.
Addr
->
members_if
(
DFG
.
IsRef
<
NodeAttrs
::
Use
>
,
DFG
)
)
{
NodeAddr
<
UseNode
*
>
UA
=
U
;
auto
&
UpPhis
=
PhiUp
[
UA
.
Id
]
;
for
(
auto
UP
:
UpPhis
)
{
bool
Changed
=
false
;
auto
&
MidDefs
=
UP
.
second
;
RegisterSet
UpReached
;
for
(
auto
T
:
RealUses
)
{
if
(
!
isRestricted
(
PA
,
UA
,
T
.
first
)
)
continue
;
if
(
!
RAI
.
covers
(
MidDefs
,
T
.
first
)
)
UpReached
.
insert
(
T
.
first
)
;
}
if
(
UpReached
.
empty
(
)
)
continue
;
auto
&
PRUs
=
RealUseMap
[
UP
.
first
]
;
for
(
auto
R
:
UpReached
)
{
unsigned
Z
=
PRUs
[
R
]
.


<s> return getAllReachingDefs ( RefA . Addr -> getRegRef ( ) , RefA , false


<s> if ( RA . Addr -> getFlags ( ) & NodeAttrs :: Shadow ) { NodeId RD = RA . Addr -> getReachingDef ( ) ; assert ( RD ) ; RA = DFG . addr < DefNode * > (


<s> RegisterRef Liveness :: getRestrictedRegRef ( NodeAddr < RefNode * > RA ) const { assert ( DFG . IsRef < NodeAttrs :: Use >


<s> for ( NodeAddr < RefNode * > TA = DFG . getNextShadow ( IA , RA ) ; TA . Id != <NUM_LIT> && TA . Id != Start ; TA = DFG . getNextShadow ( IA , TA ) ) { NodeId RD = TA . Addr -> getReachingDef ( ) ; if ( RD ==


<s> Result . insert ( DA . Id ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) continue ; NodeAddr < PhiNode * > PA = DA . Addr -> getOwner ( DFG ) ; if ( Visited . count ( PA . Id ) ) continue ; Visited . insert ( PA . Id ) ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { const auto & T = getAllReachingDefsRec ( RefRR , U , Visited , TmpDefs ) ; Result . insert ( T . begin ( ) , T . end ( ) ) ; } } return Result


<s> auto F1 = MDF . find ( & B ) ; if ( F1 == MDF . end ( ) ) continue ; SetVector < MachineBasicBlock * > IDFB ( F1 -> second . begin ( ) , F1 -> second . end ( ) ) ; for ( unsigned i = <NUM_LIT> ; i < IDFB . size ( ) ; ++ i ) { auto F2 = MDF . find ( IDFB [ i ] ) ; if ( F2 != MDF . end ( ) ) IDFB . insert ( F2 -> second . begin ( ) , F2 -> second . end ( ) ) ; } IDFB . insert ( & B ) ; IDF [ & B ] . insert ( IDFB . begin ( ) , IDFB . end ( ) ) ; } for ( auto I : IDF ) for ( auto S : I . second ) IIDF [ S ] . insert ( I . first ) ; computePhiInfo ( ) ; NodeAddr < FuncNode * > FA = DFG . getFunc ( ) ; NodeList Blocks = FA . Addr -> members ( DFG ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) { MachineBasicBlock * MB = BA . Addr -> getCode ( ) ; RefMap & LON = PhiLON [ MB ] ; for ( auto P : BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ) for ( const RefMap :: value_type & S : RealUseMap [ P . Id ] ) LON [ S . first ] . insert ( S . second . begin ( ) , S . second . end ( ) ) ; } if ( Trace ) { dbgs ( ) << <STR_LIT> ; for ( auto & I : PhiLON ) dbgs ( ) << <STR_LIT> << I . first -> getNumber ( ) << <STR_LIT> << Print < RefMap > ( I . second , DFG ) << '\n' ; } for ( NodeAddr < BlockNode * > BA : Blocks ) { NodeList Phis = BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ; for ( NodeAddr < PhiNode * > PA : Phis ) { RefMap & RUs = RealUseMap [ PA . Id ] ; if ( RUs . empty (


<s> if ( Trace ) { dbgs ( ) << <STR_LIT> ; dbgs ( ) << <STR_LIT> << Print < RefMap > ( LiveIn , DFG ) << '\n' ; dbgs ( ) << <STR_LIT> << Print < RegisterAggr > ( LiveMap [ B ] , DFG ) << '\n' ; } for ( auto I : DFG . getFunc ( ) . Addr -> findBlock ( B , DFG ) . Addr -> members ( DFG ) ) { NodeAddr < InstrNode * > IA = I ; if ( IA . Addr -> getKind ( ) != NodeAttrs :: Stmt ) continue ; for ( NodeAddr < UseNode * > UA : IA . Addr -> members_if ( DFG . IsUse , DFG ) ) { if ( UA . Addr -> getFlags ( ) & NodeAttrs :: Undef ) continue ; RegisterRef RR = PRI . normalize ( UA . Addr -> getRegRef ( DFG ) ) ; for ( NodeAddr < DefNode * > D : getAllReachingDefs ( UA ) ) if ( getBlockWithRef ( D . Id ) != B ) LiveIn [ RR . Reg ] . insert ( { D . Id , RR . Mask } ) ; } } if ( Trace ) { dbgs ( ) << <STR_LIT> ; dbgs ( ) << <STR_LIT> << Print < RefMap > ( LiveIn , DFG ) << '\n' ; dbgs ( ) << <STR_LIT> << Print < RegisterAggr > ( LiveMap [ B ] , DFG ) << '\n' ; } RegisterAggr & Local = LiveMap [ B ] ; RefMap & LON = PhiLON [ B ] ; for ( auto & R : LON ) { LaneBitmask M ; for ( auto P : R . second ) M |= P . second ; Local . insert ( RegisterRef ( R . first , M ) ) ; } if ( Trace ) { dbgs ( ) << <STR_LIT> ; dbgs ( ) << <STR_LIT> << Print < RefMap > ( LiveIn , DFG )


<s> U = UA . Addr -> getSibling ( ) ; } for ( NodeId D = DefA . Addr -> getReachedDef ( ) , NextD ; D != <NUM_LIT> ; D = NextD ) { auto DA = DFG . addr < DefNode * > ( D ) ; NextD = DA . Addr -> getSibling ( ) ; auto DR = DA . Addr -> getRegRef ( ) ; if ( RAI . covers ( DefRRs , DR ) || ! RAI . alias (


<s> NodeSet Result = Defs ; for ( NodeAddr < DefNode * > DA : RDs ) { Result . insert ( DA . Id ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) continue ; NodeAddr < PhiNode * > PA = DA . Addr -> getOwner ( DFG ) ; if ( Visited . count ( PA . Id ) ) continue ; Visited . insert ( PA . Id ) ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { const auto & T = getAllReachingDefsRec ( RefRR , U , Visited , TmpDefs ) ; Result . insert ( T . begin ( ) , T . end ( ) ) ; } } return Result


<s> RegisterSet DefRRs ; for ( const auto D : Defs ) { const auto DA = DFG . addr < const DefNode * > ( D ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) DefRRs . insert ( DA . Addr -> getRegRef ( ) ) ; } auto RDs = getAllReachingDefs ( RefRR , RefA , true , DefRRs ) ; if ( RDs . empty ( ) ) return Defs ; NodeSet TmpDefs = Defs ; for ( auto R : RDs ) TmpDefs . insert ( R


<s> if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; for ( MCSubRegIterator SR ( R , & TRI , true ) ; SR . isValid ( ) ; ++ SR ) Live . reset ( * SR ) ; } for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isUse ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; bool IsLive


<s> for ( NodeId N : DefQ ) { auto TA = DFG . addr < DefNode * > ( N ) ; bool IsPhi = TA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ; if ( ! IsPhi && ! PRI . alias ( RefRR , TA . Addr -> getRegRef ( DFG ) ) ) continue ; Defs . insert ( TA . Id ) ; Owners . insert ( TA . Addr -> getOwner ( DFG ) . Id ) ; } auto Block = [ this ] ( NodeAddr < InstrNode * > IA ) -> MachineBasicBlock * { if ( IA . Addr -> getKind ( ) == NodeAttrs :: Stmt ) return NodeAddr < StmtNode * > ( IA ) . Addr -> getCode ( ) -> getParent ( ) ; assert ( IA . Addr -> getKind ( ) == NodeAttrs :: Phi ) ; NodeAddr < PhiNode * > PA = IA ; NodeAddr < BlockNode * > BA = PA . Addr -> getOwner ( DFG ) ; return BA . Addr -> getCode ( ) ; } ; auto Less = [ & Block , this ] ( NodeId A , NodeId B ) -> bool { if ( A == B ) return false ; auto OA = DFG . addr < InstrNode * > ( A ) , OB = DFG . addr < InstrNode * > ( B ) ; MachineBasicBlock * BA = Block ( OA ) , * BB = Block ( OB ) ; if ( BA != BB ) return MDT . dominates ( BB , BA ) ; bool StmtA = OA . Addr -> getKind ( ) == NodeAttrs :: Stmt ; bool StmtB = OB . Addr -> getKind ( ) == NodeAttrs :: Stmt ; if ( StmtA ) { if ( ! StmtB ) return true ; MachineInstr * CA = NodeAddr < StmtNode * > ( OA ) . Addr -> getCode ( ) ; MachineInstr * CB = NodeAddr < StmtNode * > ( OB ) . Addr -> getCode ( ) ; if ( CA == CB ) return A < B ; return MDT . dominates ( CB , CA ) ; } else { if ( StmtB ) return false ; return A <


<s> } IDFB . insert ( & B ) ; IDF [ & B ] . insert ( IDFB . begin ( ) , IDFB . end ( ) ) ; } for ( auto I : IDF ) for ( auto S : I . second ) IIDF [ S ] . insert ( I . first ) ; computePhiInfo ( ) ; NodeAddr < FuncNode * > FA = DFG . getFunc ( ) ; NodeList Blocks = FA . Addr -> members ( DFG ) ; for ( NodeAddr < BlockNode * > BA : Blocks ) { MachineBasicBlock * MB = BA . Addr -> getCode ( ) ; RefMap & LON = PhiLON [ MB ] ; for ( auto P : BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ) for ( const RefMap :: value_type & S : RealUseMap [ P . Id ] ) LON [ S . first ] . insert ( S . second . begin ( ) , S . second . end ( ) ) ; } if ( Trace ) { dbgs ( ) << <STR_LIT> ; for ( auto & I : PhiLON ) dbgs ( ) << <STR_LIT> << I . first -> getNumber ( ) << <STR_LIT> << Print < RefMap > ( I . second , DFG ) << '\n' ; } for ( NodeAddr < BlockNode * > BA : Blocks ) { NodeList Phis = BA . Addr -> members_if ( DFG . IsCode < NodeAttrs :: Phi > , DFG ) ; for ( NodeAddr < PhiNode * > PA : Phis ) { RefMap & RUs = RealUseMap [ PA . Id ] ; if ( RUs . empty ( ) ) continue ; NodeSet SeenUses ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { if ( ! SeenUses . insert ( U . Id )


<s> continue ; } do { LaneBitmask M = TRI . getSubRegIndexLaneMask ( S . getSubRegIndex ( ) ) ; if ( ( M & I . LaneMask ) . any ( ) ) LV . set ( S . getSubReg ( ) ) ; ++ S ; } while ( S . isValid ( ) ) ; } } ; BitVector LiveIn ( TRI . getNumRegs ( ) ) , Live ( TRI . getNumRegs ( ) ) ; CopyLiveIns ( B , LiveIn ) ; for ( auto SI : B -> successors ( ) ) CopyLiveIns ( SI , Live ) ; for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = & * I ; if ( MI -> isDebugValue ( ) ) continue ; MI -> clearKillInfo ( ) ; for ( auto & Op : MI -> operands (


<s> do { LaneBitmask M = TRI . getSubRegIndexLaneMask ( S . getSubRegIndex ( ) ) ; if ( ( M & I . LaneMask ) . any ( ) ) LV . set ( S . getSubReg ( ) ) ; ++ S ; } while ( S . isValid ( ) ) ; } } ; BitVector LiveIn ( TRI . getNumRegs ( ) ) , Live ( TRI . getNumRegs ( ) ) ; CopyLiveIns ( B , LiveIn ) ; for ( auto SI : B -> successors ( ) ) CopyLiveIns ( SI , Live ) ; for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = & * I ; if ( MI -> isDebugValue ( ) ) continue ; MI -> clearKillInfo ( ) ; for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; for ( MCSubRegIterator SR ( R , & TRI , true ) ; SR . isValid ( ) ; ++ SR ) Live . reset ( * SR ) ; } for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isUse ( ) || Op . isUndef ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; bool IsLive = false ; for ( MCRegAliasIterator AR ( R , & TRI , true ) ; AR . isValid ( ) ; ++


<s> for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = & * I ; if ( MI -> isDebugValue ( ) ) continue ; MI -> clearKillInfo ( ) ; for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; for ( MCSubRegIterator SR ( R , & TRI , true ) ; SR . isValid ( ) ; ++ SR ) Live . reset ( * SR ) ; } for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isUse ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; bool IsLive = false ; for ( MCRegAliasIterator AR ( R , & TRI , true ) ; AR . isValid ( ) ; ++ AR ) { if ( ! Live [ * AR ] ) continue ; IsLive = true


<s> auto CopyLiveIns = [ this ] ( MachineBasicBlock * B , BitVector & LV ) -> void { for ( auto I : B -> liveins ( ) ) { MCSubRegIndexIterator S ( I . PhysReg , & TRI ) ; if ( ! S . isValid ( ) ) { LV . set ( I . PhysReg ) ; continue ; } do { LaneBitmask M = TRI . getSubRegIndexLaneMask ( S . getSubRegIndex ( ) ) ; if ( ( M & I . LaneMask ) . any ( ) ) LV . set ( S . getSubReg ( ) ) ; ++ S ; } while ( S . isValid ( ) ) ; } } ; BitVector LiveIn ( TRI . getNumRegs ( ) ) , Live ( TRI . getNumRegs ( ) ) ; CopyLiveIns ( B , LiveIn ) ; for ( auto SI : B -> successors ( ) ) CopyLiveIns ( SI , Live ) ; for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = &


<s> return getAllReachingDefs ( RefRR , RefA , false ,


<s> NodeSet Liveness :: getAllReachingDefsRec ( RegisterRef RefRR , NodeAddr < RefNode * > RefA , NodeSet & Visited , const NodeSet & Defs ) { RegisterAggr DefRRs ( PRI ) ; for ( NodeId D : Defs ) { const auto DA = DFG . addr < const DefNode * > ( D ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) DefRRs . insert ( DA . Addr -> getRegRef ( DFG ) ) ; } NodeList RDs = getAllReachingDefs ( RefRR , RefA , true , DefRRs ) ; if ( RDs . empty ( ) ) return


<s>
RefMap
&
LOX
=
PhiLOX
[
PrA
.
Addr
->
getCode
(
)
]
;
for
(
const
std
::
pair
<
RegisterId
,
NodeRefSet
>
&
RS
:
RUs
)
{
for
(
std
::
pair
<
NodeId
,
LaneBitmask
>
P
:
RS
.
second
)
{
RegisterRef
S
(
RS
.
first
,
P
.
second
)
;
NodeList
Ds
=
getAllReachingDefs
(
S
,
PUA
,
true
,
false
,
NoRegs
)
;
for
(
NodeAddr
<
DefNode
*
>
D
:
Ds
)
{
RegisterAggr
TA
(
PRI
)
;
TA
.
insert
(
D
.
Addr
->
getRegRef
(
DFG
)
)
.
intersect
(
S
)
;
LaneBitmask
TM
=
TA
.
makeRegRef
(
)
.
Mask
;
LOX
[
S
.
Reg
]
.
insert
(
{
D
.
Id
,
TM
}
)
;
}
}
}
for
(
NodeAddr
<
PhiUseNode
*
>
T
:
DFG
.
getRelatedRefs
(
PA
,
PUA
)
)
SeenUses
.
insert
(
T
.
Id
)
;
}
}
}
if
(
Trace
)
{
dbgs
(
)
<<
<STR_LIT>
;
for
(
auto
&
I
:
PhiLOX
)
dbgs
(
)
<<
<STR_LIT>
<<
I
.
first
->
getNumber
(
)
<<
<STR_LIT>
<<
Print
<
RefMap
>
(
I
.
second
,
DFG
)
<<
'\n'
;
}
RefMap
LiveIn
;
traverse
(
&
MF
.
front
(
)
,
LiveIn
)
;
LiveMap
[
&
MF
.
front
(
)
]
.
insert
(
DFG
.
getLiveIns
(
)
)
;
if
(
Trace
)
{
for
(
MachineBasicBlock
&
B
:
MF
)
{
std
::
vector
<
RegisterRef
>
LV
;
for
(
auto
I
=
B
.
livein_begin
(
)
,
E
=
B
.
livein_end
(
)
;
I
!=
E
;
++
I
)
LV
.
push_back
(
RegisterRef
(
I
->
PhysReg
,
I
->
LaneMask
)
)
;
llvm
::
sort
(
LV
.
begin
(
)
,
LV
.
end
(
)
)
;
dbgs
(
)
<<
printMBBReference
(
B
)
<<
<STR_LIT>
;
for
(
auto
I
:
LV
)
dbgs
(
)
<<
' '
<<
Print
<
RegisterRef
>
(
I
,
DFG
)
;
dbgs
(
)
<<
<STR_LIT>
;
LV
.
clear
(
)
;
const
RegisterAggr
&
LG
=
LiveMap
[
&
B
]
;
for
(
auto
I
=
LG
.
rr_begin
(
)
,
E
=
LG
.
rr_end
(
)
;
I
!=
E
;
++
I
)
LV
.
push_back
(
*
I
)
;
llvm
::
sort
(
LV
.
begin
(
)
,
LV
.
end
(
)
)
;
dbgs
(
)
<<
<STR_LIT>
;
for
(
auto
I
:
LV
)
dbgs
(
)
<<
' '
<<


<s> void Liveness :: emptify ( RefMap & M ) { for ( auto I = M . begin ( ) , E = M . end ( ) ; I != E ; ) I = I

-> second . empty ( ) ? M . erase ( I ) : std :: next ( I ) ;


<s> return getAllReachingDefs ( RefRR , RefA , false


<s> std :: pair < NodeSet , bool > Liveness :: getAllReachingDefsRec ( RegisterRef RefRR , NodeAddr

< RefNode * > RefA , NodeSet & Visited , const NodeSet & Defs ) {


<s> std :: pair < NodeSet , bool > Liveness :: getAllReachingDefsRec ( RegisterRef RefRR , NodeAddr < RefNode * > RefA

, NodeSet & Visited , const NodeSet & Defs ) {


<s> RegisterAggr DefRRs ( PRI ) ; for ( NodeId D : Defs ) { const auto DA = DFG . addr < const DefNode * > ( D ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) DefRRs . insert ( DA . Addr -> getRegRef ( DFG ) ) ; } NodeList RDs = getAllReachingDefs ( RefRR , RefA , false , true , DefRRs ) ; if ( RDs . empty ( ) ) return { Defs , true } ; NodeSet TmpDefs = Defs ; for ( NodeAddr < NodeBase * > R : RDs ) TmpDefs . insert ( R . Id ) ; NodeSet Result = Defs ; for ( NodeAddr < DefNode * > DA : RDs ) { Result . insert ( DA . Id ) ; if ( ! ( DA . Addr -> getFlags ( ) & NodeAttrs :: PhiRef ) ) continue ; NodeAddr < PhiNode * > PA = DA . Addr -> getOwner ( DFG ) ; if ( Visited . count ( PA . Id ) ) continue ; Visited . insert ( PA . Id ) ; for ( auto U : PA . Addr -> members_if ( DFG . IsRef < NodeAttrs :: Use > , DFG ) ) { const auto & T = getAllReachingDefsRecImpl ( RefRR , U , Visited , TmpDefs , Nest + <NUM_LIT> , MaxNest ) ; if ( ! T . second ) return { T . first , false } ; Result . insert ( T . first . begin ( ) , T . first . end ( ) ) ; } } return { Result ,


<s> do { LaneBitmask M = TRI . getSubRegIndexLaneMask ( S . getSubRegIndex ( ) ) ; if ( ( M & I . LaneMask ) . any ( ) ) LV . set ( S . getSubReg ( ) ) ; ++ S ; } while ( S . isValid ( ) ) ; } } ; BitVector LiveIn ( TRI . getNumRegs ( ) ) , Live ( TRI . getNumRegs ( ) ) ; CopyLiveIns ( B , LiveIn ) ; for ( auto SI : B -> successors ( ) ) CopyLiveIns ( SI , Live ) ; for ( auto I = B -> rbegin ( ) , E = B -> rend ( ) ; I != E ; ++ I ) { MachineInstr * MI = & * I ; if ( MI -> isDebugInstr ( ) ) continue ; MI -> clearKillInfo ( ) ; for ( auto & Op : MI -> operands ( ) ) { if ( ! Op . isReg ( ) || ! Op . isDef ( ) || Op . isImplicit ( ) ) continue ; unsigned R = Op . getReg ( ) ; if ( ! TargetRegisterInfo :: isPhysicalRegister ( R ) ) continue ; for ( MCSubRegIterator SR ( R , & TRI , true ) ; SR . isValid ( ) ; ++ SR ) Live . reset ( * SR ) ; } for ( auto & Op : MI


<s> for ( auto I = B . livein_begin ( ) , E = B . livein_end ( ) ; I != E ; ++ I ) T . push_back ( I -> PhysReg ) ; for ( auto I : T ) B . removeLiveIn (


<s> LaneBitmask LA = PA . second & RA . Mask ; LaneBitmask LB = PB . second & RB . Mask ; if ( LA . any ( ) && LB . any ( ) ) { unsigned Root = * MCRegUnitRootIterator ( PA . first , & TRI ) ; uint32_t SubA = TRI . getSubRegIndex ( RA . Reg , Root ) ; uint32_t SubB = TRI . getSubRegIndex ( RB . Reg , Root ) ; const TargetRegisterClass * RC = RegInfos [ Root ] . RegClass ; LaneBitmask RCMask = RC != nullptr ? RC -> LaneMask : LaneBitmask ( <NUM_LIT> ) ; LaneBitmask MaskA = TRI . reverseComposeSubRegIndexLaneMask ( SubA , LA ) ; LaneBitmask MaskB = TRI . reverseComposeSubRegIndexLaneMask ( SubB , LB ) ; if ( ( MaskA & MaskB & RCMask ) . any ( ) ) return true ; } ++ UMA ; ++ UMB ; continue ; } if ( PA . first < PB . first )


<s> unsigned Root = * MCRegUnitRootIterator ( PA . first , & TRI ) ; uint32_t SubA = TRI . getSubRegIndex ( RA . Reg , Root ) ; uint32_t SubB = TRI . getSubRegIndex ( RB . Reg , Root ) ; const TargetRegisterClass * RC = RegInfos [ Root ] . RegClass ; LaneBitmask RCMask = RC != nullptr ? RC -> LaneMask : LaneBitmask ( <NUM_LIT> ) ; LaneBitmask MaskA = TRI . reverseComposeSubRegIndexLaneMask ( SubA , LA ) ; LaneBitmask MaskB = TRI . reverseComposeSubRegIndexLaneMask ( SubB , LB ) ; if ( ( MaskA & MaskB & RCMask ) . any ( ) ) return true ; } ++ UMA ; ++ UMB ; continue ; } if ( PA . first < PB . first ) ++ UMA ; else if ( PB . first < PA . first ) ++ UMB ; } return


<s> for ( unsigned i = <NUM_LIT> , e = PRI . getTRI ( ) . getNumRegs ( ) ; i != e ; ++ i ) { if ( MB [ i / <NUM_LIT> ] & ( <NUM_LIT> << ( i % <NUM_LIT> ) ) ) continue ; clear ( RegisterRef ( i , LaneBitmask :: getAll ( ) ) ) ; } return * this ; } RegisterRef NR = PRI . normalize ( RR ) ; auto F = Masks . find


<s> clear ( RegisterRef ( i , LaneBitmask :: getAll ( ) ) ) ; } return * this ; } RegisterRef NR = PRI . normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F == Masks . end ( ) ) return * this ; LaneBitmask NewM = F -> second & ~ NR . Mask ; if ( NewM . none ( ) ) Masks . erase ( F ) ; else F -> second


<s> RegisterRef RegisterAggr :: clearIn ( RegisterRef RR ) const { RegisterAggr T ( PRI ) ; T . insert ( RR ) .


<s> RegisterRef RegisterAggr :: clearIn ( RegisterRef RR ) const { RegisterAggr T ( PRI ) ; T . insert ( RR ) . clear


<s> const uint32_t * MB = PRI . getRegMaskBits ( RR . Reg ) ; for ( unsigned i = <NUM_LIT> , e = PRI . getTRI ( ) . getNumRegs ( ) ; i != e ; ++ i ) { if ( MB [ i / <NUM_LIT> ] & ( <NUM_LIT> << ( i % <NUM_LIT> ) ) ) continue ; if ( ! hasCoverOf ( RegisterRef ( i , LaneBitmask :: getAll ( ) ) ) ) return false ; } return true ; } RegisterRef NR = PRI . normalize ( RR ) ; if ( NR . Mask . none (


<s> } RegisterRef NR = PRI . normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F == Masks . end ( ) ) Masks . insert ( { NR . Reg , NR . Mask } ) ; else F -> second |= NR . Mask ; const TargetRegisterInfo & TRI = PRI . getTRI ( ) ; for ( MCRegUnitIterator U ( RR . Reg , & TRI ) ; U . isValid ( ) ; ++ U ) { MCRegUnitRootIterator R ( * U , & TRI ) ; ++


<s> if ( PhysicalRegisterInfo :: isRegMaskId ( RR . Reg ) ) return RR ; const TargetRegisterClass * RC = RegInfos [ RR . Reg ] . RegClass ; LaneBitmask RCMask = RC != nullptr ? RC -> LaneMask : LaneBitmask ( <NUM_LIT> ) ; LaneBitmask Common = RR . Mask


<s>
OS
<<
'{'
;
for
(
auto
I
:
Masks
)
OS
<<
' '
<<
PrintReg
(
I
.
first
,
&
PRI

. getTRI ( ) ) << PrintLaneMaskOpt ( I . second ) ;


<s> const BitVector & getMaskUnits ( RegisterId MaskId )


<s> const uint32_t * getRegMaskBits ( RegisterId R )


<s> return Register :: index2StackSlot ( RegMasks .


<s> return Register :: index2StackSlot ( RegMasks . find


<s>
for
(
int
U
=
Units
.
find_first
(
)
;
U
>=
<NUM_LIT>
;
U
=
Units
.
find_next
(
U
)
)
OS
<<
' '
<<
PrintRegUnit
(
U
,
&
PRI
.
getTRI
(
)
)
;
OS
<<


<s> uint32_t SubA = TRI . getSubRegIndex ( RA . Reg , Root ) ; uint32_t SubB = TRI . getSubRegIndex ( RB . Reg , Root ) ; const TargetRegisterClass * RC = RegInfos [ Root ] . RegClass ; LaneBitmask RCMask = RC != nullptr ? RC -> LaneMask : LaneBitmask ( <NUM_LIT> ) ; LaneBitmask MaskA = TRI . reverseComposeSubRegIndexLaneMask ( SubA , LA ) ; LaneBitmask MaskB = TRI . reverseComposeSubRegIndexLaneMask ( SubB , LB ) ; if ( ( MaskA & MaskB & RCMask ) . any ( ) ) return true ; } ++ UMA ; ++ UMB


<s> RegisterRef NR = normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F != Masks . end ( ) ) { if ( ( F -> second & NR . Mask ) . any ( ) ) return true ; } if ( CheckUnits ) { for ( MCRegUnitIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid ( ) ; ++ U ) if ( ExpAliasUnits . test ( *


<s> const TargetRegisterInfo & TRI = PRI . getTRI ( ) ; for ( MCRegUnitIterator U ( RR . Reg , & TRI ) ; U . isValid ( ) ; ++ U ) { MCRegUnitRootIterator R ( * U , & TRI ) ; ++ R ; if ( ! R . isValid ( )


<s> RegisterAggr & RegisterAggr :: insert ( RegisterRef RR ) { RegisterRef NR = normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F == Masks . end ( ) ) Masks . insert ( { NR . Reg , NR . Mask } ) ; else F -> second |= NR . Mask ; const TargetRegisterInfo & TRI = PRI . getTRI ( ) ; for ( MCRegUnitIterator U ( RR . Reg , & TRI ) ; U . isValid ( ) ; ++ U ) { MCRegUnitRootIterator R ( * U , & TRI ) ; ++ R ; if ( ! R . isValid ( ) ) continue ; ExpAliasUnits . set ( * U ) ; CheckUnits =


<s> LaneBitmask Common = RR . Mask & RCMask ; RegisterId SuperReg = PRI . RegInfos [ RR . Reg ] . MaxSuper ; uint32_t Sub = PRI . getTRI ( ) . getSubRegIndex ( SuperReg , RR . Reg ) ; LaneBitmask SuperMask = PRI . getTRI ( ) . composeSubRegIndexLaneMask ( Sub , Common ) ; return RegisterRef ( SuperReg ,


<s> if ( RI . RegClass != nullptr && ! BadRC [ R ] ) { if ( RC -> LaneMask != RI . RegClass -> LaneMask ) { BadRC . set ( R ) ; RI . RegClass = nullptr ; } } else RI . RegClass = RC ; } } for ( MCPhysReg R = <NUM_LIT> , NR = TRI . getNumRegs ( ) ; R !=


<s> } RegisterRef NR = PRI . normalize ( RR ) ; auto F = Masks . find ( NR . Reg ) ; if ( F != Masks . end ( ) ) { if ( ( F -> second & NR . Mask ) . any ( ) ) return true ; } if ( CheckUnits || PRI . hasPartialOverlaps ( NR . Reg ) ) { for ( MCRegUnitMaskIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid ( ) ; ++ U ) { std :: pair < RegisterId , LaneBitmask > P


<s> if ( PhysicalRegisterInfo :: isRegMaskId ( RR . Reg ) ) { const uint32_t * MB = PRI . getRegMaskBits ( RR . Reg ) ; for ( unsigned i = <NUM_LIT> , e = PRI . getTRI ( ) . getNumRegs ( ) ; i != e ; ++ i ) { if ( MB [ i / <NUM_LIT> ] & ( <NUM_LIT> << ( i % <NUM_LIT> ) ) ) continue ; if ( ! hasCoverOf ( RegisterRef ( i , LaneBitmask :: getAll ( ) ) ) ) return false ; } return true ; } RegisterRef NR = PRI . normalize ( RR ) ; if ( NR . Mask . none ( ) ) return true ; auto F = Masks . find ( NR . Reg ) ; if ( F != Masks . end ( ) ) { if ( ( NR . Mask & F -> second )


<s> if ( PRI . hasPartialOverlaps ( NR . Reg ) ) { for ( MCRegUnitMaskIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid ( ) ; ++ U ) { std :: pair < RegisterId , LaneBitmask > P = * U ; if ( P . second . none ( ) || ( P . second & RR . Mask ) . none ( ) ) continue ; ExpUnits . set ( P . first ) ; CheckUnits =


<s> if ( RI . RegClass != nullptr && ! BadRC [ R ] ) { if ( RC -> LaneMask != RI . RegClass -> LaneMask ) { BadRC . set ( R ) ; RI . RegClass = nullptr ; } } else RI . RegClass = RC ; } } auto HasPartialOverlaps = [ this ] ( uint32_t Reg ) -> bool { for ( MCRegAliasIterator A ( Reg , & TRI , false ) ; A . isValid ( ) ; ++ A ) if ( ! TRI . isSubRegister ( Reg , * A ) && ! TRI . isSubRegister ( * A , Reg ) ) return true ; return


<s> std :: set < RegisterId > PhysicalRegisterInfo :: getAliasSet ( RegisterId Reg ) const { std :: set < RegisterId > AS ; assert ( isRegMaskId ( Reg ) || Register :: isPhysicalRegister ( Reg ) ) ; if ( isRegMaskId ( Reg ) ) { const uint32_t * MB = getRegMaskBits ( Reg ) ; for ( unsigned i = <NUM_LIT> , e = TRI . getNumRegs ( ) ; i != e ; ++ i ) { if ( MB [ i / <NUM_LIT> ] & ( <NUM_LIT> << ( i % <NUM_LIT> ) ) ) continue ; AS . insert ( i ) ; } for ( const uint32_t * RM : RegMasks ) { RegisterId MI = getRegMaskId ( RM ) ; if ( MI != Reg && aliasMM ( RegisterRef ( Reg ) , RegisterRef ( MI ) ) ) AS . insert ( MI ) ; } return AS ; } for ( MCRegAliasIterator AI ( Reg , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . insert ( * AI ) ; for ( const uint32_t * RM : RegMasks


<s> if ( ! isRegMaskId ( RA . Reg ) ) return ! isRegMaskId ( RB . Reg ) ? aliasRR ( RA , RB ) : aliasRM ( RA , RB ) ; return ! isRegMaskId ( RB . Reg ) ? aliasRM ( RB , RA ) : aliasMM ( RA


<s> if ( w == <NUM_LIT> ) C &= ~ <NUM_LIT> ; if ( C ) return true ; } unsigned TailRegs = NumRegs % <NUM_LIT> ; if ( TailRegs == <NUM_LIT> ) return false ; unsigned TW = NumRegs / <NUM_LIT> ; uint32_t TailMask = ( <NUM_LIT> << TailRegs ) - <NUM_LIT> ; if ( ~ BM [ TW ] & ~ BN [ TW ] & TailMask ) return true ; return


<s> assert ( isRegMaskId ( RM . Reg ) && isRegMaskId ( RN . Reg ) ) ; unsigned NumRegs = TRI . getNumRegs ( ) ; const uint32_t * BM = getRegMaskBits ( RM . Reg ) ; const uint32_t * BN = getRegMaskBits ( RN . Reg ) ; for ( unsigned w = <NUM_LIT> , nw = NumRegs / <NUM_LIT> ; w != nw ; ++ w ) { uint32_t C = ~ BM [ w ] & ~ BN [ w ] ; if ( w == <NUM_LIT> ) C &= ~ <NUM_LIT> ; if ( C ) return


<s> for ( MCSubRegIndexIterator SI ( RR . Reg , & TRI ) ; SI . isValid ( ) ; ++ SI ) { LaneBitmask SM = TRI . getSubRegIndexLaneMask ( SI . getSubRegIndex ( ) ) ; if ( ( SM & RR . Mask ) . none ( ) ) continue ; unsigned SR = SI . getSubReg ( ) ; if ( ! ( MB [ SR / <NUM_LIT> ] & ( <NUM_LIT> << ( SR % <NUM_LIT> ) ) ) ) continue ; M &= ~ SM ; if ( M . none ( ) ) return false ; } return true


<s> Units . reset ( RG . Units ) ; return *


<s> RegisterRef RegisterAggr :: clearIn ( RegisterRef RR )


<s> if ( isRegMaskId ( Reg ) ) { const uint32_t * MB = getRegMaskBits ( Reg ) ; for ( unsigned i = <NUM_LIT> , e = TRI . getNumRegs ( ) ; i != e ; ++ i ) { if ( MB [ i / <NUM_LIT> ] & ( <NUM_LIT> << ( i % <NUM_LIT> ) ) ) continue ; AS . insert ( i ) ; } for ( const uint32_t * RM : RegMasks ) { RegisterId MI = getRegMaskId ( RM ) ; if ( MI != Reg && aliasMM ( RegisterRef ( Reg ) , RegisterRef ( MI ) ) ) AS . insert ( MI ) ; } return AS ; } for ( MCRegAliasIterator AI ( Reg , & TRI , false ) ; AI . isValid ( ) ; ++ AI ) AS . insert ( * AI ) ; for ( const uint32_t * RM : RegMasks


<s> const BitVector & getMaskUnits ( RegisterId MaskId ) const { return MaskInfos [ TargetRegisterInfo :: stackSlot2Index ( MaskId ) ] .


<s> RegisterRef getRefForUnit ( uint32_t U ) const { return RegisterRef ( UnitInfos [ U ] . Reg


<s> const uint32_t * getRegMaskBits ( RegisterId R ) const { return RegMasks . get ( TargetRegisterInfo :: stackSlot2Index ( R


<s> return RegMasks . get ( TargetRegisterInfo :: stackSlot2Index (


<s> RegisterId getRegMaskId ( const uint32_t * RM ) const { return TargetRegisterInfo :: index2StackSlot ( RegMasks .


<s> if ( PhysicalRegisterInfo :: isRegMaskId ( RR . Reg ) ) return Units . anyCommon ( PRI . getMaskUnits ( RR . Reg ) ) ; for ( MCRegUnitMaskIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid (


<s> BitVector T ( PRI . getMaskUnits ( RR . Reg ) ) ; return T . reset ( Units ) . none ( ) ; } for ( MCRegUnitMaskIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid ( ) ; ++ U ) { std :: pair < uint32_t , LaneBitmask > P = * U ; if ( P . second . none ( ) || ( P . second & RR . Mask ) . any ( ) ) if ( ! Units . test ( P . first ) ) return false ; } return true


<s> return T . reset ( Units ) . none ( ) ; } for ( MCRegUnitMaskIterator U ( RR . Reg , & PRI . getTRI ( ) ) ; U . isValid ( ) ; ++ U ) { std :: pair < uint32_t , LaneBitmask > P = * U ; if ( P . second . none ( ) || ( P . second & RR . Mask ) . any ( ) ) if ( ! Units . test ( P . first ) ) return false ; } return true


<s> RegisterAggr & RegisterAggr :: insert ( const RegisterAggr & RG ) { Units |= RG . Units


<s> return TargetRegisterInfo :: isStackSlot (


<s> for ( MCRegUnitRootIterator R ( Unit , & PRI . getTRI ( ) ) ; R . isValid ( ) ; ++ R ) for ( MCSuperRegIterator S ( * R , & PRI . getTRI ( ) , true ) ; S . isValid ( ) ; ++ S ) Regs . set ( * S ) ; } ; BitVector Regs ( PRI . getTRI ( ) . getNumRegs ( ) ) ; AliasedRegs ( U , Regs ) ; U = Units . find_next ( U ) ; while ( U >= <NUM_LIT> ) { BitVector AR ( PRI . getTRI ( ) . getNumRegs ( ) ) ; AliasedRegs ( U , AR ) ; Regs &= AR ; U = Units . find_next ( U ) ; } int F = Regs . find_first ( ) ; if ( F <= <NUM_LIT> ) return RegisterRef ( ) ; LaneBitmask M ; for ( MCRegUnitMaskIterator I ( F , & PRI . getTRI ( ) ) ; I . isValid ( ) ; ++ I ) { std :: pair < uint32_t , LaneBitmask


<s> const RegInfo & RI = RegInfos [ R ] ; LaneBitmask RCM = RI . RegClass ? RI . RegClass -> LaneMask : LaneBitmask :: getAll ( ) ; LaneBitmask M = TRI . reverseComposeSubRegIndexLaneMask ( Idx , RR . Mask ) ; return RegisterRef ( R , M & RCM ) ; } llvm_unreachable (


<s> RegisterRef PhysicalRegisterInfo :: normalize ( RegisterRef RR ) const { return


<s> } } UnitInfos . resize ( TRI . getNumRegUnits ( ) ) ; for ( uint32_t U = <NUM_LIT> , NU = TRI . getNumRegUnits ( ) ; U != NU ; ++ U ) { if ( UnitInfos [ U ] . Reg != <NUM_LIT> ) continue ; MCRegUnitRootIterator R ( U , & TRI ) ; assert ( R . isValid ( ) ) ; RegisterId F = * R ; ++ R ; if ( R . isValid ( ) ) { UnitInfos [ U ] . Mask = LaneBitmask :: getAll ( ) ; UnitInfos [ U ] . Reg = F ; } else { for ( MCRegUnitMaskIterator I ( F , & TRI ) ; I . isValid ( ) ; ++ I ) { std :: pair < uint32_t , LaneBitmask > P = * I ; UnitInfo & UI = UnitInfos [ P . first ] ; UI . Reg = F ; if ( P . second . any ( ) ) { UI . Mask = P


<s>
for
(
int
U
=
Units
.
find_first
(
)
;
U
>=
<NUM_LIT>
;
U
=
Units
.
find_next
(
U
)
)
OS
<<
' '
<<
printRegUnit
(
U
,
&
PRI
.
getTRI
(
)
)
;
OS
<<


<s> rr_iterator rr_begin ( ) const { return rr_iterator ( * this ,


<s> rr_iterator rr_end ( ) const { return rr_iterator ( * this