title: What is the fastest way to generate a dynamic lag variable that updates with each iteration?
body: <p>I have a few hundred thousand groups through which I want to iterate this particular lag operation. Below is a sample where Buy_Ord_No is the group by variable:</p>
  
  <p><a href="https://i.stack.imgur.com/2wA1B.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2wA1B.png" alt="Sample Table\e"></a></p>
  
  <p>I would like to generate Lag_Exec_Qty and Exec_Qty. What I am basically doing here is initially setting Exec_Qty equal to 0 when Buy_Act_Type = 1 or Buy_Act_Type = 4. Then, I take the lag value of Exec_Qty ad Lag_Exec_Qty. In the same row, I sum up Trd_Qty and Lag_Exec_Qty to get the updated Exec_Qty.</p>
  
  <p>This is the code that I currently have:</p>
  
  <pre><code>for b in buy:
      temp=buy_sorted_file[buy_sorted_file["Buy_Ord_No"]==b]
      temp=temp.sort_values(["Buy_Ord_No","Buy_Ord_Txn_Time"], ascending=[True, True]).reset_index(drop=True)
  
      for index in range(len(temp.index)):
          if(int(temp["Buy_Act_Type"].iloc[index])==1 or int(temp["Buy_Act_Type"].iloc[index])==4):
              temp["Exec_Qty"].iloc[index]=0
              temp["Lag_Exec_Qty"].iloc[index]=0
          else:
              temp["Lag_Exec_Qty"].iloc[index]=temp["Exec_Qty"].iloc[index-1]
              temp["Exec_Qty"].iloc[index]=temp["Trd_Qty"].iloc[index]+temp["Lag_Exec_Qty"].iloc[index]
      if (len(buy_sorted_exec_file.index) == 0):
          buy_sorted_exec_file = temp.copy()
      else:
          buy_sorted_exec_file = pd.concat([temp,buy_sorted_exec_file]).reset_index(drop=True)
  buy_sorted_file= buy_sorted_exec_file.sort_values(["Buy_Ord_Txn_Time", "Buy_Ord_Limit_Pr"],ascending=[True, True]).reset_index(drop=True)
  </code></pre>
  
  <p>The code takes a really long time to run. Is there anyway through which I can speed this process up?</p>