title: How to Avoid very long time python calculations while iterating through pandas dataframe
body: <p>I am trying to add 2 columns to dataframe to calculate vehicle surroundings neighbor 50 and neighbor 100 for each vehicle in each direction and in each time frame based on distance, if distance below 50 I add one count to the neighbor 50 and so on, I should do this task using Pandas only</p>
  <p>I should calculate distance based on x and y position of each vehicle through the equation:</p>
  <p>distance = ((x2 - x1)**2 + (y2 - y1)**2)**0.5</p>
  <p>I have used this code:</p>
  <pre><code>#import numpy as np
  
  df['neighbor_50'] = 0
  df['neighbor_100'] = 0
  frame_group =  df.groupby(['frame','direction'])
  list_keys = list(frame_group.indices.keys())
  
  for key in list_keys :
      frame , direction = key[0] , key[1]
      #new_df =  df.loc[(df['frame'] == frame) &amp; (df['direction'] == direction)]
      mask1 = (df['frame'] == frame) &amp; (df['direction'] == direction) 
      ids = df[mask1]['id']
      for i in ids:
          for j in ids:
              if i != j:
                  #distance = sqrt((x2-x1)**2 + (y2-y1)**2)
                  maski = (df['frame'] == frame) &amp; (df['direction'] == direction)&amp; (df['id'] == i)
                  maskj = (df['frame'] == frame) &amp; (df['direction'] == direction)&amp; (df['id'] == j)
                  x2 = df[maski]['x'].iloc[0]
                  x1 = df[maskj]['x'].iloc[0]
                  y2 = df[maski]['y'].iloc[0]
                  y1 = df[maskj]['y'].iloc[0]
                  
                  distance = ((x2 - x1)**2 + (y2 - y1)**2)**0.5
                  #distance = np.hypot((x2 - x1),(y2 - y1))
                  mask = (df['frame'] == frame) &amp; (df['direction'] == direction) &amp;( df['id']== i)
                  if distance &lt;= 50:
                      df.loc[mask , 'neighbor_50'] += 1
                  if distance &lt;= 100 :
                      df.loc[mask ,'neighbor_100'] += 1
  </code></pre>
  <p>the problem is that it takes forever to complete because the data is big even when i use NumPy.</p>
  <p>input sample
  <a href="https://i.stack.imgur.com/bYF7m.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/bYF7m.png" alt="input sample" /></a></p>
  <p>output sample
  <a href="https://i.stack.imgur.com/7tmKP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7tmKP.png" alt="enter image description here" /></a></p>
  <p>update:
  I have managed to reduce the time by half by avoiding repeating calculations for same ids , but still so slow</p>
  <pre><code>import numpy as np
  
  df['neighbor_50'] = 0
  df['neighbor_100'] = 0
  frame_group =  df.groupby(['frame','direction'])
  list_keys = list(frame_group.indices.keys())
  
  for key in list_keys :
      frame , direction = key[0] , key[1]
      #new_df =  df.loc[(df['frame'] == frame) &amp; (df['direction'] == direction)]
      mask = (df['frame'] == frame) &amp; (df['direction'] == direction) 
      ids = df[mask]['id'].values
      for i in range(len(ids)-1):
          id1 = ids[i]
          for j in range(i+1,len(ids)):
              id2 = ids[j]
              
              maski = (df['frame'] == frame) &amp; (df['direction'] == direction)&amp; (df['id'] == id1)
              maskj = (df['frame'] == frame) &amp; (df['direction'] == direction)&amp; (df['id'] == id2)
              x2 = df[maski]['x'].iloc[0]
              x1 = df[maskj]['x'].iloc[0]
              y2 = df[maski]['y'].iloc[0]
              y1 = df[maskj]['y'].iloc[0]
              
              #distance = ((x2 - x1)**2 + (y2 - y1)**2)**0.5
              distance = np.hypot((x2 - x1),(y2 - y1))
             
              if distance &lt;= 100 :
                  df.loc[maski ,'neighbor_100'] += 1
                  df.loc[maskj ,'neighbor_100'] += 1
                  
                  if distance &lt;= 50:
                      df.loc[maski , 'neighbor_50'] += 1
                      df.loc[maskj , 'neighbor_50'] += 1
  </code></pre>