title: Data cleaning using pandas generating index error
body: <p>I'm using <code>jupyter notbook</code> and trying to do some data cleaning using <code>pandas</code> dataframe but having some issues regarding <code>indexing</code>.</p>
  <p>my dataframe is</p>
  <pre><code>import numpy as np
  import pandas as pd
  
  df= pd.DataFrame({'ID':{0:'0x1602',1:'0x1603',2:'0x1604',3:'0x1605',4:'0x1606',5:'0x1607',6:'0x1608',7:'0x1609',8:'0x160e',9:'0x160f',10:'0x160f',11:'0x1611',12:'0x1612',13:'0x1613',14:'0x1614',15:'0x1615',16:'0x161a',17:'0x161b',18:'0x161c',19:'0x161d'},'Customer_ID':{0:'CUS_0xd40',1:'CUS_0xd40',2:'CUS_0xd40',3:'CUS_0xd40',4:'CUS_0xd40',5:'CUS_0xd40',6:'CUS_0xd40',7:'CUS_0xd40',8:'CUS_0x21b1',9:'CUS_0x21b1',10:'CUS_0x21b1',11:'CUS_0x21b1',12:'CUS_0x21b1',13:'CUS_0x21b1',14:'CUS_0x21b1',15:'CUS_0x21b1',16:'CUS_0x2dbc',17:'CUS_0x2dbc',18:'CUS_0x2dbc',19:'CUS_0x2dbc'},'Name':{0:'AaronMaashoh',1:'AaronMaashoh',2:'AaronMaashoh',3:'AaronMaashoh',4:'AaronMaashoh',5:'AaronMaashoh',6:'AaronMaashoh',7:np.nan,8:'RickRothackerj',9:'RickRothackerj',10:np.nan,11:'RickRothackerj',12:'RickRothackerj',13:'RickRothackerj',14:'RickRothackerj',15:'RickRothackerj',16:'Langep',17:np.nan,18:'Langep',19:'Langep'},'Num_of_Delayed_Payment':{0:7,1:4,2:7,3:4,4:4,5:np.nan,6:8,7:6,8:4,9:1,10:0,11:3,12:1,13:0,14:4,15:4,16:8,17:6,18:np.nan,19:5}})
  df
      ID      Cus_ID      Name           Num_of_Delayed_Payment
  0   0x1602  CUS_0xd40   AaronMaashoh    7.0
  1   0x1603  CUS_0xd40   AaronMaashoh    4.0
  2   0x1604  CUS_0xd40   AaronMaashoh    7.0
  3   0x1605  CUS_0xd40   AaronMaashoh    4.0
  4   0x1606  CUS_0xd40   AaronMaashoh    4.0
  5   0x1607  CUS_0xd40   AaronMaashoh    NaN
  6   0x1608  CUS_0xd40   AaronMaashoh    8.0
  7   0x1609  CUS_0xd40   NaN             6.0
  8   0x160e  CUS_0x21b1  RickRothackerj  4.0
  9   0x160f  CUS_0x21b1  RickRothackerj  1.0
  10  0x160f  CUS_0x21b1  NaN             0.0
  11  0x1611  CUS_0x21b1  RickRothackerj  3.0
  12  0x1612  CUS_0x21b1  RickRothackerj  1.0
  13  0x1613  CUS_0x21b1  RickRothackerj  0.0
  14  0x1614  CUS_0x21b1  RickRothackerj  3.0
  15  0x1615  CUS_0x21b1  RickRothackerj  4.0
  16  0x161a  CUS_0x2dbc  Langep          8.0
  17  0x161b  CUS_0x2dbc  NaN             6.0
  18  0x161c  CUS_0x2dbc  Langep          NaN
  19  0x161d  CUS_0x2dbc  Langep          6.0
  
  </code></pre>
  <p>After running the below piece of <code>code cell</code></p>
  <pre><code>df['Num_of_Delayed_Payment'] = df['Num_of_Delayed_Payment'].fillna(df.groupby('Customer_ID')['Num_of_Delayed_Payment'].transform(lambda x:x.mode().iat[0]))
  df['Num_of_Delayed_Payment'] = df['Num_of_Delayed_Payment'].str.rstrip('_') 
  df[&quot;Num_of_Delayed_Payment&quot;] = pd.to_numeric(df[&quot;Num_of_Delayed_Payment&quot;])
  
  lower_percentile = 0.01
  higher_percentile = 0.87
  #returns values at the given quantile within the specified range
  low, high = df['Num_of_Delayed_Payment'].quantile([lower_percentile, higher_percentile])
  #cap values below low to low
  df[df['Num_of_Delayed_Payment'] &lt; low] = low
  #cap values above high to high
  df[df['Num_of_Delayed_Payment'] &gt; high] = high
  
  </code></pre>
  <p>all the <code>code cells</code> which contained similar operation of filling <code>missing values</code> generates <code>Index</code> error for example see below <code>code cell</code>:</p>
  <p><code>df['Name']=df['Name'].fillna(df.groupby('Customer_ID')['Name'].transform(lambda x:x.mode().iat[0]))</code></p>
  <pre><code>Input In [31], in &lt;cell line: 1&gt;()
  ----&gt; 1 df['Name']=df['Name'].fillna(df.groupby('Customer_ID')['Name'].transform(lambda x:x.mode().iat[0]))
  
  File ~\anaconda3\lib\site-packages\pandas\core\groupby\generic.py:428, in SeriesGroupBy.transform(self, func, engine, engine_kwargs, *args, **kwargs)
      425 @Substitution(klass=&quot;Series&quot;)
      426 @Appender(_transform_template)
      427 def transform(self, func, *args, engine=None, engine_kwargs=None, **kwargs):
  --&gt; 428     return self._transform(
      429         func, *args, engine=engine, engine_kwargs=engine_kwargs, **kwargs
      430     )
  
  File ~\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py:1633, in GroupBy._transform(self, func, engine, engine_kwargs, *args, **kwargs)
     1630 func = com.get_cython_func(func) or func
     1632 if not isinstance(func, str):
  -&gt; 1633     return self._transform_general(func, *args, **kwargs)
     1635 elif func not in base.transform_kernel_allowlist:
     1636     msg = f&quot;'{func}' is not a valid function name for transform(name)&quot;
  
  File ~\anaconda3\lib\site-packages\pandas\core\groupby\generic.py:459, in SeriesGroupBy._transform_general(self, func, *args, **kwargs)
      456 for name, group in self:
      457     # this setattr is needed for test_transform_lambda_with_datetimetz
      458     object.__setattr__(group, &quot;name&quot;, name)
  --&gt; 459     res = func(group, *args, **kwargs)
      461     results.append(klass(res, index=group.index))
      463 # check for empty &quot;results&quot; to avoid concat ValueError
  
  Input In [31], in &lt;lambda&gt;(x)
  ----&gt; 1 df['Name']=df['Name'].fillna(df.groupby('Customer_ID')['Name'].transform(lambda x:x.mode().iat[0]))
  
  File ~\anaconda3\lib\site-packages\pandas\core\indexing.py:2221, in _ScalarAccessIndexer.__getitem__(self, key)
     2218         raise ValueError(&quot;Invalid call for scalar access (getting)!&quot;)
     2220 key = self._convert_key(key)
  -&gt; 2221 return self.obj._get_value(*key, takeable=self._takeable)
  
  File ~\anaconda3\lib\site-packages\pandas\core\series.py:1066, in Series._get_value(self, label, takeable)
     1053 &quot;&quot;&quot;
     1054 Quickly retrieve single value at passed index label.
     1055 
     (...)
     1063 scalar value
     1064 &quot;&quot;&quot;
     1065 if takeable:
  -&gt; 1066     return self._values[label]
     1068 # Similar to Index.get_value, but we do not fall back to positional
     1069 loc = self.index.get_loc(label)
  
  IndexError: index 0 is out of bounds for axis 0 with size 0
  </code></pre>
  <p>Expected output should look like</p>
  <pre><code>
      ID      Cus_ID      Name           Num_of_Delayed_Payment
  0   0x1602  CUS_0xd40   AaronMaashoh    7.0
  1   0x1603  CUS_0xd40   AaronMaashoh    4.0
  2   0x1604  CUS_0xd40   AaronMaashoh    7.0
  3   0x1605  CUS_0xd40   AaronMaashoh    4.0
  4   0x1606  CUS_0xd40   AaronMaashoh    4.0
  5   0x1607  CUS_0xd40   AaronMaashoh    4.0
  6   0x1608  CUS_0xd40   AaronMaashoh    8.0
  7   0x1609  CUS_0xd40   AaronMaashoh    6.0
  8   0x160e  CUS_0x21b1  RickRothackerj  4.0
  9   0x160f  CUS_0x21b1  RickRothackerj  1.0
  10  0x160f  CUS_0x21b1  RickRothackerj  0.0
  11  0x1611  CUS_0x21b1  RickRothackerj  3.0
  12  0x1612  CUS_0x21b1  RickRothackerj  1.0
  13  0x1613  CUS_0x21b1  RickRothackerj  0.0
  14  0x1614  CUS_0x21b1  RickRothackerj  3.0
  15  0x1615  CUS_0x21b1  RickRothackerj  4.0
  16  0x161a  CUS_0x2dbc  Langep          8.0
  17  0x161b  CUS_0x2dbc  Langep          6.0
  18  0x161c  CUS_0x2dbc  Langep          6.0
  19  0x161d  CUS_0x2dbc  Langep          6.0
  
  </code></pre>
  <p>please help me understand the error and the solution.</p>
  <p>Thanks</p>