title: Data Imputation with Mean in Python
body: <p>I'm working with some data where I have hourly observations for patients. In some cases, some of the features for a specific patient are completely empty. I'm trying to find a way to impute the data by using constant average that's based off a population subset of 50 other patients who have the same gender and a similar age. I've given a simplified look at the data below:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>HR</th>
  <th>O2Sat</th>
  <th>Temp</th>
  <th>Platelets</th>
  <th>Age</th>
  <th>Gender</th>
  <th>PatientID</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>80</td>
  <td>98</td>
  <td>36.5</td>
  <td>NaN</td>
  <td>52</td>
  <td>1</td>
  <td>A0</td>
  </tr>
  <tr>
  <td>82</td>
  <td>96</td>
  <td>37.0</td>
  <td>NaN</td>
  <td>52</td>
  <td>1</td>
  <td>A0</td>
  </tr>
  <tr>
  <td>82</td>
  <td>100</td>
  <td>36.3</td>
  <td>160</td>
  <td>53</td>
  <td>1</td>
  <td>A1</td>
  </tr>
  <tr>
  <td>90</td>
  <td>93</td>
  <td>36.6</td>
  <td>165</td>
  <td>53</td>
  <td>1</td>
  <td>A1</td>
  </tr>
  <tr>
  <td>83</td>
  <td>95</td>
  <td>35.9</td>
  <td>140</td>
  <td>23</td>
  <td>0</td>
  <td>A2</td>
  </tr>
  <tr>
  <td>79</td>
  <td>98</td>
  <td>36.2</td>
  <td>155</td>
  <td>23</td>
  <td>0</td>
  <td>A2</td>
  </tr>
  <tr>
  <td>88</td>
  <td>92</td>
  <td>36.6</td>
  <td>163</td>
  <td>60</td>
  <td>0</td>
  <td>A3</td>
  </tr>
  <tr>
  <td>90</td>
  <td>91</td>
  <td>36.3</td>
  <td>165</td>
  <td>60</td>
  <td>0</td>
  <td>A3</td>
  </tr>
  <tr>
  <td>81</td>
  <td>95</td>
  <td>37.1</td>
  <td>NaN</td>
  <td>20</td>
  <td>0</td>
  <td>A4</td>
  </tr>
  <tr>
  <td>81</td>
  <td>92</td>
  <td>36.9</td>
  <td>NaN</td>
  <td>20</td>
  <td>0</td>
  <td>A4</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>I've reordered the dataframe by age and have this code so far</p>
  <p><code>data = data.sort_values(['Age']).groupby(['PatientID','Gender']).apply(lambda x: x.fillna(x.mean()))</code></p>
  <p>But I know that that's going to use all of the available data to find the mean but I'm not sure how to limit it to 50 patients of a similar age.</p>