title: Not expected result when using KNN Imputer
body: <pre><code>#Load the dataframe
  df.drop(labels=['Unnamed: 0'], axis=1,inplace=True)
  
  #Drop unnecessary columns
  col_to_drop = ['ACTECDR', 'APRESR', 'APRESM', 'ACTECDM', 'TCDM',
                 'BLOCKCOMP','TOTGAS','SSRPMAVM', 'SSRPMINM', 
                 'SSRPMAXM', 'HERC_V2_AVGZM','HERC_V2_MAXXYM', 
                 'HERC_V2_AVGXYM', 'HERC_V2_MAXZM','HKLD_AVG']
  
  df.drop(labels=col_to_drop,axis=1,inplace=True)
  df.replace(-999.25,np.nan,inplace=True)
  
  #Define a dataframe with only numerical values
  df_num = df.drop(labels='TIME',axis=1)
  
  #Create train and test sets
  train = df_num.sample(frac=0.7,random_state=200)
  test = df_num.drop(train.index)
  
  #Normalizing train and test sets
  scaler = preprocessing.MinMaxScaler(feature_range=(0,1))
  norm = scaler.fit(train)
  train_norm = norm.transform(train)
  df_train_norm = pd.DataFrame(data=train_norm, columns=list(df_num.columns))
  df_train_norm.set_index(train.index, inplace=True)
  test_norm = norm.transform(test)
  df_test_norm = pd.DataFrame(data=test_norm, columns=list(df_num.columns))
  df_test_norm.set_index(test.index, inplace=True)
  
  df_norm = pd.concat([df_train_norm,df_test_norm],axis=0,ignore_index=False)
  df_norm.sort_index(axis=0,ascending=True,inplace=True)
  
  #After evaluating mean and std. dev. before and after KNN imputer, 102 neighbors seemed to be the best choice
  
  #Using KNN Imputer with 102 neighbors
  n_neighbors = 102
  knn = KNNImputer(n_neighbors=n_neighbors)
  
  #KNN Imputer in train set
  train_trans = pd.DataFrame(knn.fit_transform(df_train_norm),columns=list(df_norm.columns))
  
  #KNN Imputer in test set
  test_trans = pd.DataFrame(knn.transform(df_test_norm), columns=list(df_norm.columns))
  
  train_trans.set_index(train.index, inplace=True)
  test_trans.set_index(test.index, inplace=True)
  
  #Concatenate and sorting train and test set
  df_norm_trans = pd.concat([train_trans,test_trans],axis=0,ignore_index=False)
  df_norm_trans.sort_index(axis=0, ascending=True,inplace=True)
  
  #Denormalizing data
  df_denorm_trans = scaler.inverse_transform(df_norm_trans)
  df_denorm_trans = pd.DataFrame(df_denorm_trans,columns=list(df_norm_trans.columns))
  </code></pre>
  <p>Once I got the dataframe with No Nan and denormalized. I plotted BITDEP and what I got was this:
  <a href="https://i.stack.imgur.com/FhacS.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/FhacS.png" alt="enter image description here" /></a></p>
  <p>I don't know why that &quot;horizontal line&quot; appeared on approximately 315 meters. Is there something wrong in the code ? or Did I apply wrong the KNN Imputer?</p>
  <p>Also I try to plot other variable of the dataframe after KNN Imputer and denormalizing and always appeared that &quot;horizontal line&quot; but in different value.</p>
  <p>I leave this link where you can find the data that I used. This is part of a research that's why I can't give all the information, I hope you guys can help me.</p>
  <p><a href="https://docs.google.com/spreadsheets/d/1qceLImqCIrT7QacHOsFYrAxkvVuFb81L/edit?usp=sharing&amp;ouid=115792207054787691125&amp;rtpof=true&amp;sd=true" rel="nofollow noreferrer">https://docs.google.com/spreadsheets/d/1qceLImqCIrT7QacHOsFYrAxkvVuFb81L/edit?usp=sharing&amp;ouid=115792207054787691125&amp;rtpof=true&amp;sd=true</a></p>