title: Correspondence between R's caret knnImpute and Python's sklearn.impute KNNImputer results
body: <p>I am trying to replicate a machine learning process across different programming platforms but I am getting different imputed values from R Caret's preProcessing function compared to python's sklearn process. Using the example dataset and process below:</p>
  <pre><code>library(caret)
  
  set.seed(197)
  simulated.ds &lt;- data.frame(
    var1 = rbinom(n=10000, size=1, prob=0.05),
    var2 = rbinom(n=10000, size=1, prob=0.4),
    var3 = rbinom(n=10000, size=1, prob=0.2),
    var4 = rbinom(n=10000, size=1, prob=0.03),
    var5 = rbinom(n=10000, size=1, prob=0.7),
    var6 = rbinom(n=10000, size=1, prob=0.1),
    var7 = rbinom(n=10000, size=1, prob=0.2)
  )
  
  
  set.seed(50)
  ind1 &lt;- sample(c(1:10000), 1250)
  simulated.ds$var1[ind1] &lt;- NA
  
  set.seed(150)
  ind2 &lt;- sample(c(1:10000), 1250)
  simulated.ds$var2[ind2] &lt;- NA
  
  set.seed(1000)
  ind5 &lt;- sample(c(1:10000), 1250)
  simulated.ds$var5[ind5] &lt;- NA
  
  set.seed(500)
  ind6 &lt;- sample(c(1:10000), 1250)
  simulated.ds$var6[ind6] &lt;- NA
  
  write.csv(simulated.ds, &quot;rawDataR.csv&quot;, row.names = F)
  
  prepRoutine &lt;- caret::preProcess(simulated.ds, method = &quot;knnImpute&quot;, k=5)
  imputed_dataset &lt;- predict(prepRoutine, simulated.ds)
  
  write.csv(imputed_dataset, &quot;imputedDataR.csv&quot;, row.names = F)
  </code></pre>
  <p>When I preprocess the same simulated dataset by trying to replicate the imputation process in Python 3 using the code below, the results from the imputed dataset are different.</p>
  <pre><code>import random
  import pandas as pd
  import numpy as np
  from sklearn.impute import KNNImputer
  from sklearn.preprocessing import StandardScaler
  from sklearn.pipeline import Pipeline
  
  rawDataR = pd.read_csv(&quot;C:/Users/AfrikanaScholar/Documents/rawDataR.csv&quot;)
  imputedDataR = pd.read_csv(&quot;C:/Users/AfrikanaScholar/Documents/imputedDataR.csv&quot;)
  
  random.seed(197)
  scaler = StandardScaler()
  imputer = KNNImputer(n_neighbors=5, weights=&quot;distance&quot;)
  
  data_pipeline = Pipeline([
      ('scaler', scaler),
      ('imputer', imputer)
  ])
  
  imputedDataPy = data_pipeline.fit_transform(rawDataR)
  
  dataPy = pd.DataFrame(imputedDataPy).round(3)
  dataR = imputedDataR.round(3) 
  
  np.array_equal(dataR.values, dataPy.values)
  &gt;&gt; ***False***
  </code></pre>
  <p>My questions are:</p>
  <ol>
  <li><p>Why would the different platforms, taking the same approach, using the same data, produce such differences? For example, using the imputed values from the 5th column,</p>
  <p>From python 3:</p>
  <pre><code> Counter({
     -1.522: 2644,
     -1.086: 55,
     -0.65: 193,
     -0.215: 485,
      0.221: 478,
      0.398: 1,
      0.657: 6144        
      })
  </code></pre>
  <p>From R:</p>
  <pre><code>     -1.522 -1.086  -0.65 -0.215  0.221  0.657 
        2638      3    222    272    582   6283
  </code></pre>
  </li>
  <li><p>What could be done to ensure that the values between the different platforms are as similar as possible?</p>
  </li>
  </ol>
  <p>Such discrepancies would make the modelling process produce varying findings across different platforms.</p>