<h3 dir="auto">Description</h3>
<p dir="auto">IterativeImputer seems to give pretty rubbish values when using the two-step <code class="notranslate">fit()</code> and <code class="notranslate">transform()</code> on a train/test data split versus simply using <code class="notranslate">fit_transform()</code> on the entire dataset. Issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="467756971" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/14338" data-hovercard-type="issue" data-hovercard-url="/scikit-learn/scikit-learn/issues/14338/hovercard" href="https://github.com/scikit-learn/scikit-learn/issues/14338">#14338</a> may be related.</p>
<h4 dir="auto">Steps/Code to Reproduce</h4>
<p dir="auto">Here's a fairly minimal example:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer
from sklearn.linear_model import LinearRegression
import numpy as np

# x_0 = x_1 + x_2 + x_3
train = np.array([
    [ 5, 2, 2, 1],
    [10, 1, 2, 7],
    [ 3, 1, 1, 1],
    [ 8, 4, 2, 2]
])

test = np.array([
    [np.nan, 2, 4, 5],
    [np.nan, 4, 1, 2],
    [np.nan, 1, 10, 1]
])

y_actual = np.array([11, 7, 12])

## Prediction using IterativeImputer and  the 1-step fit_transform()
imputer     = IterativeImputer(estimator = LinearRegression())
fullData    = np.concatenate([train,test])
imputedData = imputer.fit_transform(fullData)
y_1imputed  = imputedData[-3:,0]

## Prediction using IterativeImputer and the 2-step fit() / transform()
imputer2     = IterativeImputer(estimator = LinearRegression())
imputer2     = imputer2.fit(train)
imputedTest  = imputer2.transform(test)
y_2imputed   = imputedTest[:,0]

print('Actual y:             {}'.format(y_actual))
print('1-step Imputed y:     {}'.format(y_1imputed))
print('2-step Imputed y:     {}'.format(y_2imputed))"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">experimental</span> <span class="pl-k">import</span> <span class="pl-s1">enable_iterative_imputer</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">impute</span> <span class="pl-k">import</span> <span class="pl-v">IterativeImputer</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">linear_model</span> <span class="pl-k">import</span> <span class="pl-v">LinearRegression</span>
<span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-c"># x_0 = x_1 + x_2 + x_3</span>
<span class="pl-s1">train</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">array</span>([
    [ <span class="pl-c1">5</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>, <span class="pl-c1">1</span>],
    [<span class="pl-c1">10</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">7</span>],
    [ <span class="pl-c1">3</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>],
    [ <span class="pl-c1">8</span>, <span class="pl-c1">4</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>]
])

<span class="pl-s1">test</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">array</span>([
    [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">2</span>, <span class="pl-c1">4</span>, <span class="pl-c1">5</span>],
    [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">4</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>],
    [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">1</span>, <span class="pl-c1">10</span>, <span class="pl-c1">1</span>]
])

<span class="pl-s1">y_actual</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">array</span>([<span class="pl-c1">11</span>, <span class="pl-c1">7</span>, <span class="pl-c1">12</span>])

<span class="pl-c">## Prediction using IterativeImputer and  the 1-step fit_transform()</span>
<span class="pl-s1">imputer</span>     <span class="pl-c1">=</span> <span class="pl-v">IterativeImputer</span>(<span class="pl-s1">estimator</span> <span class="pl-c1">=</span> <span class="pl-v">LinearRegression</span>())
<span class="pl-s1">fullData</span>    <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">concatenate</span>([<span class="pl-s1">train</span>,<span class="pl-s1">test</span>])
<span class="pl-s1">imputedData</span> <span class="pl-c1">=</span> <span class="pl-s1">imputer</span>.<span class="pl-en">fit_transform</span>(<span class="pl-s1">fullData</span>)
<span class="pl-s1">y_1imputed</span>  <span class="pl-c1">=</span> <span class="pl-s1">imputedData</span>[<span class="pl-c1">-</span><span class="pl-c1">3</span>:,<span class="pl-c1">0</span>]

<span class="pl-c">## Prediction using IterativeImputer and the 2-step fit() / transform()</span>
<span class="pl-s1">imputer2</span>     <span class="pl-c1">=</span> <span class="pl-v">IterativeImputer</span>(<span class="pl-s1">estimator</span> <span class="pl-c1">=</span> <span class="pl-v">LinearRegression</span>())
<span class="pl-s1">imputer2</span>     <span class="pl-c1">=</span> <span class="pl-s1">imputer2</span>.<span class="pl-en">fit</span>(<span class="pl-s1">train</span>)
<span class="pl-s1">imputedTest</span>  <span class="pl-c1">=</span> <span class="pl-s1">imputer2</span>.<span class="pl-en">transform</span>(<span class="pl-s1">test</span>)
<span class="pl-s1">y_2imputed</span>   <span class="pl-c1">=</span> <span class="pl-s1">imputedTest</span>[:,<span class="pl-c1">0</span>]

<span class="pl-en">print</span>(<span class="pl-s">'Actual y:             {}'</span>.<span class="pl-en">format</span>(<span class="pl-s1">y_actual</span>))
<span class="pl-en">print</span>(<span class="pl-s">'1-step Imputed y:     {}'</span>.<span class="pl-en">format</span>(<span class="pl-s1">y_1imputed</span>))
<span class="pl-en">print</span>(<span class="pl-s">'2-step Imputed y:     {}'</span>.<span class="pl-en">format</span>(<span class="pl-s1">y_2imputed</span>))</pre></div>
<h4 dir="auto">Expected Results</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Actual y:             [11  7 12]
1-step Imputed y:     [11.  7. 12.]
2-step Imputed y:     [11.  7. 12.]"><pre class="notranslate"><code class="notranslate">Actual y:             [11  7 12]
1-step Imputed y:     [11.  7. 12.]
2-step Imputed y:     [11.  7. 12.]
</code></pre></div>
<h4 dir="auto">Actual Results</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Actual y:             [11  7 12]
1-step Imputed y:     [11.  7. 12.]
2-step Imputed y:     [6.5 6.5 6.5]"><pre class="notranslate"><code class="notranslate">Actual y:             [11  7 12]
1-step Imputed y:     [11.  7. 12.]
2-step Imputed y:     [6.5 6.5 6.5]
</code></pre></div>
<h4 dir="auto">Versions</h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="System:
    python: 3.7.2 (default, Dec 29 2018, 06:19:36)  [GCC 7.3.0]
executable: /home/jack/miniconda3/bin/python
   machine: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid

BLAS:
    macros: NO_ATLAS_INFO=1, HAVE_CBLAS=None
  lib_dirs: /usr/lib/x86_64-linux-gnu
cblas_libs: cblas

Python deps:
       pip: 19.1.1
setuptools: 40.2.0
   sklearn: 0.22.dev0
     numpy: 1.15.3
     scipy: 1.1.0
    Cython: 0.29.12
    pandas: 0.24.2
matplotlib: 3.0.2
    joblib: 0.13.2"><pre class="notranslate"><code class="notranslate">System:
    python: 3.7.2 (default, Dec 29 2018, 06:19:36)  [GCC 7.3.0]
executable: /home/jack/miniconda3/bin/python
   machine: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid

BLAS:
    macros: NO_ATLAS_INFO=1, HAVE_CBLAS=None
  lib_dirs: /usr/lib/x86_64-linux-gnu
cblas_libs: cblas

Python deps:
       pip: 19.1.1
setuptools: 40.2.0
   sklearn: 0.22.dev0
     numpy: 1.15.3
     scipy: 1.1.0
    Cython: 0.29.12
    pandas: 0.24.2
matplotlib: 3.0.2
    joblib: 0.13.2
</code></pre></div>
<p dir="auto">Why is this behaviour forced:</p>
<p dir="auto"><em>Features with missing values during transform which did not have any missing values during fit will be imputed with the initial imputation method only.</em></p>
<p dir="auto"><a href="https://scikit-learn.org/dev/modules/generated/sklearn.impute.IterativeImputer.html#sklearn.impute.IterativeImputer" rel="nofollow">https://scikit-learn.org/dev/modules/generated/sklearn.impute.IterativeImputer.html#sklearn.impute.IterativeImputer</a></p>
<p dir="auto">This means by default it will return the mean of that feature. I would prefer just fit one iteration of the chosen estimator and use that fitted estimator to impute missing values.</p>
<p dir="auto">Actual behaviour:<br>
Example - The second feature missing np.nan --&gt; mean imputation</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np
from sklearn.impute import IterativeImputer
imp = IterativeImputer(max_iter=10, verbose=0)
imp.fit([[1, 2], [3, 6], [4, 8], [10, 20], [np.nan, 22], [7, 14]])

X_test = [[np.nan, 4], [6, np.nan], [np.nan, 6], [4, np.nan], [33, np.nan]]
print(np.round(imp.transform(X_test)))"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">impute</span> <span class="pl-k">import</span> <span class="pl-v">IterativeImputer</span>
<span class="pl-s1">imp</span> <span class="pl-c1">=</span> <span class="pl-v">IterativeImputer</span>(<span class="pl-s1">max_iter</span><span class="pl-c1">=</span><span class="pl-c1">10</span>, <span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">0</span>)
<span class="pl-s1">imp</span>.<span class="pl-en">fit</span>([[<span class="pl-c1">1</span>, <span class="pl-c1">2</span>], [<span class="pl-c1">3</span>, <span class="pl-c1">6</span>], [<span class="pl-c1">4</span>, <span class="pl-c1">8</span>], [<span class="pl-c1">10</span>, <span class="pl-c1">20</span>], [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">22</span>], [<span class="pl-c1">7</span>, <span class="pl-c1">14</span>]])

<span class="pl-v">X_test</span> <span class="pl-c1">=</span> [[<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">4</span>], [<span class="pl-c1">6</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>], [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">6</span>], [<span class="pl-c1">4</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>], [<span class="pl-c1">33</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>]]
<span class="pl-en">print</span>(<span class="pl-s1">np</span>.<span class="pl-en">round</span>(<span class="pl-s1">imp</span>.<span class="pl-en">transform</span>(<span class="pl-v">X_test</span>)))</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Return:
[[ 2.  4.]
 [ 6. 12.]
 [ 3.  6.]
 [ 4. 12.]
 [33. 12.]]"><pre class="notranslate"><code class="notranslate">Return:
[[ 2.  4.]
 [ 6. 12.]
 [ 3.  6.]
 [ 4. 12.]
 [33. 12.]]
</code></pre></div>
<p dir="auto">Example adjusted - Second feature has np.nan values --&gt; iterative imputation with estimator</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import numpy as np
from sklearn.impute import IterativeImputer
imp = IterativeImputer(max_iter=10, verbose=0)
imp.fit([[1, 2], [3, 6], [4, 8], [10, 20], [np.nan, 22], [7, np.nan]])

X_test = [[np.nan, 4], [6, np.nan], [np.nan, 6], [4, np.nan], [33, np.nan]]
print(np.round(imp.transform(X_test)))"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-k">from</span> <span class="pl-s1">sklearn</span>.<span class="pl-s1">impute</span> <span class="pl-k">import</span> <span class="pl-v">IterativeImputer</span>
<span class="pl-s1">imp</span> <span class="pl-c1">=</span> <span class="pl-v">IterativeImputer</span>(<span class="pl-s1">max_iter</span><span class="pl-c1">=</span><span class="pl-c1">10</span>, <span class="pl-s1">verbose</span><span class="pl-c1">=</span><span class="pl-c1">0</span>)
<span class="pl-s1">imp</span>.<span class="pl-en">fit</span>([[<span class="pl-c1">1</span>, <span class="pl-c1">2</span>], [<span class="pl-c1">3</span>, <span class="pl-c1">6</span>], [<span class="pl-c1">4</span>, <span class="pl-c1">8</span>], [<span class="pl-c1">10</span>, <span class="pl-c1">20</span>], [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">22</span>], [<span class="pl-c1">7</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>]])

<span class="pl-v">X_test</span> <span class="pl-c1">=</span> [[<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">4</span>], [<span class="pl-c1">6</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>], [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>, <span class="pl-c1">6</span>], [<span class="pl-c1">4</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>], [<span class="pl-c1">33</span>, <span class="pl-s1">np</span>.<span class="pl-s1">nan</span>]]
<span class="pl-en">print</span>(<span class="pl-s1">np</span>.<span class="pl-en">round</span>(<span class="pl-s1">imp</span>.<span class="pl-en">transform</span>(<span class="pl-v">X_test</span>)))</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Return:
[[ 2.  4.]
 [ 6. 12.]
 [ 3.  6.]
 [ 4. 8.]
 [33. 66.]]"><pre class="notranslate"><code class="notranslate">Return:
[[ 2.  4.]
 [ 6. 12.]
 [ 3.  6.]
 [ 4. 8.]
 [33. 66.]]
</code></pre></div>
<p dir="auto">Maybe <a href="https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/impute.py">sklearn/impute.py</a> line 679 to 683 should be optional with a parameter like force-iterimpute.</p>