<p dir="auto">I have run into a possible bug where passing feature vectors in a different order to a tree-based classifier when training affects their performance.</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://cloud.githubusercontent.com/assets/1719223/10463635/b0ea8802-71b3-11e5-86a1-38b0b4faea4d.png"><img src="https://cloud.githubusercontent.com/assets/1719223/10463635/b0ea8802-71b3-11e5-86a1-38b0b4faea4d.png" alt="crnygcbuaaa06h7" style="max-width: 100%;"></a></p>
<p dir="auto">I have the full minimal working example here: <a href="http://nbviewer.ipython.org/urls/gist.githubusercontent.com/rhiever/7e6de0e51685b32bd4e6/raw/dce1843f6e92f141469d44c5847fc99ee36a0496/scikit-learn-feature-order-bug.ipynb" rel="nofollow">link</a></p>
<p dir="auto">This is a serious issue for reproducibility if passing the same feature vectors (even in a different order) results in different classification performance.</p>
<p dir="auto">I would like us to refer to the Glossary in API reference for parameter descriptions that come up frequently, or which have associated caveats that are too long for parameter descriptions, most notably <code class="notranslate">n_jobs</code> and <code class="notranslate">random_state</code>.</p>
<p dir="auto">So instead of something like:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    random_state : int, RandomState instance or None, optional, default: None
        If int, random_state is the seed used by the random number generator;
        If RandomState instance, random_state is the random number generator;
        If None, the random number generator is the RandomState instance used
        by `np.random`."><pre class="notranslate"><code class="notranslate">    random_state : int, RandomState instance or None, optional, default: None
        If int, random_state is the seed used by the random number generator;
        If RandomState instance, random_state is the random number generator;
        If None, the random number generator is the RandomState instance used
        by `np.random`.
</code></pre></div>
<p dir="auto">in both KMeans and MiniBatchKMeans, we might have:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="KMeans:
    random_state : int, RandomState instance or None (default)
        Determines random number generation for centroid initialization.
        See :term:`random_state`.


MiniBatchKMeans:
    random_state : int, RandomState instance or None (default)
        Determines random number generation for centroid initialization and
        random reassignment.  See :term:`random_state`."><pre class="notranslate"><code class="notranslate">KMeans:
    random_state : int, RandomState instance or None (default)
        Determines random number generation for centroid initialization.
        See :term:`random_state`.


MiniBatchKMeans:
    random_state : int, RandomState instance or None (default)
        Determines random number generation for centroid initialization and
        random reassignment.  See :term:`random_state`.
</code></pre></div>
<p dir="auto">One question is how much verbosity we should have in describing how the user may parametrise random_state. We could have just <code class="notranslate">See :term:`random_state`.</code>, or we could have <code class="notranslate">An int seeds the random number generator deterministically, while None uses the current np.random state. See :term:`random_state`.</code></p>
<p dir="auto">Just as I see us trying to describe what is random about the algorithm when describing random_state, I would like to see <code class="notranslate">n_jobs</code> stating whether parallelism is only in fit, or in fit and predict, and what backend is used by default.</p>
<p dir="auto">What do others think?</p>