<h3 dir="auto">Describe the workflow you want to enable</h3>
<p dir="auto">I think it would be awesome if the RF regressor returns the standard deviation (not only the mean) of the output of the different trees.</p>
<h3 dir="auto">Describe your proposed solution</h3>
<p dir="auto">This is not a definite option, but contains the core idea:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from sklearn.ensemble import RandomForestRegressor
import numpy as np

class RandomForestWithUncertainty:

def __init__(self,**args):

    self.model = RandomForestRegressor(**args)

def predict(self, X, return_std = False):

    pred = np.array([tree.predict(X) for tree in self.model]).T
    pred_mean = np.mean(pred, axis=1)

    if return_std:

      pred_var = (pred - pred_mean.reshape(-1,1))**2
      pred_std = np.sqrt(np.mean(pred_var, axis=1))
      return pred_mean, pred_std

    else:
      return pred_mean"><pre class="notranslate"><code class="notranslate">from sklearn.ensemble import RandomForestRegressor
import numpy as np

class RandomForestWithUncertainty:

def __init__(self,**args):

    self.model = RandomForestRegressor(**args)

def predict(self, X, return_std = False):

    pred = np.array([tree.predict(X) for tree in self.model]).T
    pred_mean = np.mean(pred, axis=1)

    if return_std:

      pred_var = (pred - pred_mean.reshape(-1,1))**2
      pred_std = np.sqrt(np.mean(pred_var, axis=1))
      return pred_mean, pred_std

    else:
      return pred_mean
</code></pre></div>
<h3 dir="auto">Describe alternatives you've considered, if relevant</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Additional context</h3>
<p dir="auto">This is useful if we want to use the RF in algorithms that require uncertainties estimatios, such as Bayesian Optimization.</p>
<p dir="auto">It would be nice if we could do:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="rf = RandomForestRegressor()
rf.fit(X, y)
y_mean, y_var = rf.predict(X, ret_variance=True)"><pre class="notranslate"><span class="pl-s1">rf</span> <span class="pl-c1">=</span> <span class="pl-v">RandomForestRegressor</span>()
<span class="pl-s1">rf</span>.<span class="pl-en">fit</span>(<span class="pl-v">X</span>, <span class="pl-s1">y</span>)
<span class="pl-s1">y_mean</span>, <span class="pl-s1">y_var</span> <span class="pl-c1">=</span> <span class="pl-s1">rf</span>.<span class="pl-en">predict</span>(<span class="pl-v">X</span>, <span class="pl-s1">ret_variance</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)</pre></div>
<p dir="auto">Same for BaggingRegressor.</p>