In random forest, can features with high Gini importance show insignificant p-values in t-tests? In have two classes in my data, and one of my features with Gini importance 0.023 (8th highest out of 95 features) shows a p-value of 0.44 when doing a t-test on the values from the two classes. I'm wondering if this makes sense. Intuitively, wouldn't features with high Gini importances have very different values between classes? I'm wondering how to reconcile these two facts.

EDIT: If it's helpful, all of the Gini importances range between 0.0011 to 0.0295, and the p-values ranged from 5.13e-109 to 1. These p-values were seen for both t-tests and rank-sum tests.
Intuitively, wouldn't features with high Gini importances have very different values between classes? I'm wondering how to reconcile these two facts.


If I understood you correctly, these two facts perfectly reconcile in case your classes are not separable in one-dimensional feature space. 

The following exaggerated toy example should illustrate such situation (and power of RF).

#uncomment the following line to remove all variables
# rm(list=ls())

library(ggplot2)
library(caret)
library(MASS)
library(GGally)
library(dplyr)

SEED &lt;- 123456
SIMS &lt;- 1000

set.seed(SEED)

x.signal &lt;- rbind(
    mvrnorm(n = SIMS/2, c(0,0), matrix(c(1,0.9,0.9,1),2,2)),
    mvrnorm(n = SIMS/2, c(0,0), matrix(c(1,-0.9,-0.9,1),2,2)))

x.noise &lt;- mvrnorm(n = SIMS, c(0,0), matrix(c(1,0.9,0.9,1),2,2))
y &lt;- rep(c(TRUE, FALSE), each=SIMS/2)
df &lt;- data.frame(
    x1.signal=x.signal[,1],
    x2.signal=x.signal[,2], 
    x1.noise=x.noise[,1], 
    x2.noise=x.noise[,2], 
    y=factor(y))

yName &lt;- "y"
xName &lt;- setdiff(colnames(df), yName)

ggpairs(df, aes(colour=y))




ctrl &lt;- trainControl("cv", number = 5, verboseIter = TRUE)
rffit &lt;- train(df[,xName], df[,yName], 
    method = "rf", trControl = ctrl, tuneLength = 5)

varImpPlot(rffit$finalModel)




## t = 0.061767, df = 997.98, p-value = 0.9508
t.test(filter(df, y=="TRUE")$x1.signal, 
    filter(df, y=="FALSE")$x1.signal,
    var.equal = FALSE, paired = FALSE)

## t = 0.22695, df = 996.04, p-value = 0.8205
t.test(filter(df, y=="TRUE")$x2.signal,
    filter(df, y=="FALSE")$x2.signal, 
    var.equal = FALSE, paired = FALSE)


dplyr::group_by(df, y) %&gt;% 
    dplyr::summarise(mx1=mean(x1.signal), 
    mx2=mean(x2.signal))