In random forest, can features with high Gini importance show insignificant p-values in t-tests?
Intuitively, wouldn't features with high Gini importances have very different values between classes? I'm wondering how to reconcile these two facts.


If I understood you correctly, these two facts perfectly reconcile in case your classes are not separable in one-dimensional feature space. 

The following exaggerated toy example should illustrate such situation (and power of RF).

#uncomment the following line to remove all variables
# rm(list=ls())

library(ggplot2)
library(caret)
library(MASS)
library(GGally)
library(dplyr)

SEED &lt;- 123456
SIMS &lt;- 1000

set.seed(SEED)

x.signal &lt;- rbind(
    mvrnorm(n = SIMS/2, c(0,0), matrix(c(1,0.9,0.9,1),2,2)),
    mvrnorm(n = SIMS/2, c(0,0), matrix(c(1,-0.9,-0.9,1),2,2)))

x.noise &lt;- mvrnorm(n = SIMS, c(0,0), matrix(c(1,0.9,0.9,1),2,2))
y &lt;- rep(c(TRUE, FALSE), each=SIMS/2)
df &lt;- data.frame(
    x1.signal=x.signal[,1],
    x2.signal=x.signal[,2], 
    x1.noise=x.noise[,1], 
    x2.noise=x.noise[,2], 
    y=factor(y))

yName &lt;- "y"
xName &lt;- setdiff(colnames(df), yName)

ggpairs(df, aes(colour=y))




ctrl &lt;- trainControl("cv", number = 5, verboseIter = TRUE)
rffit &lt;- train(df[,xName], df[,yName], 
    method = "rf", trControl = ctrl, tuneLength = 5)

varImpPlot(rffit$finalModel)




## t = 0.061767, df = 997.98, p-value = 0.9508
t.test(filter(df, y=="TRUE")$x1.signal, 
    filter(df, y=="FALSE")$x1.signal,
    var.equal = FALSE, paired = FALSE)

## t = 0.22695, df = 996.04, p-value = 0.8205
t.test(filter(df, y=="TRUE")$x2.signal,
    filter(df, y=="FALSE")$x2.signal, 
    var.equal = FALSE, paired = FALSE)


dplyr::group_by(df, y) %&gt;% 
    dplyr::summarise(mx1=mean(x1.signal), 
    mx2=mean(x2.signal))