What does it mean when a model "statistically outperforms" another?
Most model-fitting is stochastic, so you get different parameters every time you train, and you usually can't say that one algorithm will always give you a better-performing model.
However, since you can retrain many times to get a distribution of models, you can use a statistical test like the T-Test to say &quot;algorithm A usually produces a better model than algorithm B,&quot; which is what they mean by &quot;statistically outperforms.&quot;
p-value is usually set by consensus in the field. The higher the p-value, the less confidence you have that there's a statistical difference between the distribution of values being compared. 0.1 might be normal in a field where data is very expensive to collect (like risky, long-term studies of humans), but in machine learning, it's usually easy enough to retrain a model that 0.01 is common, and demonstrates very high confidence. To know more about selecting and interpreting the values, I recommend Wikipedia's page on statistical significance.