Dropout in other machine learning models Dropout is a widely used technique in deep learning. Dropout was built for neural networks, but I wonder if other prediction models can use this idea as well as a regularizer. Do you know of any similar technique in linear regression, SVMs or tree-based methods?
Random forests could be thought of as using a kind of dropout-esque technique as each split node only considers a random subset of the features, effectively 'dropping out' the other ones. Also, sometimes in large tree ensembles, each tree is only given a random subset of features to begin with, akin to dropout on the input layer of a neural network.