Deep Learning: Does starting the training on a smaller subset of the data make sense?
It makes the most sense to train a deep learning model on the entire dataset. If you train on a subset of training, you are more likely to end up a local minimum.
One option is to adjust the size of the batch for stochastic gradient descent (SGD). The batch size is the number of data points given to the model before performing a learning update.