Dividing a dataset to parallelize machine learning training on the cloud
There are many ways to parallelism machine learning. It is often better to distribute the model parameters, not the data.
Training models only a subset of data will result in worse parameter estimates than training a model on random samples of the data.
Additionally, moving data around is more expensive than moving parameters.