How to train my model efficiently? I am new to ML and have been reading online about training bottlenecks when there are frequent updates to data.

Let's say I have a built a model based on a dataset of 10M records.

Now, in another 2 months, I might receive another 1M records which we would like to feed into our model as well.

Similarly this goes on for every 2 months. We would like to update/train our model with latest data as and when it's available

1) But for example, let's say the training takes 1 week for every new data update

2) Any suggestions on how we can minimize the training time (when we train every 2 months?)?

3) Should we select a representative sample from 1 Million datapoints? Is that good enough

4) I understand it's all about tradeoff but I am curious to know whether I am missing any known approaches to save training time? I am thinking representative sample can reduce the sample size and help us fasten the training process

Can you share your suggestions on this?
From what I understand - your problem is "sample selection bias" problem. Any kind of pattern to select a subset out of large data may lead to bias. This raises two question.


How to choose? Random/stratified random (If you have multiple classes) under sampling to obtain a smaller subset.
How big to choose? we can set percentage of undersampling. 


Reducing training time : We can perform preprocessing : dimensionality reduction techniques to remove correlated data by aplying techniques like- PCA. Or Sparse reconstruction methods - Transform data to sparse data and then process.