We know that most of the machine learning algorithms assume the samples are i.i.d. However if the samples are from time series (like ARMA in the prediction of time series), then they are highly correlated. Then how to deal with such correlated samples? Note that, here I mean the correlation between samples not the correlation between features.

1. Example 1: we have a time series $\{x\_1,\cdots,x\_T\}.$ Then we use three times to predict the forth time to construct the training samples:

$$(X\_1 = (x\_1,x\_2,x\_3), y\_1 = x\_4), (X\_2 = (x\_2,x\_3,x\_4), y\_2 = x\_5), \cdots.$$
Then the training samples $(X\_i,y\_i)$ must be highly correlated, which is contradict to the assumption of i.i.d. samples for most machine learning algorithms.

2. Example 2: we have a time series $\{x\_1,\cdots,x\_T\}$ and the corresponding label series $\{y\_1,\cdots,y\_T\}.$ Then the training samples:

$$(x\_1, y\_1), (x\_2, y\_2), \cdots$$

is also highly correlated.

I have the following guess:

1. In principle, machine learning algorithm is not a proper way to deal with the time series?
2. The assumption of i.i.d. samples is actually not necessary for the machine learning algorithm?

I think it is very basic questions. Can anyone give me some references?