Regression error increase after shuffing data I'm trying to do multivariate regression using a 3-dimension data set. I noticed a strange problem that my fitting error increase dramatically after I pre-shuffled the data matrix comparing using original data. Is that possible? I've checked my code and confirmed that it's the shuffled process that increased the error. Why is that? Shouldn't shuffle process only change the order of the data?
You need to shuffle the whole dataset together before separating the features (X) and target variable (y). This is the only reason I can think of for getting this error.