How to handle correlated variables in a dataset? I am wondering how to handle correlated fields in a dataset.

Some people suggest to drop this correlated fields because they don't provide any inside, other claim that a lot of information is getting lost. I am not sure if I should keep this correlated fields or not. 




How to handle correlated data. Drop it, keep it, or does it depend on additional parameters? If yes, which? Are there rules I can follow?
What if the fields are sparse? (because the dataset is incomplete)
Does 1) depend on what kind of model is used? Regression, classification, NN?




To give a concrete example, the correlated values from the Zillow competition on Kaggle.
If your model has dependent variables, then it's hard to interpret the coefficients down the road.

For example, say that your model has 3 variables: profits, costs and revenue.
You trained your linear regression model to predict revenue (silly, I know).

You could get the following coefficients: $c_{cost}=-1,c_{profits}=1,c_{revenue}=0$

Someone looking at these results, could conclude that revenue is not predictive of revenue.

This is kind of a trivial example because the linear dependency is very clear (revenue=profits-costs).

But consider it a motivation for not adding dependent variables to your model.

Correlation is a measurement of pairwise linear dependency.

It's more of a question of what type are your variables.

If your variables are continuous, Dimensionality reduction algorithms like PCA should account for the correlation.