Dusty water, different taste if water has been sitting?
There is an easy experiment to see if your water is picking up dust. Simply cover a glass of water with a saucer and see if it tastes the same as a glass left uncovered. 

You don't say where your water is coming from. Is this from a public water supply or from a well? 

Either way it is possible that either the water is either absorbing gasses or releasing them. 

Outgassing

In years gone by in southern Michigan there was hydrogen sulfide in the water which gave the water a rotten egg smell and taste. The hydrogen sulfide would outgas over time. 

Water can be naturally carbonated so that carbon dioxide will outgas. 

Absorption

It is also possible to absorb carbon dioxide from the atmosphere.