How can one estimate the difficulty change? I understand that the difficulty is growing and think that it should be possible to estimate how quickly that happens. I would like to analyze the previous difficulty progression and feel that I then should be able to predict the change in difficulty for the next year. 

Are there already calculators that predict the change in difficulty? Where can I find a chart that compares the Bitcoin price to the difficulty progression?
You cannot predict the change in difficulty because that is a function of a number of factors that are completely unknowable.

For example, the difficulty changes because a large number of mining pools suddenly appear and have very powerful ASICs and a lot of them. How would you ever be able to "predict" this happening?

Another example is the opposite case, that the technology has reached it's absolute maximum and there are no new chips yet invented that could give performance boost. Although new miners enter the market, this does not fundamentally change the rate at which the new blocks are discovered, because fundamentally the chips being used are no more powerful.

How on earth would you be able to determine that this has happened or is occurring?

At best any calculator would be guessing.