Smoothing Proportions :: Massive User Database
A simple way would be to consider Laplace Smoothing (http://en.wikipedia.org/wiki/Additive_smoothing ) or something like it. 

Basically, instead of calculating your response rate as (Clicks)/(Impressions) you calculate (Clicks + X)/(Impressions + Y), with X and Y chosen, for example, so that X/Y is the global average of clicks/impressions. 

When Clicks and Impressions are both high, this smoothed response rate is basically equal to the true response rate (signal dominates the prior). When Clicks and Impressions are both low, the this smoothed response rate will be close to the global average response rate - a good guess when you have little data and don't want to put much weight on it!

The absolute scale of X and Y will determine how many data points you consider "enough data". It's been argued that the right thing to do is set X to 1, and Y appropriately given that.