Wouldn't it make more sense to give less importance to gradient far away in past in AdaGrad?
One small correction- G here is the sum of squares of the gradient of a particular parameter which is kept track of and is monotonic in nature.
That being said, the way I see the equation functions is, it lets the past dictate the future or simply put, the past gradients of the parameter decides by how much it's corresponding learning rate drops.
In SGD no matter what they gradients were in the distant past their importance drops exponentially(assuming $\eta &lt; 1$). But AdaGrad, seems to be meant for data with sparse features and here, though the update rule gives equal importance to the past gradients, the update is sporadic instead of being regular.
I understood your question and for me at least the concept of sparse features makes the update rule of AdaGrad sense.
Quoting from the paper for quick reference-
In many applications of online and stochastic learning, the input instances are of very 
high dimension, yet within any particular instance only a few features are non-zero. It is 
often the case, however, that infrequently occurring features are highly informative and 
discriminative.
.
.
.
In contrast, our algorithms dynamically incorporate knowledge of the geometry of the data
observed in earlier iterations to perform more informative gradient-based learning. 
Informally, our procedures give frequently occurring features very low learning rates and 
infrequent features high learning rates, where the intuition is that each time an 
infrequent feature is seen, the learner should “take notice.”