Can Adagrad be used to optimize non-differentiable functions?
The book has a misunderstanding (but it's understandable where it came from).

If you can compute subgradients, you can use gradient descent.  You don't have to use AdaGrad -- you can use any gradient method you like.  Basically, you just use a subgradient in place of the gradient in the update step.  See, e.g., https://en.wikipedia.org/wiki/Subgradient_method.

AdaGrad is an adjustment to gradient descent that adjusts the update step.  I believe AdaGrad is orthogonal to whether you use subgradients or gradients in the update step.

I can see where the misunderstanding comes from.  The original paper on AdaGrad talks about subgradient methods.  However, if you read the introduction carefully, you will recognize what's going on there.  Subgradient methods are a broader category, that includes both ordinary gradient descent and subgradient descent.  So, the paper is just trying to be as general as possible.  Their method applies both to ordinary gradient descent and also to subgradient descent.