Different learning rates for each dimension
The answer to your question is yes, and it is already implemented due to popular approaches which are introduced in academic papers. For instance, Adam optimisaiton approach is an optimiser which assumes it is not valid to multiply each dimension, feature, with a specified value. It uses the idea of momentum and geometric average and a bunch of other methods to employ the idea that you should not go down the hill with the same speed for different dimensions. What it says is that you may be in a location which each dimension has different slope, and due to this fact you should not go downhill with the same learning rate. In that approach, although you specify the same learning rate for the optimiser, due to using momentum, it changes in practice for different dimensions. At least as far as I know, the idea of different learning rates for each dimension was introduced by Pr. Hinton with his approache, namely RMSProp.