What are the limitations of the hill climbing algorithm and how to overcome them?
As @nbro has already said that Hill Climbing is a family of local search algorithms. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario:


Local Maxima: Hill-climbing algorithm reaching on the vicinity a local maximum value, gets drawn towards the peak and gets stuck there, having no other place to go.
Ridges: These are sequences of local maxima, making it difficult for the algorithm to navigate.
Plateaux: This is a flat state-space region. As there is no uphill to go, algorithm often gets lost in the plateau.



To resolve these issues many variants of hill climb algorithms have been developed. These are most commonly used:


Stochastic Hill Climbing selects at random from the uphill moves. The probability of selection varies with the steepness of the uphill move.
First-Choice Climbing implements the above one by generating successors randomly until a better one is found.
Random-restart hill climbing searches from randomly generated initial moves until the goal state is reached.


The success of hill climb algorithms depends on the architecture of the state-space landscape. Whenever there are few maxima and plateaux the variants of hill climb searching algorithms work very fine. But in real-world problems have a landscape that looks more like a widely scattered family of balding porcupines on a flat floor, with miniature porcupines living on the tip of each porcupine needle (as described in the 4th Chapter of the book Artificial Intelligence: A Modern Approach). NP-Hard problems typically have an exponential number of local maxima to get stuck on.
Given algorithms have been developed to overcome these kinds of issues:


Stimulated Annealing
Local Beam Search
Genetic Algorithms


Reference Book - Artificial Intelligence: A Modern Approach