Reinforcement learning, pendulum python
You could use the same reward function that Openai's Inverted Pendulum is using:

$costs=-(\Delta_{2\pi}\theta)^2 - 0.1(\dot{\theta})^2 - 0.001u^2$

where $(\Delta_{2\pi}\theta)$ is the difference between current and desired angular position performed using modulo $2\pi$. The variable $u$ denotes the torque (the action of your RL agent). The optimal is to be as close to zero costs as it gets.

The idea here is that you have a control problem in which you can come up with a quadratic 'energy' or cost function that tells you the cost of performing an action at EVERY single time step. In this paper (p.33 section 5.2) you can find a detailed description.

I have tested RL algorithms in this objective function and I did not encounter any problems for convergence in both MATLAB and Python. If you still have problems let us know what kind of RL approach you implemented and how you encoded the location of the pendulum.

Hope it helps!