Why do we use the log in gradient-based reinforcement algorithms?
We often take the logarithm because:


Maximizing $\log \Phi(x)$ is equivalent to maximizing $\Phi(x)$, so in maximum-likelihood problems, we can maximize the log of the likelihood instead of maximizing the likelihood directly and the result will be equivalent.
The logarithm converts multiplication to addition, and the derivative of a sum is "nicer" than the derivative of a product.  Products often arise when maximizing likelihoods, because the probability of something can often be written as the product of other probabilities   (e.g., when we are calculating the probability of that multiple independent events all happen).  The derivative of the product is ugly, but the derivative of the log of the product is nice and simple.
Gradient descent often involves maximizing a likelihood, so we'll need to take derivatives.  The expressions get simpler/cleaner if we can take the derivative of the log likelihood instead.