Policy Gradient with continuous action space
Yes, that is possible. It can be done in the following way:

We assume that the action distribution is guassian, i.e. that we need to learn the parameters $\theta$ of $\mathcal{N}(a|\mu_\theta,\sigma_\theta)$. Let's say that $\theta$ is given by the weights of a neural network, which we find by optimizing the objective
$$\max_\theta \mathbb{E}_{p_{\theta}}\left[ R(s,a)p_\theta(a|s)\right],$$
where $p_\theta(s,a) = \mathcal{N}(a|\mu_\theta, \sigma_\theta)$ and $R(s,a)$ is the cumulative discounted reward. The gradient is then per policy gradient theorem simply $\mathbb{E}_{p_{\theta}}\left[\nabla_\theta R(s,a) \log p_\theta(a|s) \right]$.

In practice we design a neural network to output one $\mu$ per action dimension and $\sigma$ can either be learned or kept fixed. If learned, interpret the output as $\log \sigma$, so it can take en value. To sample the action we use this outputs learned by our network.

See e.g. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. “Continuous control with deep reinforcement learning,” International Conference on Learning Representations, 2016.