What are the available exploration strategies for continuous action space scenarios in RL?
I have not personally worked enough with continuous action spaces to be capable of confidently giving advise based on my own experience, but I can point you to likely relevant research (more recent than the research you already pointed to yourself):



The most common / "popular" area of research in recent years that involves RL and continuous action spaces uses robot / physics simulators such as MuJoCo. Some examples:


Asynchronous Methods for Deep Reinforcement Learning mentions using an entropy cost term in the loss function used for training to encourage exploration (see Supplementary Material after References).
Parameter Space Noise for Exploration describes the idea of adding noise directly to the learned parameters on the Neural Network. That way it basically always predicts a different action to be "optimal" (due to the noise), and therefore a "greedy" policy based on the noisy parameters will in fact have exploration and not be fully greedy.
DeepMind Control Suite is a somewhat recent paper that proposes a suite of benchmarks for continuous control problems. It is filled with references to relevant papers describing all kinds of algorithms, and hopefully every single one of those papers would also describe how they perform exploration.




Recently, at the ICML 2018 conference, there was a complete workshop dedicated to Exploration in Reinforcement Learning. Here is the list of accepted papers at this workshop. Note that it's about Exploration in RL, not only about continuous action spaces, so there might be papers in there that are only applicable to discrete action spaces. Nevertheless, I would be highly surprised if there's nothing in there that's relevant.