What are some resources on continuous state and action spaces MDPs for reinforcement learning?
There is a small survey of continuous states, actions and time in reinforcement learning in my thesis proposal.

Regarding books, Reinforcement Learning: State-of-the-Art seems to be pretty up-to-date from the excerpts I've read.