Does a larger action space take longer to train an RL agent?
I think there are two things in your question:


The number of parameters of your network. So if you have more actions to predict the action layer of your network will have more parameters and it will take a little bit longer theoretically to update it (the time difference thought might be negligible depending your computational resources).
The total training time. Yes, having more choices it will take longer for the algorithm to generate some good estimations of expected reward per action in order to lead to a satisfactory performance. If for Critic you are using a state value $V(s)$ instead of $Q(s,a)$ still your algorithm will take longer to explore the action space and map your input state to a probability distribution over your action space.