Pavers mortared to cement - how to grout joints
You can use cement based grout.
Cement based  has been used long prior to polymer grouts. I haven’t used polymer or epoxy grouts outside but they work great in the bathroom.
I have used addmix in my grout but I also use this in my mortar.