Accepted electrical raceway material to residential detached garage
No problem. Schedule 40 PVC conduit is suitable for direct burial, so there's no problem with that. As long as you're using schedule 40 PVC conduit, and not schedule 40 PVC DWV pipe.

It may or may not be required by code, but it's good practice to install expansion fittings wherever the conduit enters/exits the ground. This allows the ground and conduit to move, while preventing damage to the conduit during said movement.