Run new electrical wiring in backyard from existing wall outlet
I would suggest the use of a weatherproof (WR) box extender. One of which could be installed over the existing box giving you sides on to which to connect the PVC conduit.

The following is an example. No endorsement of specific products is implied.