Review this solar-powered weather station schematic?
A few things:

It's not clear if your PGND/GND should be tied together
If the P1.1 has isolation then PGND and GND should be separate, if not then they need to be tied together. I can't tell because there is no information on the 3A Buck module

Q1 is a low side switch and is facing the wrong direction (the diode will always conduct even if the switch is off) Current will flow from the PGND to the negative terminal of the battery. The fuse is sufficient to break the connection in the event of a fault.
In addition current can flow in both directions from the ground, during charging it will flow from the negative terminal to ground, if you do really want a switch there, it needs to be bidirectional.

If you are powering the RP from the header, it's probably best to use more than one header pin as ground. One header pin has 10's of mΩ of resistance and creates common mode problems if large amounts of current are going through one pin. Use multiple grounding pins (39 and 14 and 9 and 30)