Capacitor stitching for high speed differential pairs
Given that you also have to connect ground to the numerous ground pins on your connector, it would likely be better to simply move the L4-L1 vias as close as you can to the connector, and then surround them with ground vias connected to L3 ground and the connector ground pins.
This avoids having the return path transition from the ground layer to the power layer, and then have to transition again from the power layer to the ground contacts of the connector.
If you must keep the signal vias at some distance from the connector, you could consider making a split plane on L2  (and connect vias from this split plane to L3 near the signal transition, and from L1 to the split plane near the connector) to allow the signals to remain referenced to ground. Whether this is a good idea depends on where the power needs to flow to.