Intermittent Power Selection with Mosfets
The first question is: what for do you need to switch on and off power supply line dynamically? Who decides to bypass the 5 V line from the previous board or to use own power supply?

There is a simple and straightforward solution:

You reserve 2 pins on each connector for 5 V line: 1 pin for power "input" another one - for power "output".

Each board with PS does not use "input" 5 V; it makes "own" 5 V that it uses as a local power and sends out for next boards through "output" pin

Each board without PS has a jumper (or soldered zero Ohms resistor) that connects "input" +5 V straight to "output" 5 V.

Sure the 24 V to 5 V DC-DC converter should be protected against overload in your design, as we do not know how many boards it powers up.