pushbutton power source to load
First, I don't think you need source buttons, but eliminating them actually increases the number of buttons.  What you have is a SSAD (Source Select At Destination) routing switcher, but for power instead of video or audio.

For a 4x4 array, each output has 4 buttons, one for each input.  Pressing any button latches 4 bits, and since only one button is pressed at a time, the latch for that output line holds one 1 and three 0s.  If the latch outputs are negative-true, then PNP power transistors will make efficient high-side switches.

With only 8 switches, the latch input becomes a bit more complex.