How to short a push button switch to be always on
Try using a normally-closed pushbutton switch. When you apply power the unit will power up according to what you've indicated in the comments of your orignal question. The other actions invoked by the pushbutton might be edge-triggered and still work as intended.

Alternatively, connect a SPST toggle switch in parallel with the original N.O. push-button. Before you apply power to the device make sure the toggle switch is closed. Once powered up, open the toggle and use the PB switch as usual. Throw the toggle switch closed again to turn off the unit and leave it closed as you remove the power source. The toggle switch will then be ready for the next power application.