How to implement an ON/OFF switch in a simple digital circuit?
If you want to "turn off" the LEDs and ICs, just put a SPST toggle switch in the incoming power line. One position is OFF, the other position is ON. Based on the information you have provided, that is exactly the functionality you want.