2 led on latching pushbutton NO arduino
An alternating (push-on, push-off) pushbutton switch would be the easiest. It needs to be SPDT (or better). There are plenty available. 







simulate this circuit &ndash; Schematic created using CircuitLab

If you want to connect it to an MCU or other logic, then R2 can be used to assure a valid '0' logic level, otherwise you don't need R2.