Four state switching
You could use a CD4022 Octal Counter, with fully decoded outputs 0 to 7, and tie output 4 to RESET so it wraps around after 4 counts instead of 8.  It is available in a 14-pin DIP from Digi-Key for 52 cents.  You will want to provid some debounce circuitry on the input so it advances only once per push-button press.



Since I am using the 4000B CMOS family, the circuit will work with any VCC voltage from about 5V to 18V (5V minimum, not 3V to allow a little headroom for the LEDs).  For this reason I haven't listed any values for R4 through R6, as they will depend on VCC and the LEDs chosen. 

If you want to use a SPST button instead of SPDT, you will need a different debounce circuit.  There are hundreds of references on the web for this.  A good one is here, which starts off describing the SR latch used here, as well as a reliable circuit for use with a SPST switch.