(Bad) counting digital pulses with Arduino using interrupts and a 4 pin switch
Like Jon says the LEDs don't have current limiting resistors; the red one is connected directly between +5 V and ground, and the blue one between the I/O pin and ground. I'm a bit puzzled by the tact switch. This type of switches normally has the two left pins connected, and the two right pin. 



When you push the switch left and right are connected. (This way of internal connections simplifies wiring up a matric of switches.) Here you can read how to wire a switch to a microcontroller's input. 

If you do that you're shorting the red LED, so that will go out. But, and worse, you'll also short-circuit the power supply: the switch will connect the +5 V on the left side with ground on the right.

Please fix those issues first and report back.