Bicolor/RGB 2x2 LED Matrix?
As you are sharing the control signals, you need to investigate multiplexing, whereby you quickly cycle thru each LED and turn on the one you want lit, and leave off those you do not want lit.

Or, just use a shift register, they have 8 outputs and then you can control each LED individually.  Are these common anode or common cathode dual LEDs?  MIC2981 would be good for common cathode, and TPIC6C595 would be good for common anode.

Don't forget current limit resistors for each LED, 270 ohm to 1K would be good generally.