Hooking up an 8x8 LED board
The problem you are experiencing sounds like your lack of knowledge on how to wire up LEDs.

Every individual LED needs its own separate resistor.  Running two LEDs in parallel with different forward voltages (such as a red and a blue) will mean that only one LED operates - the one with the lowest forward voltage (the red one in this case).

If you want to use a potentiometer instead of resistors then you would require one separate potentiometer for each and every colour of each and every LED you want to light up.