When I connect the Vcc and GND of my circuit to my Arduino, it is powered off?
Note: if you are changing something have the power off.  

The first thing to try is unplug the IC. If it powers up after the IC is removed it is Possibly the IC is bad; However measure the Voltages on the proto board where the IC is inserted, be sure Pin 16 is +5V when referenced to pin 8.  Check the data sheet for the correct pinout, I do not know what IC you are using. If the voltage is correct it is pointing more to a bad IC. 

The final check for the IC is to insert the IC and remove the ground lead to the LEDs.  If the system comes up then the problem is not enough power to drive the LEDs as is. If it does not come up replace the IC.  Even a momentary reverse voltage or a ESD (Electro Static Discharge) incident will damage the IC, we know is is going to fail, we do not know when.

Be sure your resistors are at least 300 Ohm or higher, with that resistance the LEDs when all on will draw a total of 80mA. If you want to do the calculations try this link: "http://www.onlineconversion.com/ohms_law.htm". 

If you want to check the LEDs, remove the IC and connect +5V one at a time to the wire that goes to the resistor from the IC.
Good Luck,
Gil