12V DC power supply - how to make sure it's properly constructed
A deep cycle battery is a good choice for this it will not "push" extra current to your loads. That does not happen unless the voltage is too high.

Each cigar lighter plug should already have its own fuse this will be rated to protect the wire to the device and the device.  

For fuses in your box you are protecting the wiring in your box. Wires have an "ampacity" this can be found on the web. This is the amount of current it is safe to push through a wire of that size. In fault situations a lead acid battery can push very large currents and this is what you need to protect against. A single fuse at the battery rated at the ampacity of all your wire will adequately protect your box. If you need more current than this fuse will supply instead use a similarly  sized fuse in each of the feeders to the cigar lighters again at the battery.

Why do we fuse at the power source, the battery. Taking a pessimistic view of the world a fault can develop anywhere along the wire. If this point is before the fuse it can result in a fire. From a safety point of view therefore it is good to get into the habit of fusing at the source.