Grounding a capacitor?
Notice that the capacitor symbol shows a gap between two plates. That's literally what a capacitor is. A capacitor doesn't allow current to flow through it. It only allows current to cause a charge buildup on it. You're converting excess voltage and current into an electric field between those two plates. Then when you need a little extra voltage/current the electric field converts some of the energy back into voltage/current. At no time is any significant amount of current supposed to pass through.

In your schematic, you can think of the white plate of the capacitors as holding excess energy and then pushing that energy back up and into the In terminal of your regulator when the voltage of the fullwave rectifier droops.

The reason the other side is connected to ground is because in order to convert the voltage/current into the electric field, you have to remove positive charge from the negative side of the capacitors as you're adding positive charge to to the positive side of the cap. Without the connection to ground, there would be nowhere for that charge to go and the capacitor would be virtually useless.



EDIT to address question in comment:

Capacitors connected to the next component down the line have an entirely different purpose than decoupling capacitors you ask about in your question. Because charge can never flow through caps, a capacitor setup in that topology can have a little bit of charge pushed onto the cap before the plates saturate with charge. Once they saturate (or before), you can go ahead and pull that charge off by lowering the voltage. Then you keep switching it back and forth to prevent the capacitor from ever saturating. In this way, although no electrons actually cross the gap, you're able to put charge onto the cap on one side (while drawing it off on the other) and then performing the reverse of this. This means that you're passing through alternating current but you're prevented from passing direct current. The key here is that in order to put charge onto one side of the capacitor, you have to remove an equal amount from the other side. This removal of charge from the other side is what allows you to still get work done even though there's no physical connection through the gap.