Gnd Loop for Bypass Capacitors
if it's good to have the Gnd pin of a Bypass Caps tied to IC's Gnd? 


Absolutely, if the capacitor ground isn't connected to the IC ground, it can't do its job of providing a low-impedance path between those two nets.

If you want to think of it in terms of ground loops, think about it this way:

If the capacitor wasn't there, then when the IC draws current it would have to draw it from the power supply itself, or from a bulk capacitor. In either case, the part supplying the current would be much further away. 

That means the current would flow in a loop from the power supply to the IC and back to the power supply. That would be a much larger loop than from the very close-by bypass capacitor to the IC and back.

So what the bypass capacitor is doing is shorting out a large loop by making a small one. Which reduces the emissions from the loop and reduces the inductance of the loop, causing less voltage ripple as the current demand changes.