How does a water molecule bond to an anhydrous salt to form a hydrate?
"Loose bond" is really a loose term which engenders loose understanding. The bonds that hold water molecules in the structure are usually dative bonds, which is a variety of polar covalent bonds. They are formed between water's O and the metal ion, using spare electron pairs of the former. There is nothing unusual about having covalent bonds in an ionic crystal. Think, for example, of $\ce{S-O}$ bonds in the sulphate anions within $\ce{MgSO4}$ or any other ionic sulphate.

Alternatively, in some relatively rare cases water molecules may be held by hydrogen bonds to the anions, like you suggested. This is the case with that fifth water molecule in $\ce{CuSO4\cdot5H2O}$, as explained by @ringo.