Water and Ice - density
The most straightforward answer is that water in the form of ice I (or $\ce{I_h}$) has an open hexagonal structure (see figure) whereas liquid water does not. In ice the oxygens are precisely tetrahedrally positioned where each oxygen is hydrogen bonded by four neighbouring oxygens with an O..O distance of approx 0.28 nm. The H atoms lie very closely along the O-O axes. Two of the H bonds are short, as in liquid water, and two are long. This is expected from residual entropy calculations and also from thermal and neutron diffraction experiments.

As this ice structure has a rather open lattice this is the reason its density is less at its freezing point than that of liquid water, rather than being due to the high density of water. One could argue that the reason that water density is a maximum at 4 $\ce{^0C}$ (277 K) is that the structure of  liquid water is changing from that of ice $\ce{I_h}$ to the short range structure of the liquid.

Liquid water is, of course, also highly hydrogen bonded but the extra energy it contains means that its structure is far less regular and so becomes slightly denser than ice. As the water molecules can move about more in the liquid than in ice, the hydrogen bonds will bend a little or break as molecules rotate. This will lead to a shorter O - O distance and hence slightly higher density in liquid than in the solid.

see also answers to this post Why really is ice less dense than water?



This figure shows the hexagonal ice structure; The O atoms are shown as dots and the lines are the (almost linear) O-H-O bonds.(Image from Murrell &amp; Jenkins 'Properties of liquids &amp; solutions')

(Ice also exhibits eight different phases depending on temperature  and pressure of which $\ce{I_h}$ is the one found at normal atmospheric pressure. In these other phases, angles are distorted from being tetrahedral)