Importance of dielectric in a capacitor
The purpose of the dielectric is to be able to store as much charge as possible in the capacitor, with the smallest size possible. The capacitance for a parallel plate capacitor is given by:

$$
C= [k] \epsilon _{0} \frac {A}{d}
$$
where \$A\$ is area and \$d\$ is distance; \$ \epsilon _{0} \$ is vacuum permittivity, and k is the relative dielectric constant of the dielectric material (if any).

So you want C big, and a small package; so you have to reduce d - but this means higher E-field because

$$
V= \int_{d} {E \mathrm{\ ds}}
$$

so you can't push it too much, because it becomes easier to break. So you can either try to increase A (but then the package becomes bigger) or k, which is done with dielectrics.

About the energy, this is given by

$$
E = \frac {1}{2} CV^{2}
\\
Q= CV
\\
E = \frac {1}{2} Q^{2} \frac{1}{C}
$$

which means that if you can increase the capacity, you can store the same charge with less voltage, so the overall energy is decreased; but this is more an advantage than a disadvantage, because as pointed before, the limit for a given distance between the plates is the E-field, so with the dielectric you can store more charge with the same energy.

Note that in modern MOS technology, to reduce the thicknes of the Gate insulation layer it's common the use of high-k dielectrics, which allow a bigger gate capacitance with reduced leakage.