Which has higher lattice energy: MgO or LiF?
$\ce {MgO}$ ($\approx 3800$ $\pu{kJ mol^{-1}}$) has higher lattice energy than $\ce {LiF}$ ($\approx 1045$ $\pu{kJ mol^{-1}}$) mainly because of the greater charge on  $\ce{Mg^2^{+}}$ ion and $\ce {O^{2-}}$ as lattice energy is directly proportional to the charges of the combing atoms. 

Your reasoning isn't incorrect but remember that $\ce{Li}$ and $\ce{Mg}$ show diagonal relationship so the size of their ions has only slight difference in magnitude. Similarly, oxygen and fluorine are placed next to each other in the periodic table so their anions don't have a large difference in size. 

Thus, the "charge factor" dominates.