Does magnesium carbonate exist? what makes magnesium bicarbonate different from calcium bicarbonate
Both of these reactions are thermal decomposition reactions, but the difference basically occurs in the polarizing power of the cations $\ce{Ca^2+}$ and $\ce{Mg^2+}$ .

The thermal decomposition of $\ce{Ca(HCO3)2}$ is just the reverse reaction of the lime water test for the detection of $\ce{CO2}$ , where the carbonate combines with $\ce{CO2}$ and $\ce{H2O}$ to give the bicarbonate, so it stands to reason that the bicarbonate will break down to give the reactants we started with.

We can therefore, imagine that a similar decomposition will occur with $\ce{Mg(HCO3)2}$ to yield $\ce{CO2}$, $\ce{H2O}$ and $\ce{MgCO3}$. And in fact, a similar reaction does occur on drying the bicarbonate solution.

However, a small amount of $\ce{Mg(OH)2}$ is also obtained as a side product.
The reason is that, $\ce{CO3^2-}$ is relatively quite a large-sized anion.
Also, if you recall the diagonal relationship between $\ce{Li}$ and $\ce{Mg}$, then sizes of both $\ce{Li+}$ and $\ce{Mg^2+}$ will end up being similarly quite small relative to other members of their respective groups. Thereby,Fajan's rule will become important to consider here, and the carbonates of both these cations will experience similar internal polarization, and tendency to decompose relatively.

Hence, since we know that $\ce{Li2CO3}$ is the only Group 1 carbonate to decompose at laboratory temperatures to give $\ce{Li2O}$ and $\ce{CO2}$ ,(see this) we can expect $\ce{MgCO3}$ to also  break down into $\ce{MgO}$ and $\ce{CO2}$ on prolonged heating.

$\ce{CO2}$ being a gas will escape, and since Group 2 bicarbonates exist only in aqueous form, so the magnesium oxide produced will hydrolyse within the solution to produce the hydroxide, that is, $\ce{Mg(OH)2}$.

Note: Decomposition of $\ce{Li2CO3}$ is only "easy" relative to other members of Group 1. It still has a pretty high decomposition temperature of 1300 °C approximately

This value of 1300 °C might still not be accurate, see the green Note under 'Heating the carbonates' for more insight.