Calcium sulfate soluble in water
The overall reaction is really better given as: $$\ce{2H2SO4 + CaC2O4 -&gt; H2C2O4 + Ca^{2+} + 2HSO4^{-}}$$ since at that strong of a pH the predominate sulfuric acid species is $\ce{HSO4^{-}}$. 

Better yet the reaction could be written as  $$\ce{2H^+ + CaC2O4 -&gt;C[{0.2 N \ H2SO4, 70 °C}]\ H2C2O4 + Ca^{2+}}$$

As was already pointed out in the comments, calcium sulfate has a fair solubility 0.2 g per 100 ml of water at at 20 °C, and the $K_\text{sp} = 4\times10^{-5}$. 

The solubility varies with temperature. The solubility of calcium sulfate dihydrate increases from 0.223 g/100 ml at 0 C to about 0.265 g/100 ml at 40 C, then decreases to 0.205 g/100 ml at 100 C.