Structure of phosphate ion
Disclaimer: hybridization is a mathematical concept and fails at higher levels.  As per the linked answer, it is clear that the participation of $d$-orbitals in phosphate ion is minimum. But this is just a high-school level, layman's explanation for your question.




  So, could anyone please tell me where my understanding is flawed?


Your reasoning is flawed in assuming that carbon atom has four unpaired electrons in its orbitals after hybridisaton, while phosphorus does not. Actually, both of them do. 

In fact, carbon atom in its ground state does have a lone pair of electrons in its $\mathrm{2s}$ orbital. It is only when it gets into its excited state, that that lone pair is broken up, yielding four unpaired electrons. The excited carbon atom now undergoes hybridization as depicted in the figure below, yielding four equivalent covalent bonds as in carbon tetrachloride.


(source)

As you noted, phosphorus also has a lone pair of electrons in its $\mathrm{3s}$ orbital. But, that is its ground state electronic configuration. Once it gets excited, it will have five unpaired electrons, the fifth electron going into its $\mathrm{3d}$ orbital. Now, phosphorus will undergo $\mathrm{sp^3}$ hybridization, to form $4$ sigma bonds. The fifth electron helps form the $d\pi-p\pi$ bond with the neutral oxygen atom.

I hope this helps!