The logic for hybridisation formula
Since you understand why it works for monovalent surrounding atoms, I will assume   that we have a molecule $\ce{MA_{$m$} B_{$n$}}$ where $\ce{M}$ is the central atom, $\ce{A}$ represents the monovalent atoms which may or may not be identical, and $\ce{B}$ represents not necessarily identical atoms which are polyvalent. First up, there are some assumptions to be made:


All "surrounding atoms" are directly bonded to the central atom $\ce{M}$, and with no other atom;  
All atoms form as many covalent bonds as is indicated by valency;  


Now say the $n\ \ce{B}$ atoms are absent. Let the steric number be $x$,
Then, $x=0.5\times (V+m-C+A)$
Now add the $\ce{B}$ atoms. Now the central atom forms $n$ more sigma bonds, but if the sigma bonds were not present, these would exist as lone pairs, and hence already counted in $V$. Take for example, $\ce{PCl_3}$ and $\ce{POCl_3}$. For $\ce{PCl_3}$ you would get $0.5\times (5+3)=4$ and for $\ce{POCl_3}$ you would get $0.5\times (5+3)=4$. The reason the $\ce{O}$ made no difference is because phosphorus uses two of its electrons to bond with both of which are nothing but valence electrons of phosphorus, only present as a lone pair in one case and a sigma bond in the other. Also, this formula does not work when you have triple bonds, say for $\ce{HCN}$.
As to how it works for mono valent atoms being the only surrounding atoms, it is simple.The number of sigma bonds is the number of mono valent atoms, and thus by adding that to the number of lone pairs we get the hybridization. Say you have $ n $ lone pairs and $s$ sigma bonds.You can see that $V=2n+s$, and that $s=H$. So $0.5*(H+V)=n+s$, as required.