Why can't I find Phase to phase voltage like this
What you are calculating (the black line) isn't actually \$V_{AB}\$, it is rather the vector sum of \$V_{AN}\$ and \$V_{BN}\$ which is actually just \$-V_{CN}\$.

\$V_{AB}\$ is the voltage of \$V_A\$ referenced to \$V_B\$. To calculate that, you have to subtract one from the other. So the vector sum is actually:

$$V_{AB} = V_{AN} - V_{BN}$$

Doing the calculation, we get:

$$\begin{align}\\
V_{AB} = V_{AN} - V_{BN} &amp;= 230\angle0^\circ - 230\angle120^\circ\\
&amp;=398\angle30^\circ\\
&amp;=(230\sqrt{3})\angle30^\circ\\
\end{align}$$



Now lets look at it in a vector diagram:



Notice how the calculation you did (left) and the correct form (right) differ. You can see from the diagram that the line drawn for \$V_{AB}\$ is actually equal to the vector that takes us from the point \$V_B\$ to \$V_N\$ (-\$V_{BN}\$), and then from \$V_N\$ to \$V_A\$ (\$V_{AN}\$).

If you want to calculate that it is exactly \$\sqrt{3}\$, we can do a bit of trigonometry on the newly formed triangle:



From that we can see that:

$$\frac{V_{AB}}{2} = 230\times\sin{60}$$

We know that \$\sin(60) = \frac{\sqrt{3}}{2}\$, so we can say directly that: 

$$V_{AB} = 2\times230\times\frac{\sqrt{3}}{2} = 230\sqrt{3} = V_{AN}\sqrt{3}$$