Equilibrium concentration, where does the 5.00 for iron thiocyanate complex come from?
You are just making a small sign error:
\begin{align}
ax^2 +bx +c &amp;= 0 &amp; x_{(1;2)} &amp;= \frac{\color{\red}{\pmb{-}}b \pm\sqrt{b^2 - 4ac}}{2a}
\end{align}

Taking $K=\frac13$:
\begin{align}
\frac13x^2 - \frac{21}{3}x +\frac{80}{3} &amp;= 0 &amp;
 x_{(1;2)} &amp;= \frac{\color{\red}{\pmb{-}}(-\frac{21}{3})\pm\sqrt{\left(\frac{21}{3}\right)^2 - 4\cdot\frac13\frac{80}{3}}}{\frac{2}{3}}\\
&amp;&amp; x_{(1;2)} &amp;= \frac{7\pm\sqrt{49-\frac{320}{9}}}{\frac{2}{3}}\\
&amp;&amp;           &amp;= \left(7\pm\frac{11}{3}\right)\cdot\frac{3}{2}\\
&amp;&amp; x_{(1;2)} &amp;= 16; 5
\end{align}

The first value obviously does not make any sense, hence $c(\ce{[Fe(SCN)]^2+})= 5~\mathrm{mol\, L^{-1}}$.