Proving that a certain graph contains a 4-cycle
The question says "IF $G$ contains a certain number of edges, then..." and, as you've demonstrated, if $G$ has only four vertices, then it can't contain that many edges, since a graph (without self-loops) can contain at most $\tfrac12|V|\,(|V|-1)$ edges. So, for the condition to be true, we must have
$$\begin{align*}
    2|V|^{3/2} &amp;\leq \tfrac12|V|\,(|V|-1) \\
    4|V|^{1/2} &amp;\leq |V|-1 &amp;\text{(dividing through by $\tfrac12|V|$)}\\
    16|V| &amp;\leq |V|^2-2|V| + 1 &amp;\text{(squaring both sides)}\\
    |V|^2-18|V|+1&amp;\geq 0 &amp;\text{(collecting terms)}
\end{align*}$$

Then, by the quadratic formula, we see that $|V|\geq 9 + 4\sqrt5 \approx 17.94$ (or $|V|\leq9-4\sqrt5$ but we're only interested in positive solutions for $|V|$).
It's not a problem that $17.94$ isn't an integer: the number of vertices must be greater than or equal to that number, so we actually have $|V|\geq 18$.