cannot construct the infinite type
Okay, let's look at what we know about the type of applyTwice, $\alpha$:


It is given two arguments, so $\alpha = \beta_1 \to \beta_2 \to \gamma$, and $f$ has type $\beta\_1$ and $x$ has type $\beta_2$
$f$ is applied to $f$, so $\beta_1 = \beta_1 \to \gamma_2$
$(f\ f)$ is applied to $x$, so $\gamma_2 = \beta_2 \to \gamma$


The kicker here is the second equation, $\beta_1 = \beta_1 \to \gamma_2$. This equation has no finite solution, because $S \to T$ is structurally larger (uses at least one more type constructor) than either $S$ or $T$. 

So the only type that satisfies this equation is infinite recursive type $\mu T\ldotp T \to \gamma_2 = ((\ldots \to \gamma\_2) \to \gamma\_2) \to \gamma\_2) \to \gamma\_2) \to \gamma\_2) \to \gamma\_2)$. But this type doesn't really have any meaning in Haskell or related type systems, and it's pretty hard represent an infinite type in finite memory in a concrete form.

Remember that $f\ f$ is giving $f$ as an argument to $f$. If you want to apply $f$ twice, you could use function composition (f . f) x, or $(f \circ f)\ x$ in math notation.