Disproving using the definition of O
Here is one possible definition of big O (your definition might vary in details):


  Let $f(n),g(n)$ be two real-valued functions such that $f(n),g(n) &gt; 0$ for all $n$. We say that $f(n) = O(g(n))$ if there exists an integer $N$ and a real $C &gt; 0$ such that for all $n \geq N$, $f(n) \leq Cg(n)$.


Using the rules of logic, we get a definition of the negation of big O:


  Let $f(n),g(n)$ be two real-valued functions such that $f(n),g(n) &gt; 0$ for all $n$. We say that $f(n)$ is not big $O(g(n))$ if for every integer $N$ and real $C &gt; 0$ there exists $n \geq N$ such that $f(n) &gt; Cg(n)$.


This is what you should prove. For each integer $N$ and real $C &gt; 0$ you should find $n \geq N$ such that $0.1n + 10\sqrt{n} &gt; C\sqrt{n}$.