If $m \leq a \log m + b$, why does $m = O(\max\{a \log a, b\})$?
$m \leq a \log m + b$ has no asymptotic behavior. This is because regardless of constants $a, b$, the left hand side grows faster than the right side, leaving only a finite range where $m$ is valid.

It is likely what was meant is $m \leq a \log a + b$. In this case we have that $m$ is bounded by something, so $m = O(a \log a + b)$. If you have that $O(x + y) = O(\max(x, y))$, then the question is solved.

But why is $O(x + y) = O(\max(x, y))$ true? With big-$O$ notation we are interested in the asymptotically fastest growing part. We ignore constant factors. Now check three cases (in the limit):

$$x &lt; y \rightarrow O(x + y) = O(y)$$
$$x &gt; y \rightarrow O(x + y) = O(x)$$
$$x = y \rightarrow O(x + y) = O(x + x) = O(2x) = O(x)$$

Since our choice for $x = y$ doesn't matter, we can see that choosing the minimum gives us the right asymptotic behavior.