How to prove an implication about an upper bound mentioned in the proof of master theorem?
The statement $f = O(h)$ just states that there exists a constant $C&gt;0$ such that $f(N) \leq Ch(N)$ for all $N$ (some variants have this hold only for large enough $N$, but in most cases there is no difference). In your case, $f(N) \leq CN^{\log_b a-\epsilon}$ for all $N$. This holds for $N = n/b^j$ in particular, and implies that
$$
g(n) \leq C \sum_{j=0}^{\log_b n-1} a^j (n/b_j)^{\log_b a-\epsilon}.
$$

As for the two different definitions of big O: suppose that $f(N) \leq CN^{\log_b a-\epsilon}$ holds only for $N \geq N_0$, and let $M = \max(C,f(1),\ldots,f(N_0))$. Then $f(N) \leq MN^{\log_b a-\epsilon}$.