How to derive the answer to this convolution problem?
HINT: what is the highest power of $x$ after multiplying $A(x)$ and $B(x)$? This gives you directly the value of $N_3$. Then rewrite $B(x)$ as

$$B(x)=\sum_{n=0}^{2N_2}\hat{b}_nx^n$$

and you can use normal convolution of $a(n)$ and $\hat{b}(n)$ to derive $c(n)$. Now you just need to express $\hat{b}(n)$ in terms of $b(n)$.

EDIT: OK, so here's the solution:

We have

$$C(x)=\sum_{n=0}^{N_3}c_nx^n$$

Since $C(x)=A(x)B(x)$, the highest power of $C(x)$ must be $N_3=N_1+2N_2$. You can rewrite $B(x)$ as

$$B(x)=\sum_{n=0}^{2N_2}\hat{b}_nx^n$$

with

$$\hat{b}_n=\begin{cases}b_{n/2},&amp;n \text{ even}\\
0,&amp;n \text{ odd}\end{cases}$$

Now the coefficients $c_n$ can be written as the convolution of $a_n$ and  $\hat{b}_n$:

$$c_n=\sum_k\hat{b}_ka_{n-k}$$

For the index $k$ in the above sum we have the following constraints:

$$0\le k\le 2N_2\quad\text{and}\quad 0\le n-k\le N_1$$

which results in the summation limits

$$c_n=\sum_{k=\max\{0,n-N_1\}}^{\min\{n,2N_2\}}\hat{b}_ka_{n-k},\quad 0\le n\le N_3$$

If you sum only over even $k$, you can replace $\hat{b}_k$ by the coefficients $b_{k/2}$:

$$c_n=\sum_{k=\max\{0,n-N_1\},k\text{ even}}^{\min\{n,2N_2\}}b_{k/2}a_{n-k},\quad 0\le n\le N_3$$

which can be rewritten once more as

$$c_n=\sum_{k=\lceil{\max\{0,n-N_1\}/2}\rceil}^{\lfloor{\min\{n,2N_2\}/2}\rfloor}b_{k}a_{n-2k},\quad 0\le n\le N_3$$