Gradients for bias terms in backpropagation
I would like to explain the meaning of db2=np.sum(dz2,axis=0,keepdims=True) as it also confused me once and it didn't get answered.

The derivative of L (loss) w.r.t. b is the upstream derivative multiplied with the local derivate:
$$
\frac{ \partial L}{\partial \mathbf{b}} = \frac{ \partial L}{\partial Z} \frac{ \partial Z}{\partial \mathbf{b}}
$$

If we have multiple samples Z and L are both matrices. b is still a vector.

The local derivative is simply a vector of ones:
$$
\frac{ \partial Z}{\partial \mathbf{b}} = \frac{\partial}{\partial \mathbf{b}} W \times X + \mathbf{b} = \mathbf{1}
$$

That means our complete derivative is a matrix multiplication, that looks as follows (e.g. 2 samples with 3 outputs):
$$
\frac{\partial L}{\partial Z} \times \mathbf{1} = 
\begin{bmatrix}
. &amp;.  &amp;. \\ 
. &amp;.  &amp;. 
\end{bmatrix}
\begin{bmatrix}
1\\ 
1\\
1\\
\end{bmatrix}
$$

Note that this is the sum of the rows. 

And that's where db2=np.sum(dz2, axis=0, keepdims=True) comes from. It is simply an abbreviation for the matrix multiplication of the local and the upstream derivatives.