How to find the various matrix sizes in designing a CNN
For a CNN layer with input of dimensions h * w * d, kernel size k * k and number of kernel filters as f, we have the number of parameters as k * k * d * f, if we ignore the biases. If use biases then the number of parameters becomes (k * k * d + 1) * f

For e.g., the 1st conv layer has 5 * 5 * 2 * 20 parameters if we are ignoring the biases. With bias, the number of parameters would become (5 * 5 * 2 + 1) * 20.

Note that the number of parameters does not depend on the stride, padding, pooling, dropout nor on the spatial dimensions of the input or output!

To find the total number of parameters in the network one needs to add the parameters of all the individual layers in the model.