What does depth mean in the SqueezeNet architectural dimensions table?
In this paper the depth is defined as the number of layers. The table shows a depth=2 for the fire layers because each is comprised of 2 layers. First is the squeeze layer and then followed by the expand layer.

To calculate the number of parameters of a CNN we can do as follows for a single layer. Assume an input that is 28*28*64. This is the size of the MNIST dataset with 64 channels. Now assuming we want to convolve this input with 32 filters of a 3*3 kernel.

Let us give the following variable names, $l=64$ is the number of channels in the input, $m=n=3$ is the size of the kernel and $k = 32$ is the number of filters. The number of parameters is then calculated as 

$((n*m*l)+1)*k = ((3*3*64)+1)*32 = 18,464$.

The $+1$ is used to add the biases.



To calculate the number of parameters in fire2 we first note that this module is comprised of 2 distinct layers. 

Layer 1 - The squeeze layer - $s_{1x1}$

$l = 96$, $m = n = 1$ and $k = 16$. Thus giving a total of 1,552.

Layer 2 - The expansion layer

Kernel size 1 - $e_{1x1}$

$l = 16$, $m = n = 1$ and $k = 64$. Thus giving a total of 1,088.

Kernel size 3 - $e_{3x3}$

$l = 16$, $m = n = 3$ and $k = 64$. Thus giving a total of 9,280.

Total

Thus, a total of $1,552 + 1,088 + 9,280 = 11,920$.