What is the depth of an image in Convolutional Neural Network?
It means that the number of filters (a.k.a. kernels, or  feature detector) in the previous convolutional layer is 96. You may want to watch the video of the lecture, and in particular this slide, which mentions that a filter is applied to the full depth of your previous layer: