How do we know that the neurons of an artificial neural network start by learning small features? I'd like to ask you how do we know that neural networks start by learning small, basic features or &quot;parts&quot; of the data and then use them to build up more complex features as we go through the layers. I've heard this a lot and seen it on videos like this one of 3Blue1Brown on neural networks for digit recognition. It says that in the first layer the neurons learn and detect small edges and then the neurons of the second layer get to know more complex patterns like circles... But I can't figure out based on pure maths how it's possible.
We do it experimentally; you're able to look at what each layer is learning by tweaking various values throughout the network and doing gradient ascent. For more detail, watch this lecture: https://www.youtube.com/watch?v=6wcs6szJWMY&amp;list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&amp;index=12 it provides many methods used for understanding exactly what your model is doing at a certain layer, and what features it has learnt.