Calculating the entropy of a neural network
To start, see Information Theory, Inference, and Learning Algorithms by David J.C. MacKay, starting with chapter 40 for information capacity of a single neuron (two bits per weight) through to at least chapter 42 for Hopfield Networks (fully connected feedback). The classic reference for information of a Hopfield Network is Information Capacity of the Hopfield Model by Abu-Mostafa &amp; St. Jacques, but the textbook should have enough.