What does "statistical efficiency" mean in this context?
Statistical efficiency in this context essentially means that a CNN would require fewer training examples than a fully connected network to learn. Intuitively this seems reasonable: more parameters to learn should mean more samples needed. Of course it is always desirable to minimise the number of training samples needed, so that's a definite advantage of CNNs.
There is a paper on the efficiency of CNNs which attempts to make that statement more precise. They examine the case of a convolutional network using a linear activation function.