What exactly representational bottleneck in InceptionV3 means? I am trying to understand the concepts behind the InceptionNet V3 and got confused with the meaning of representational bottleneck. They said


  One should avoid bottlenecks with extreme compression. In general the
  representation size should gently decrease from the inputs to the outputs 
  before reaching the final representation used for the task at hand.


and one thing I get from this is that we should not use polling or convolutions with large strides. But when I arrived in the section 5 (Efficient Grid Size reduction) they said that doing polling before the convolution violates that mentioned principle but doing polling after convolution don't violate. 



Doing the left one we decrease the number of features by a factor of one quarter and then the features are doubled. In other words a decrease of 35*35*320*0.75 = 294K features and then an increase of 17*17*320 = 92.48K features. And doing the one in the right the features are doubled and then decrease by the same factor of one quarter. And doing this one, the real decrease in the pooling phase is bigger: 35*35*640*0.75 = 588K.

So, why one approach violates the principle and the other don't? What am I missing here?

Another thing that confused me is the solution found by them to avoid the computational cost of doing conv. before polling.



The polling part of the solution is not violating the principle of representational bottleneck once it make the exactly same operation with dimensionality reduction mentioned earlier?
Cool question! I managed to start thinking about its meaning thanks to your asking.

I believe you would like to consider it in the sense of information flow. As the information is compressed to a same size in both cases, the question is which one is more effective. Information with amount of 35*35*640 can flow through a wider pipe more freely. A pooling layer just prunes information directly in the left solution, while doing Inception in the right case expands the representation to a more sparse space and makes meaningful inferences before get pruned. Think of the pooling operation in left as a neck or a narrow path, and the middle representation in right as a broad lake. The pooling layer in right can also select better output since the source has more abstract (semantics) information.

As to figure 10, I consider it as a balance between the representation size and the computation burden.