When should we use separable convolution?
Context of the question
This is a link to the text cited in the question.
It refers to the usage of SeparableConv2D (tf, keras name). A related question on StackOverflow is &quot;What is the difference between SeparableConv2D and Conv2D layers&quot;. This answer points to this excellent article by Chi-Feng Wang:
A Basic Introduction to Separable Convolutions
Answer to the question
In image processing, a separable convolution converts a NxM convolution to two convolutions with kernels Nx1 and 1xM. Using this idea, in NN a SeparableConv2D converts a WxHxD convolution (width x height x depth, where depth means number of incoming features ) to two convolutions with kernels WxHx1 and 1x1xD.
Note the first kernel doesn't handles information across features, thus, it is &quot;learning of spatial features&quot;. The 1x1xD kernel doesn't handles different points, it is &quot;learning of channel-wise features&quot;.
About the phrase &quot;spatial locations in the input are highly correlated&quot;, my understanding of what the author means is: Assume we have a channel (feature) image that each pixel measures the &quot;distance to the background&quot;. When we pass from one pixel to a neighbors one, it is expected some continuity in the value (except for edge pixels): correlation. Instead, if we have a channel that measures &quot;brightness&quot; and another one that measures &quot;distance to background&quot; the two values for one specific pixel has little correlation.
Finally, about title question &quot;When should we use separable convolution?&quot; : if the final output must depend of some features of one pixel and some other features of neighbors pixels in a very unpredictable way, a complete WxHxD convolution must be used. However if, as is more usual, you can handle first spatial dependencies (neighborhood) to extract pixel features and next handle pixel-by-pixel these features to get the output, better use a WxHx1 followed by 1x1xD, saving lots of network parameters, thus, saving training time.