Question about "1x3 and 3x1 conv is equivalent to 3x3 conv" I see a lot of sites talk that we can substitute 1x3 conv + 3x1 conv for 3x3 conv.

In order to demonstrate easily, we use a 3x3 image as an example.

From the point of view of parameters, I know that we will get fewer parameters and end up with the same result.
If the image(3x3) perform a 3x3 conv, we will get a 1x1 scalar, and this kernel size is 3x3, means we have 9 parameters.
On the other hand, if the image first performs a 1x3 convolution, and then performs a 3x1 convolution on its output. It will intuitively get the same result, a 1x1 scalar, and this kernel size is 2x3x1 = 6, which means we have 6 parameters.

However, from the point of view of calculation time, I don't think 3x3 should be substituted by 1x3 &amp; 3x1.  

For example, if we use the image to perform a 3x3 kernel, we should do 9 times of pixel-wise multiply operations, and then one time of addition. Totally 10 operations to get the result. 

But, if we use the image(3x3) to perform a 3x1 kernel, we have to implement [3(multiply) + 1(add)] * 3 times to get a 1x3 feature map, then we still need to perform a 1x3 kernel. It will then takes 3(multiply) + 1(add) operations. Totally 16 operations throughout the whole process.

What I want to show is that although we reduce the parameters from replacing 3x3 conv to 3x1 &amp; 1x3 conv, we still need more time to calculate during the training process. 

Training spends lots of time and implements 3x1 &amp; 1x3 doesn't save our time. So, should we substitute 1x3 conv + 3x1 conv for 3x3 conv?
A 3x3 convolution can do things a 1x3 followed by a 3x1 cannot do, for instance diagonal filtering.

As you said, it's all about the number of parameters, the first convolution has far more capabilities than the second solution.