title: Convolution2D "Depth of input is not a multiple of input depth of filter"
body: <p>I want to use fft2d transform in tensorflow and then analyse the magnitude and phase result with convolutional layers... I have made a system with Lambda layers to use tensorflow functions and get the magnitude and the phase. But when I add the Conv2d layer, I get the error </p>
  
  <p><code>Depth of input (3) is not a multiple of input depth of filter (199) for '1_Magn_Conv_f500_k2_2/convolution' (op: 'Conv2D') with input shapes: [?,199,199,3], [2,2,199,500]</code> </p>
  
  <p>I don't understand what is the <code>shape [2,2,199,500]</code> and what is causing this error.</p>
  
  <p>I have tried to reduce the number of layers in my networks to detect which one creates the problem. I have checked that magn_angle outputs two tensors with <code>shapes [None,199,199,3]</code>.
  I am working with google colab.</p>
  
  <p>Here is the minimal code to reproduce the error</p>
  
  <pre class="lang-py prettyprint-override"><code>inpt = Input(shape = (199, 199, 3),name=str(0)+'_'+'Image')
  def magn_angle(x):
      x = Lambda(lambda x:K.cast(x,dtype=tf.complex64))(x)
      x_list_magn = []
      x_list_angle = []
      for i in range(3):
          fft = Lambda(lambda x: tf.fft2d(x[:,:,:,i]), output_shape=(None,199,199))(x)# 2-dimensional discrete Fourier transform over the inner-most 2 dimensions
          x_list_magn.append(Lambda(lambda fft:K.expand_dims(tf.math.abs(fft),axis=-1), output_shape=(None,199,199))(fft))
          x_list_angle.append(Lambda(lambda fft: K.expand_dims(tf.math.angle(fft),axis=-1), output_shape=(None,199,199))(fft))
      magn = Concatenate()(x_list_magn)
      angle = Concatenate()(x_list_angle)
      magn = Lambda(lambda magn: K.cast(magn,dtype=tf.float32), output_shape=(None,199,199))(magn)
      angle = Lambda(lambda angle: K.cast(angle,dtype=tf.float32), output_shape=(None,199,199))(angle)
      return magn,angle
  magn, angle = magn_angle(inpt)
  magn = Conv2D(filters=500,kernel_size=(2,2),activation=None,strides=(1,1),padding='SAME',name=str(1)+'_'+'Magn_Conv_f500_k2',data_format="channels_last")(magn)
  ...
  </code></pre>
  
  <p>Which ouputs</p>
  
  <p><code>InvalidArgumentError: Depth of input (3) is not a multiple of input depth of filter (199) for '1_Magn_Conv_f500_k2_3/convolution' (op: 'Conv2D') with input shapes: [?,199,199,3], [2,2,199,500]</code>.</p>