title: 'transpose expects a vector of size 5. But input(1) is a vector of size 3\n\t " when making inference POST request to tensorflow serving model
body: <p>I have trained a model and deployed it to tensorflow-serving for inference.</p>
  
  <p>I am getting this error when making a request: </p>
  
  <pre><code>&lt;Response [400]&gt;
  {'error': 'transpose expects a vector of size 5. But input(1) is a vector of size 3\n\t [[{{node bidirectional_1/transpose}} = Transpose[T=DT_FLOAT, Tperm=DT_INT32, _class=["loc:@bidirectional_1/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3"], _output_shapes=[[50,?,512]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](embedding_1/embedding_lookup, Attention/transpose/perm)]]'}
  </code></pre>
  
  <p>The notable difference between this model and the first I deployed that worked without issue is that it contains a Keras custom Layer whereas my successful attempt contained only standard Keras layers.</p>
  
  <p>This is how I am testing the POST request to my tf-serving model:</p>
  
  <pre><code>with open("CNN_last_test_set.pkl", "rb") as fp:
      x_arr_test, y_test = pickle.load(fp)
  
  out = x_arr_test[:1, :]
  out = out.tolist()
  payload = {
      "instances": [{'input': [out]}]
  }
  
  r = requests.post('http://localhost:9000/v1/models/prod_mod:predict', json=payload)
  pred = json.loads(r.content.decode('utf-8'))
  </code></pre>
  
  <p>To create the tensorflow model object to use with tf-serving I am using this function:</p>
  
  <pre><code>def export_model_custom_layer(filename, export_path_base):
      # set the mode to test time.
      K.set_learning_phase(0)
      model = keras.models.load_model(filename, custom_objects={"Attention": Attention})
      sess = K.get_session()
  
      # set the path to save the model and model version
      export_version = 1
  
      export_path = os.path.join(
          tf.compat.as_bytes(export_path_base),
          tf.compat.as_bytes(str(export_version)))
      tf.saved_model.simple_save(
          sess,
          export_path,
          inputs={'input': model.input},
          outputs={t.name.split(':')[0]: t for t in model.outputs},
          legacy_init_op=tf.tables_initializer())
  </code></pre>
  
  <p>Where I've defined my customer layer as a custom object, in order for this to work I've added this function to my customer layer:</p>
  
  <pre><code>    def get_config(self):
          config = {
              'name': "Attention"
                    }
          base_config = super(Attention, self).get_config()
          return dict(list(base_config.items()) + list(config.items()))
  </code></pre>
  
  <p>When I predict with the model using the same data format as the tf-serving model is receiving using standard keras model.predict(), it works as intended:</p>
  
  <pre><code>class Attention(Layer):...
  
  with open("CNN_last_test_set.pkl", "rb") as fp:
      x_arr_test, y_test = pickle.load(fp)
  
  model = keras.models.load_model(r"Data/modelCNN.model", custom_objects={"Attention": Attention})
  out = x_arr_test[:1, :]
  test1 = out.shape
  out = out.tolist()
  
  test = model.predict([out])
  
  &gt;&gt; print(test)
  &gt;&gt; [[0.21351092]]
  </code></pre>
  
  <p>This leads me to believe that the issue is happening either when I export the model from keras to the .pb file or in some way the model is being run in the docker container.</p>
  
  <p>I am not sure what to make of this error but I'm assuming that this is related to my custom layer object considering that it worked with my previous model that only contained standard Keras layers.</p>
  
  <p>Any help is greatly appreciated, thanks!</p>
  
  <p>EDIT: I solved the issue, the problem was that my input data had two extra dimensions than necessary. I realized that when I removed the brackets from around the variable "out" my error changed from being 'transpose expects a vector of size 5' to 'transpose expects a vector of size 4'. So I reshaped my "out" variable from being (1, 50) to (50,) &amp; removed the brackets and the problem resolved itself.</p>