title: Deploy keras model use tensorflow serving got 501 Server Error: Not Implemented for url: http://localhost:8501/v1/models/genre:predict
body: <p>I saved keras .h5 model to .pb using SavedModelBuilder. After I use docker image of tensorflow/serving:1.14.0 deploy my model, when I run predict process, I got the "requests.exceptions.HTTPError: 501 Server Error: Not Implemented for url: <a href="http://localhost:8501/v1/models/genre:predict" rel="nofollow noreferrer">http://localhost:8501/v1/models/genre:predict</a>"</p>
  
  <p>The model building code as follows:</p>
  
  <pre><code>from keras import backend as K
  import tensorflow as tf   
  from keras.models import load_model
  
  model=load_model('/home/li/model.h5')
  
  model_signature = 
  tf.saved_model.signature_def_utils.predict_signature_def(
      inputs={'input': model.input}, outputs={'output': model.output})
  #export_path = os.path.join(model_path,model_version)
  export_path = "/home/li/genre/1" 
  
  builder = tf.saved_model.builder.SavedModelBuilder(export_path)
  builder.add_meta_graph_and_variables(
      sess=K.get_session(),
      tags=[tf.saved_model.tag_constants.SERVING],
      signature_def_map={
          'predict':
          model_signature,
          'serving_default':
          model_signature
      })
  builder.save()
  </code></pre>
  
  <p>Then I got the .pb model:
  <a href="https://i.stack.imgur.com/dYUzC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dYUzC.png" alt="pd model sturcture"></a></p>
  
  <p>When I run <code>saved_model_cli show --dir /home/li/genre/1 --all</code>, The saved .pd model infomation as follows:</p>
  
  <pre><code>MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
  
  signature_def['predict']:
    The given SavedModel SignatureDef contains the following input(s):
      inputs['input'] tensor_info:
          dtype: DT_FLOAT
          shape: (-1, 1, 128, 1292)
          name: conv2d_1_input_2:0
    The given SavedModel SignatureDef contains the following output(s):
      outputs['output'] tensor_info:
          dtype: DT_FLOAT
          shape: (-1, 19)
          name: dense_2_2/Softmax:0
    Method name is: tensorflow/serving/predict
  
  signature_def['serving_default']:
    The given SavedModel SignatureDef contains the following input(s):
      inputs['input'] tensor_info:
          dtype: DT_FLOAT
          shape: (-1, 1, 128, 1292)
          name: conv2d_1_input_2:0
    The given SavedModel SignatureDef contains the following output(s):
      outputs['output'] tensor_info:
          dtype: DT_FLOAT
          shape: (-1, 19)
          name: dense_2_2/Softmax:0
    Method name is: tensorflow/serving/predict
  </code></pre>
  
  <p>The command I use to deploy on docker image tensorflow/serving is</p>
  
  <p><code>docker run -p 8501:8501 --name tfserving_genre --mount type=bind,source=/home/li/genre,target=/models/genre -e MODEL_NAME=genre -t tensorflow/serving:1.14.0 &amp;</code></p>
  
  <p>When open <code>http://localhost:8501/v1/models/genre</code> in browser, I got the message</p>
  
  <pre><code>{
   "model_version_status": [
    {
     "version": "1",
     "state": "AVAILABLE",
     "status": {
      "error_code": "OK",
      "error_message": ""
     }
    }
   ]
  }
  </code></pre>
  
  <p>The client prediction code as follows:</p>
  
  <pre><code>import requests
  import numpy as np
  import os
  import sys
  
  from audio_to_spectrum_v2 import split_song_to_frames
  
  
  # Define a Base client class for Tensorflow Serving
  class TFServingClient:
      """
      This is a base class that implements a Tensorflow Serving client
      """
      TF_SERVING_URL_FORMAT = '{protocol}://{hostname}:    {port}/v1/models/{endpoint}:predict'
  
      def __init__(self, hostname, port, endpoint, protocol="http"):
          self.protocol = protocol
          self.hostname = hostname
          self.port = port
          self.endpoint = endpoint
  
      def _query_service(self, req_json):
          """
          :param req_json: dict (as define in https://cloud.google.com/ml-engine/docs/v1/predict-request)
          :return: dict
          """
          server_url =   self.TF_SERVING_URL_FORMAT.format(protocol=self.protocol,
                                                         hostname=self.hostname,
                                                     port=self.port,
                                                     endpoint=self.endpoint)
          response = requests.post(server_url, json=req_json)
          response.raise_for_status()
          print(response.json())
          return np.array(response.json()['output'])
  
  
  # Define a specific client for our inception_v3 model
  class GenreClient(TFServingClient):
      # INPUT_NAME is the config value we used when saving the model (the only value in the `input_names` list)
      INPUT_NAME = "input"
  
      def load_song(self, song_path):
          """Load a song from path,slices to pieces, and extract features, returned as np.array format"""
  
          song_pieces = split_song_to_frames(song_path,False,30)
          return song_pieces
  
      def predict(self, song_path):
          song_pieces = self.load_song(song_path)
  
          # Create a request json dict
          req_json = {
                  "instances": song_pieces.tolist()
          }
          print(req_json)
          return self._query_service(req_json)
  
  def main():    
      song_path=sys.argv[1]
  
      print("file name:{}".format(os.path.split(song_path)[-1]))
  
      hostname = "localhost"
      port = "8501"
      endpoint="genre"
      client = GenreClient(hostname=hostname, port=port, endpoint=endpoint)
  
      prediction = client.predict(song_path)
      print(prediction)
  
  if __name__=='__main__':
      main()
  </code></pre>
  
  <p>After run the prediction code, I got the error information as follows:</p>
  
  <pre><code>Traceback (most recent call last):
    File "client_predict.py", line 90, in &lt;module&gt;
      main()
    File "client_predict.py", line 81, in main
      prediction = client.predict(song_path)
    File "client_predict.py", line 69, in predict
      return self._query_service(req_json)
    File "client_predict.py", line 40, in _query_service
      response.raise_for_status()
    File "/home/li/anaconda3/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
      raise HTTPError(http_error_msg, response=self)
  requests.exceptions.HTTPError: 501 Server Error: Not Implemented for url: http://localhost:8501/v1/models/genre:predict
  </code></pre>
  
  <p>I wonder what's the reason of this deployment problem, and how to solve it, Thanks for all.</p>