title: TensorFlow Serving for images as base64-encoded strings on Cloud ML Engine
body: <h2>How to implement TensorFlow Serving Input function for images as base64-encoded strings and get prediction on Cloud ML Engine</h2>
  
  <p>I am planning to deploy the model on Cloud Machine Learning (ML) Engine after training it on-prem., but I have no idea how to implement the <strong>serving input function</strong>. </p>
  
  <p>Additionally, I have tried to avoid TensorFlow low-level APIs and only focused on TensorFlow high-level APIs (<strong>TensorFlow Estimator</strong>). Below here in the blocks of code is the example code that I am working on.</p>
  
  <hr>
  
  <pre><code>import numpy as np
  import tensorflow as tf
  import datetime
  import os
  
  # create model
  from tensorflow.python.keras.applications.vgg16 import VGG16
  from tensorflow.python.keras import models
  from tensorflow.python.keras import layers
  
  conv_base = VGG16(weights='imagenet',
                    include_top=False,
                    input_shape=(150, 150, 3))
  
  model = models.Sequential()
  model.add(conv_base)
  model.add(layers.Flatten())
  model.add(layers.Dense(256, activation='relu'))
  model.add(layers.Dense(1, activation='sigmoid'))
  conv_base.trainable = False
  model.compile(loss='binary_crossentropy',
                optimizer=tf.keras.optimizers.RMSprop(lr=2e-5),
                metrics=['acc'])
  
  dt = datetime.datetime.now()
  datetime_now = dt.strftime("%y%m%d_%H%M%S")
  model_dir = 'models/imageclassifier_'+datetime_now
  model_dir = os.path.join(os.getcwd(), model_dir)
  if not os.path.exists(model_dir):
      os.makedirs(model_dir)
  print ("model_dir: ",model_dir)
  
  est_imageclassifier = tf.keras.estimator.model_to_estimator(keras_model=model, model_dir=model_dir)
  
  # input layer name
  input_name = model.input_names[0]
  input_name
  </code></pre>
  
  <p>This section is for image input function.</p>
  
  <pre><code>def imgs_input_fn(filenames, labels=None, perform_shuffle=False, repeat_count=1, batch_size=1):
      def _parse_function(filename, label):
          image_string = tf.read_file(filename)
          image = tf.image.decode_image(image_string, channels=3)
          image.set_shape([None, None, None])
          image = tf.image.resize_images(image, [150, 150])
          image = tf.subtract(image, 116.779) # Zero-center by mean pixel
          image.set_shape([150, 150, 3])
          image = tf.reverse(image, axis=[2]) # 'RGB'-&gt;'BGR'
          d = dict(zip([input_name], [image])), label
          return d
      if labels is None:
          labels = [0]*len(filenames)
      labels=np.array(labels)
      # Expand the shape of "labels" if necessary
      if len(labels.shape) == 1:
          labels = np.expand_dims(labels, axis=1)
      filenames = tf.constant(filenames)
      labels = tf.constant(labels)
      labels = tf.cast(labels, tf.float32)
      dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
      dataset = dataset.map(_parse_function)
      if perform_shuffle:
          # Randomizes input using a window of 256 elements (read into memory)
          dataset = dataset.shuffle(buffer_size=256)
      dataset = dataset.repeat(repeat_count)  # Repeats dataset this # times
      dataset = dataset.batch(batch_size)  # Batch size to use
      iterator = dataset.make_one_shot_iterator()
      batch_features, batch_labels = iterator.get_next()
      return batch_features, batch_labels
  </code></pre>
  
  <p>I would like to create a <strong>serving input function</strong> that</p>
  
  <ol>
  <li><p><strong>Get images as base64-encoded strings in JSON format</strong></p></li>
  <li><p><strong>Convert them into Tensors and reduce the size to (?, 150, 150, 3) for prediction</strong></p></li>
  </ol>
  
  <p>As shown below,</p>
  
  <pre><code>def serving_input_receiver_fn():
  
  ''' CODE HERE!'''
  
  return tf.estimator.export.ServingInputReceiver(feature_placeholders, feature_placeholders)
  </code></pre>
  
  <p>To train and evaluate the model,</p>
  
  <pre><code>train_spec = tf.estimator.TrainSpec(input_fn=lambda: imgs_input_fn(train_files,
                                                                     labels=train_labels,
                                                                     perform_shuffle=True,
                                                                     repeat_count=1,
                                                                     batch_size=20), 
                                      max_steps=500)
  
  exporter = tf.estimator.LatestExporter('Servo', serving_input_receiver_fn)
  
  eval_spec = tf.estimator.EvalSpec(input_fn=lambda: imgs_input_fn(val_files,
                                                                   labels=val_labels,
                                                                   perform_shuffle=False,
                                                                   batch_size=1),
                                   exporters=exporter)
  
  tf.estimator.train_and_evaluate(est_imageclassifier, train_spec, eval_spec)
  </code></pre>
  
  <hr>
  
  <p>If I understand it correctly, the example of the input file to get the prediction on Cloud ML Engine should be something like</p>
  
  <p>request.json</p>
  
  <pre><code>{"b64": "9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHJC...”}
  {"b64": "9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHJC...”}
  </code></pre>
  
  <p>And</p>
  
  <pre><code>gcloud ml-engine predict --model MODEL_NAME  \
                  --version MODEL_VERSION \
                  --json-instances request.json
  </code></pre>
  
  <p>If you are reading until here and have some idea, could you please suggest me how to implement the serving input function for this particular case.</p>
  
  <p>Many thanks in advance,</p>
  
  <hr>
  
  <h2>2nd Post - To update what I have done so far.</h2>
  
  <p>According to sdcbr's comment, below here is my serving_input_receiver_fn(). </p>
  
  <p>For _img_string_to_tensor() function or (prepare_image function), I guess that <strong>I should do image preparation the same way as I trained the model</strong> that you can see</p>
  
  <p>imgs_input_fn() => _parse_function().</p>
  
  <pre><code>def serving_input_receiver_fn():
      def _img_string_to_tensor(image_string):
          image = tf.image.decode_image(image_string, channels=3)
          image.set_shape([None, None, None])
          image = tf.image.resize_images(image, [150, 150])
          image = tf.subtract(image, 116.779) # Zero-center by mean pixel
          image.set_shape([150, 150, 3])
          image = tf.reverse(image, axis=[2]) # 'RGB'-&gt;'BGR'
          return image
  
      input_ph = tf.placeholder(tf.string, shape=[None])
  
      images_tensor = tf.map_fn(_img_string_to_tensor, input_ph, back_prop=False, dtype=tf.float32)
  
      return tf.estimator.export.ServingInputReceiver({model.input_names[0]: images_tensor}, {'image_bytes': input_ph})
  </code></pre>
  
  <p>After I trained the model and deployed the saved model on Cloud ML Engine. My input image was prepared into the format shown below.</p>
  
  <pre><code>{"image_bytes": {"b64": "YQ=="}}
  </code></pre>
  
  <p>But I found the error after getting a prediction via gcloud.</p>
  
  <pre><code>gcloud ml-engine predict --model model_1  \
                 --version v1 \
                 --json-instances request.json
  </code></pre>
  
  <blockquote>
    <p>{   "error": "Prediction failed: Error during model execution:
    AbortionError(code=StatusCode.INVALID_ARGUMENT, details=\"assertion
    failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]\n\t
    [[{{node
    map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert}} =
    Assert[T=[DT_STRING], summarize=3,
    _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](map/while/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp,
    map/while/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]\")"
    }</p>
  </blockquote>
  
  <p><strong>Did I do something wrong in _img_string_to_tensor function?</strong> </p>
  
  <p>and Could you please clarify me more about this tf.placeholder?</p>
  
  <pre><code>input_ph = tf.placeholder(tf.string, shape=[None])
  </code></pre>
  
  <p>For your above code, you use shape=[1], but I think it should be shape=[None].</p>