title: Google AI Platform: Unexpected error when loading the model: 'str' object has no attribute 'decode' [Keras 2.3.1, TF 1.15]
body: <p>I am trying to use the beta Google Custom Prediction Routine in Google's AI Platform to run a live version of my model.</p>
  <p>I include in my package <code>predictor.py</code> which contains a <code>Predictor</code> class as such:</p>
  <pre><code>import os
  import numpy as np
  import pickle
  import keras
  from keras.models import load_model
  
  class Predictor(object):
      &quot;&quot;&quot;Interface for constructing custom predictors.&quot;&quot;&quot;
  
      def __init__(self, model, preprocessor):
          self._model = model
          self._preprocessor = preprocessor
  
      def predict(self, instances, **kwargs):
          &quot;&quot;&quot;Performs custom prediction.
  
          Instances are the decoded values from the request. They have already
          been deserialized from JSON.
  
          Args:
              instances: A list of prediction input instances.
              **kwargs: A dictionary of keyword args provided as additional
                  fields on the predict request body.
  
          Returns:
              A list of outputs containing the prediction results. This list must
              be JSON serializable.
          &quot;&quot;&quot;
          # pre-processing
          preprocessed_inputs = self._preprocessor.preprocess(instances[0])
  
          # predict
          outputs = self._model.predict(preprocessed_inputs)
  
          # post-processing
          outputs = np.array([np.fliplr(x) for x in x_test])
          return outputs.tolist()
  
      @classmethod
      def from_path(cls, model_dir):
          &quot;&quot;&quot;Creates an instance of Predictor using the given path.
  
          Loading of the predictor should be done in this method.
  
          Args:
              model_dir: The local directory that contains the exported model
                  file along with any additional files uploaded when creating the
                  version resource.
  
          Returns:
              An instance implementing this Predictor class.
          &quot;&quot;&quot;
          model_path = os.path.join(model_dir, 'keras.model')
          model = load_model(model_path, compile=False)
  
          preprocessor_path = os.path.join(model_dir, 'preprocess.pkl')
          with open(preprocessor_path, 'rb') as f:
              preprocessor = pickle.load(f)
  
          return cls(model, preprocessor)
  </code></pre>
  <p>The full error <code>Create Version failed. Bad model detected with error: &quot;Failed to load model: Unexpected error when loading the model: 'str' object has no attribute 'decode' (Error code: 0)&quot; </code> indicates that the issue is in this script, specifically when loading the model. However, I am able to successfully load the model in my notebook locally with the same code block in <code>predict.py</code>:</p>
  <pre><code>from keras.models import load_model
  model = load_model('keras.model', compile=False)
  </code></pre>
  <p>I have seen similar posts which suggest to set the version of <code>h5py&lt;3.0.0</code> but this hasn't helped. I can set versions of modules for my custom prediction routine as such in a <code>setup.py</code> file:</p>
  <pre><code>from setuptools import setup
  
  REQUIRED_PACKAGES = ['keras==2.3.1', 'h5py==2.10.0', 'opencv-python', 'pydicom', 'scikit-image']
  
  setup(
      name='my_custom_code',
      install_requires=REQUIRED_PACKAGES,
      include_package_data=True,
      version='0.23',
      scripts=['predictor.py', 'preprocess.py'])
  </code></pre>
  <p>Unfortunately, I haven't found a good way to debug model deployment in google's AI Platform and the troubleshooting guide is unhelpful. Any pointers would be much appreciated. Thanks!</p>
  <p><strong>Edit 1:</strong></p>
  <p><strong>The h5py module's version is wrong</strong> –– at 3.1.0, despite setting it to 2.10.0 in <code>setup.py</code>. Anyone know why? I confirmed that Keras version and other modules are set properly however. I've tried <code>'h5py==2.9.0'</code> and <code>'h5py&lt;3.0.0'</code> to no avail. More on including PyPi package dependencies <a href="https://cloud.google.com/ai-platform/training/docs/versioning#pypi-package-dependencies" rel="nofollow noreferrer">here</a>.</p>
  <p><strong>Edit 2:</strong></p>
  <blockquote>
  <p>So it turns out google currently does not support this capability.</p>
  </blockquote>
  <ul>
  <li><a href="https://stackoverflow.com/questions/62816129/how-do-you-override-google-ai-platforms-standard-librarys-i-e-upgrade-scikit">StackOverflow, enzed01</a></li>
  </ul>