title: How to convert Tensorflow 2.0 SavedModel to TensorRT?
body: <p>I've trained a model in Tensorflow 2.0 and am trying to improve predict time when moving to production (on a server with GPU support). In Tensorflow 1.x I was able to get a predict speedup by using freeze graph, but this has been deprecated as of Tensorflow 2. From reading Nvidia's description of TensorRT, they suggest that using TensorRT can speedup inference by 7x compared to Tensorflow alone. Source:</p>
  
  <p><a href="https://news.developer.nvidia.com/tensorflow-2-0-with-tighter-tensorrt-integration-now-available/" rel="nofollow noreferrer">TensorFlow 2.0 with Tighter TensorRT Integration Now Available</a></p>
  
  <p>I have trained my model and saved it to a .h5 file using Tensorflow's SavedModel format. Now I follow nvidia's documentation to optimize the model for inference with tensorrt: <a href="https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#worflow-with-savedmodel" rel="nofollow noreferrer">TF-TRT 2.0 Workflow With A SavedModel</a>.</p>
  
  <p>When I run:</p>
  
  <pre><code>import tensorflow as tf
  from tensorflow.python.compiler.tensorrt import trt_convert as trt
  </code></pre>
  
  <p>I get the error: <strong>ModuleNotFoundError: No module named 'tensorflow.python.compiler.tensorrt'</strong></p>
  
  <p>They give another example with Tensorflow 2.0 here: <a href="https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#usage-example" rel="nofollow noreferrer">Examples</a>. However, they try to import the same module as above and I get the same error.</p>
  
  <p>Can anyone suggest how to optimize my model with TensorRT?</p>