title: ML.NET tensorflow half_plus_two
body: <p>I am trying to compare performance of evaluating a prebuilt tensorflow model with different inference methods.  I currently have the tensorflow serving <code>half_plus_two</code> toy model being served in an Ubuntu docker VM which produces the following results:</p>
  
  <p>GRPC: ~1700 predicts per second</p>
  
  <p>REST: ~800 predicts per second</p>
  
  <p>My end-use application (framework 4.5 with API) is a C# environment.  I'd like to compare ML.NET's performance to tensorflow serving's REST and GRPC by using the ML.NET prediction/predictionengine within my end-use application.</p>
  
  <h3>Environment</h3>
  
  <ul>
  <li>The model is already trained</li>
  <li>Inference is done by feeding a single float value to the model then processing the returned prediction</li>
  <li>End use application will be feeding real time data at very high rates per second</li>
  <li>Tensorflow 2.x to create the model and save in the <code>saved_model</code> format</li>
  </ul>
  
  <h3>ML.NET code</h3>
  
  <pre class="lang-cs prettyprint-override"><code>class TestProgram
  {
      static void Main(string[] args)
      {
          try
          {
              new ModelBuilder();
          }
          catch (Exception e)
          {
              // investigate
          }
      }
  }
  
  public class ModelBuilder
  {
      private readonly MLContext mlContext;
      private string userDesktop = Environment.SpecialFolder.Desktop.ToString();
  
      // feed these values to the pretrained tf model.
      // expected results are 2.0, 3.0, 4.0, 5.0 respectively
      private float[] testData = new float[] { 0.0f, 2.0f, 4.0f, 6.0f };
  
      public ModelBuilder()
      {
          this.mlContext = new MLContext();
          var tfPretrainedModel = this.mlContext.Model.LoadTensorFlowModel(Path.Combine(userDesktop, @"TF\half_plus_two\1\saved_model.pb"));
  
          var predictionFunction = this.mlContext.Model.CreatePredictionEngine&lt;HalfPlusTwoData, HalfPlusTwoPrediction&gt;(tfPretrainedModel);
  
          HalfPlusTwoPrediction prediction = null;
          for (int i = 0; i &lt; this.testData.Length; i++)
          {
              prediction = predictionFunction.Predict(new HalfPlusTwoData() { Input = this.testData[i] });
  
              Console.WriteLine($"Input {this.testData[i]}, Prediction {prediction.Prediction}, Expected {(this.testData[i] / 2) + 2}");
          }
      }
  }
  
  public class HalfPlusTwoData
  {
      [LoadColumn(0), ColumnName("Label")]
      public float Input;
  }
  
  public class HalfPlusTwoPrediction
  {
      [ColumnName("PredictedLabel")]
      public float Prediction { get; set; }
  }
  </code></pre>
  
  <h3>Questions</h3>
  
  <ul>
  <li>1 What is the proper way to use the <code>LoadTensorFlowModel</code> either to create the model or used in creating a pipeline?</li>
  <li>2 Is <code>HalfPlusTwoData</code> the proper way to structure input data?</li>
  <li>3 Is 'HalfPlusTwoPrediction' the proper way to structure the prediction class?</li>
  </ul>