title: can I use mlflow python API to register a spark UDF & then use the UDF in Spark scala code?
body: <p>I'm trying to use mlflow to do the machine learning work. I register the ML model as UDF using the following python code. The question is how can I use the UDF(test_predict) in my scala code? The reason is that our main code is in Scala. The problem is that UDF created below is a temporary UDF and SparkSession scoped. thanks!</p>
  
  <pre class="lang-py prettyprint-override"><code>import sys
  import mlflow
  from mlflow import pyfunc
  import numpy as np
  from pyspark import SparkContext, SparkConf
  from pyspark import SQLContext
  from pyspark.sql.session import SparkSession
  from pyspark.sql.types import *
  
  sc=SparkContext()
  spark = SparkSession.builder.appName("Python UDF example").getOrCreate()
  pyfunc_udf=mlflow.pyfunc.spark_udf(spark=spark, model_uri="./sk",result_type="float")
  spark.udf.register("test_predict",pyfunc_udf)
  </code></pre>