title: Pass additional arguments to foreachBatch in pyspark
body: <p>I am using foreachBatch in pyspark structured streaming to write each microbatch to SQL Server using JDBC. I need to use the same process for several tables, and I'd like to reuse the same writer function by adding an additional argument for table name, but I'm not sure how to pass the table name argument.</p>
  
  <p>The example <a href="https://docs.databricks.com/spark/latest/structured-streaming/examples.html" rel="noreferrer">here</a> is pretty helpful, but in the python example the table name is hardcoded, and it looks like in the scala example they're referencing a global variable(?) I would like to pass the name of the table into the function.</p>
  
  <p>The function given in the python example at the link above is:</p>
  
  <pre><code>def writeToSQLWarehose(df, epochId):
    df.write \
      .format("com.databricks.spark.sqldw") \
      .mode('overwrite') \
      .option("url", "jdbc:sqlserver://&lt;the-rest-of-the-connection-string&gt;") \
      .option("forward_spark_azure_storage_credentials", "true") \
      .option("dbtable", "my_table_in_dw_copy") \
      .option("tempdir", "wasbs://&lt;your-container-name&gt;@&lt;your-storage-account-name&gt;.blob.core.windows.net/&lt;your-directory-name&gt;") \
      .save()
  </code></pre>
  
  <p>I'd like to use something like this:</p>
  
  <pre><code>def writeToSQLWarehose(df, epochId, tableName):
    df.write \
      .format("com.databricks.spark.sqldw") \
      .mode('overwrite') \
      .option("url", "jdbc:sqlserver://&lt;the-rest-of-the-connection-string&gt;") \
      .option("forward_spark_azure_storage_credentials", "true") \
      .option("dbtable", tableName) \
      .option("tempdir", "wasbs://&lt;your-container-name&gt;@&lt;your-storage-account-name&gt;.blob.core.windows.net/&lt;your-directory-name&gt;") \
      .save()
  </code></pre>
  
  <p>But I'm not sure how to pass the additional argument through foreachBatch.</p>