class_docstr: Interface used to write a streaming :class:`DataFrame <pyspark.sql.DataFrame>` to external
  storage systems (e.g. file systems, key-value stores, etc).
  Use :attr:`DataFrame.writeStream <pyspark.sql.DataFrame.writeStream>`
  to access this.
  
  .. versionadded:: 2.0.0
  
  .. versionchanged:: 3.5.0
      Supports Spark Connect.
  
  Notes
  -----
  This API is evolving.
  
  Examples
  --------
  The example below uses Rate source that generates rows continuously.
  After that, we operate a modulo by 3, and then writes the stream out to the console.
  The streaming query stops in 3 seconds.
  
  >>> import time
  >>> df = spark.readStream.format("rate").load()
  >>> df = df.selectExpr("value % 3 as v")
  >>> q = df.writeStream.format("console").start()
  >>> time.sleep(3)
  >>> q.stop()
class_docstr_tok: ["Interface","used","to","write","a","streaming",":","class",":","`","DataFrame","<","pyspark.sql.DataFrame",">","`","to","external","storage","systems","(","e.g",".","file","systems",",","key-value","stores",",","etc",")",".","Use",":","attr",":","`","DataFrame.writeStream","<","pyspark.sql.DataFrame.writeStream",">","`","to","access","this",".","..","versionadded",":",":","2.0.0","..","versionchanged",":",":","3.5.0","Supports","Spark","Connect",".","Notes","--","--","-","This","API","is","evolving",".","Examples","--","--","--","--","The","example","below","uses","Rate","source","that","generates","rows","continuously",".","After","that",",","we","operate","a","modulo","by","3",",","and","then","writes","the","stream","out","to","the","console",".","The","streaming","query","stops","in","3","seconds",".",">",">",">","import","time",">",">",">","df","=","spark.readStream.format","(","``","rate","''",")",".load","(",")",">",">",">","df","=","df.selectExpr","(","``","value","%","3","as","v","''",")",">",">",">","q","=","df.writeStream.format","(","``","console","''",")",".start","(",")",">",">",">","time.sleep","(","3",")",">",">",">","q.stop","(",")"]
docstr: Sets the output of the streaming query to be processed using the provided
  function. This is supported only the in the micro-batch execution modes (that is, when the
  trigger is not continuous). In every micro-batch, the provided function will be called in
  every micro-batch with (i) the output rows as a DataFrame and (ii) the batch identifier.
  The batchId can be used deduplicate and transactionally write the output
  (that is, the provided Dataset) to external systems. The output DataFrame is guaranteed
  to exactly same for the same batchId (assuming all operations are deterministic in the
  query).
  
  .. versionadded:: 2.4.0
  
  .. versionchanged:: 3.5.0
      Supports Spark Connect.
  
  Notes
  -----
  This API is evolving.
  This function behaves differently in Spark Connect mode. See examples.
  In Connect, the provided function doesn't have access to variables defined outside of it.
  
  Examples
  --------
  >>> import time
  >>> df = spark.readStream.format("rate").load()
  >>> my_value = -1
  >>> def func(batch_df, batch_id):
  ...     global my_value
  ...     my_value = 100
  ...     batch_df.collect()
  ...
  >>> q = df.writeStream.foreachBatch(func).start()
  >>> time.sleep(3)
  >>> q.stop()
  >>> # if in Spark Connect, my_value = -1, else my_value = 100
docstr_tok: ["Sets","the","output","of","the","streaming","query","to","be","processed","using","the","provided","function",".","This","is","supported","only","the","in","the","micro-batch","execution","modes","(","that","is",",","when","the","trigger","is","not","continuous",")",".","In","every","micro-batch",",","the","provided","function","will","be","called","in","every","micro-batch","with","(","i",")","the","output","rows","as","a","DataFrame","and","(","ii",")","the","batch","identifier",".","The","batchId","can","be","used","deduplicate","and","transactionally","write","the","output","(","that","is",",","the","provided","Dataset",")","to","external","systems",".","The","output","DataFrame","is","guaranteed","to","exactly","same","for","the","same","batchId","(","assuming","all","operations","are","deterministic","in","the","query",")",".","..","versionadded",":",":","2.4.0","..","versionchanged",":",":","3.5.0","Supports","Spark","Connect",".","Notes","--","--","-","This","API","is","evolving",".","This","function","behaves","differently","in","Spark","Connect","mode",".","See","examples",".","In","Connect",",","the","provided","function","does","n't","have","access","to","variables","defined","outside","of","it",".","Examples","--","--","--","--",">",">",">","import","time",">",">",">","df","=","spark.readStream.format","(","``","rate","''",")",".load","(",")",">",">",">","my_value","=","-1",">",">",">","def","func","(","batch_df",",","batch_id",")",":","...","global","my_value","...","my_value","=","100","...","batch_df.collect","(",")","...",">",">",">","q","=","df.writeStream.foreachBatch","(","func",")",".start","(",")",">",">",">","time.sleep","(","3",")",">",">",">","q.stop","(",")",">",">",">","#","if","in","Spark","Connect",",","my_value","=","-1",",","else","my_value","=","100"]
code: def foreachBatch(
      self, func: Callable[["DataFrame", int], None]
  ) -> "DataStreamWriter":
      """
      Sets the output of the streaming query to be processed using the provided
      function. This is supported only the in the micro-batch execution modes (that is, when the
      trigger is not continuous). In every micro-batch, the provided function will be called in
      every micro-batch with (i) the output rows as a DataFrame and (ii) the batch identifier.
      The batchId can be used deduplicate and transactionally write the output
      (that is, the provided Dataset) to external systems. The output DataFrame is guaranteed
      to exactly same for the same batchId (assuming all operations are deterministic in the
      query).
  
      .. versionadded:: 2.4.0
  
      .. versionchanged:: 3.5.0
          Supports Spark Connect.
  
      Notes
      -----
      This API is evolving.
      This function behaves differently in Spark Connect mode. See examples.
      In Connect, the provided function doesn't have access to variables defined outside of it.
  
      Examples
      --------
      >>> import time
      >>> df = spark.readStream.format("rate").load()
      >>> my_value = -1
      >>> def func(batch_df, batch_id):
      ...     global my_value
      ...     my_value = 100
      ...     batch_df.collect()
      ...
      >>> q = df.writeStream.foreachBatch(func).start()
      >>> time.sleep(3)
      >>> q.stop()
      >>> # if in Spark Connect, my_value = -1, else my_value = 100
      """
  
      from pyspark.java_gateway import ensure_callback_server_started
  
      gw = self._spark._sc._gateway
      assert gw is not None
      java_import(
          gw.jvm, "org.apache.spark.sql.execution.streaming.sources.*"
      )
  
      wrapped_func = ForeachBatchFunction(self._spark, func)
      gw.jvm.PythonForeachBatchHelper.callForeachBatch(
          self._jwrite, wrapped_func
      )
      ensure_callback_server_started(gw)
      return self
  
imports: import sys
  from collections.abc import Iterator
  from typing import cast, overload, Any, Callable, List, Optional, TYPE_CHECKING, Union
  from py4j.java_gateway import java_import, JavaObject
  from pyspark.sql.column import _to_seq
  from pyspark.sql.readwriter import OptionUtils, to_str
  from pyspark.sql.streaming.query import StreamingQuery
  from pyspark.sql.types import Row, StructType
  from pyspark.sql.utils import ForeachBatchFunction
  from pyspark.errors import PySparkTypeError, PySparkValueError, PySparkAttributeError, PySparkRuntimeError
label_desc: Use image node_id 20 for calling the DataStreamWriter obj's underlying member method code with example usage: obj.foreachBatch(func) and returns: self