class_docstr: An iterator for reading records from a :class:`~Dataset`.
  
  For Datasets, each iteration call represents a complete read of all items in the
  Dataset.
  
  If using Ray Train, each trainer actor should get its own iterator by calling
  :meth:`ray.train.get_dataset_shard("train")
  <ray.train.get_dataset_shard>`.
  
  Examples:
      >>> import ray
      >>> ds = ray.data.range(5)
      >>> ds
      Dataset(num_blocks=..., num_rows=5, schema={id: int64})
      >>> ds.iterator()
      DataIterator(Dataset(num_blocks=..., num_rows=5, schema={id: int64}))
class_docstr_tok: ["An","iterator","for","reading","records","from","a",":","class",":","`","~Dataset","`",".","For","Datasets",",","each","iteration","call","represents","a","complete","read","of","all","items","in","the","Dataset",".","If","using","Ray","Train",",","each","trainer","actor","should","get","its","own","iterator","by","calling",":","meth",":","`","ray.train.get_dataset_shard","(","``","train","''",")","<","ray.train.get_dataset_shard",">","`",".","Examples",":",">",">",">","import","ray",">",">",">","ds","=","ray.data.range","(","5",")",">",">",">","ds","Dataset","(","num_blocks=","...",",","num_rows=5",",","schema=","{","id",":","int64","}",")",">",">",">","ds.iterator","(",")","DataIterator","(","Dataset","(","num_blocks=","...",",","num_rows=5",",","schema=","{","id",":","int64","}",")",")"]
docstr: Return a TF Dataset over this dataset.
  
  .. warning::
      If your dataset contains ragged tensors, this method errors. To prevent
      errors, :ref:`resize your tensors <transforming_tensors>`.
  
  Examples:
      >>> import ray
      >>> ds = ray.data.read_csv(
      ...     "s3://anonymous@air-example-data/iris.csv"
      ... )
      >>> it = ds.iterator(); it
      DataIterator(Dataset(
         num_blocks=...,
         num_rows=150,
         schema={
            sepal length (cm): double,
            sepal width (cm): double,
            petal length (cm): double,
            petal width (cm): double,
            target: int64
         }
      ))
  
      If your model accepts a single tensor as input, specify a single feature column.
  
      >>> it.to_tf(feature_columns="sepal length (cm)", label_columns="target")  # doctest: +SKIP
      <_OptionsDataset element_spec=(TensorSpec(shape=(None,), dtype=tf.float64, name='sepal length (cm)'), TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
      If your model accepts a dictionary as input, specify a list of feature columns.
  
      >>> it.to_tf(["sepal length (cm)", "sepal width (cm)"], "target")  # doctest: +SKIP
      <_OptionsDataset element_spec=({'sepal length (cm)': TensorSpec(shape=(None,), dtype=tf.float64, name='sepal length (cm)'), 'sepal width (cm)': TensorSpec(shape=(None,), dtype=tf.float64, name='sepal width (cm)')}, TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
      If your dataset contains multiple features but your model accepts a single
      tensor as input, combine features with
      :class:`~ray.data.preprocessors.Concatenator`.
  
      >>> from ray.data.preprocessors import Concatenator
      >>> preprocessor = Concatenator(output_column_name="features", exclude="target")
      >>> it = preprocessor.transform(ds).iterator()
      >>> it
      DataIterator(Concatenator
      +- Dataset(
            num_blocks=...,
            num_rows=150,
            schema={
               sepal length (cm): double,
               sepal width (cm): double,
               petal length (cm): double,
               petal width (cm): double,
               target: int64
            }
         ))
      >>> it.to_tf("features", "target")  # doctest: +SKIP
      <_OptionsDataset element_spec=(TensorSpec(shape=(None, 4), dtype=tf.float64, name='features'), TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
  Args:
      feature_columns: Columns that correspond to model inputs. If this is a
          string, the input data is a tensor. If this is a list, the input data
          is a ``dict`` that maps column names to their tensor representation.
      label_column: Columns that correspond to model targets. If this is a
          string, the target data is a tensor. If this is a list, the target data
          is a ``dict`` that maps column names to their tensor representation.
      prefetch_batches: The number of batches to fetch ahead of the current batch
          to fetch. If set to greater than 0, a separate threadpool will be used
          to fetch the objects to the local node, format the batches, and apply
          the collate_fn. Defaults to 1.
      batch_size: Record batch size. Defaults to 1.
      drop_last: Set to True to drop the last incomplete batch,
          if the dataset size is not divisible by the batch size. If
          False and the size of dataset is not divisible by the batch
          size, then the last batch will be smaller. Defaults to False.
      local_shuffle_buffer_size: If non-None, the data will be randomly shuffled
          using a local in-memory shuffle buffer, and this value will serve as the
          minimum number of rows that must be in the local in-memory shuffle
          buffer in order to yield a batch. When there are no more rows to add to
          the buffer, the remaining rows in the buffer will be drained. This
          buffer size must be greater than or equal to ``batch_size``, and
          therefore ``batch_size`` must also be specified when using local
          shuffling.
      local_shuffle_seed: The seed to use for the local random shuffle.
  
  Returns:
      A ``tf.data.Dataset`` that yields inputs and targets.
docstr_tok: ["Return","a","TF","Dataset","over","this","dataset",".","..","warning",":",":","If","your","dataset","contains","ragged","tensors",",","this","method","errors",".","To","prevent","errors",",",":","ref",":","`","resize","your","tensors","<","transforming_tensors",">","`",".","Examples",":",">",">",">","import","ray",">",">",">","ds","=","ray.data.read_csv","(","...","``","s3",":","\/\/anonymous","@","air-example-data\/iris.csv","''","...",")",">",">",">","it","=","ds.iterator","(",")",";","it","DataIterator","(","Dataset","(","num_blocks=","...",",","num_rows=150",",","schema=","{","sepal","length","(","cm",")",":","double",",","sepal","width","(","cm",")",":","double",",","petal","length","(","cm",")",":","double",",","petal","width","(","cm",")",":","double",",","target",":","int64","}",")",")","If","your","model","accepts","a","single","tensor","as","input",",","specify","a","single","feature","column",".",">",">",">","it.to_tf","(","feature_columns=","''","sepal","length","(","cm",")","''",",","label_columns=","''","target","''",")","#","doctest",":","+SKIP","<","_OptionsDataset","element_spec=","(","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.float64",",","name='sepal","length","(","cm",")","'",")",",","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.int64",",","name='target","'",")",")",">","If","your","model","accepts","a","dictionary","as","input",",","specify","a","list","of","feature","columns",".",">",">",">","it.to_tf","(","[","``","sepal","length","(","cm",")","''",",","``","sepal","width","(","cm",")","''","]",",","``","target","''",")","#","doctest",":","+SKIP","<","_OptionsDataset","element_spec=","(","{","'sepal","length","(","cm",")","'",":","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.float64",",","name='sepal","length","(","cm",")","'",")",",","'sepal","width","(","cm",")","'",":","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.float64",",","name='sepal","width","(","cm",")","'",")","}",",","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.int64",",","name='target","'",")",")",">","If","your","dataset","contains","multiple","features","but","your","model","accepts","a","single","tensor","as","input",",","combine","features","with",":","class",":","`","~ray.data.preprocessors.Concatenator","`",".",">",">",">","from","ray.data.preprocessors","import","Concatenator",">",">",">","preprocessor","=","Concatenator","(","output_column_name=","''","features","''",",","exclude=","''","target","''",")",">",">",">","it","=","preprocessor.transform","(","ds",")",".iterator","(",")",">",">",">","it","DataIterator","(","Concatenator","+-","Dataset","(","num_blocks=","...",",","num_rows=150",",","schema=","{","sepal","length","(","cm",")",":","double",",","sepal","width","(","cm",")",":","double",",","petal","length","(","cm",")",":","double",",","petal","width","(","cm",")",":","double",",","target",":","int64","}",")",")",">",">",">","it.to_tf","(","``","features","''",",","``","target","''",")","#","doctest",":","+SKIP","<","_OptionsDataset","element_spec=","(","TensorSpec","(","shape=","(","None",",","4",")",",","dtype=tf.float64",",","name='features","'",")",",","TensorSpec","(","shape=","(","None",",",")",",","dtype=tf.int64",",","name='target","'",")",")",">","Args",":","feature_columns",":","Columns","that","correspond","to","model","inputs",".","If","this","is","a","string",",","the","input","data","is","a","tensor",".","If","this","is","a","list",",","the","input","data","is","a","``","dict","``","that","maps","column","names","to","their","tensor","representation",".","label_column",":","Columns","that","correspond","to","model","targets",".","If","this","is","a","string",",","the","target","data","is","a","tensor",".","If","this","is","a","list",",","the","target","data","is","a","``","dict","``","that","maps","column","names","to","their","tensor","representation",".","prefetch_batches",":","The","number","of","batches","to","fetch","ahead","of","the","current","batch","to","fetch",".","If","set","to","greater","than","0",",","a","separate","threadpool","will","be","used","to","fetch","the","objects","to","the","local","node",",","format","the","batches",",","and","apply","the","collate_fn",".","Defaults","to","1.","batch_size",":","Record","batch","size",".","Defaults","to","1.","drop_last",":","Set","to","True","to","drop","the","last","incomplete","batch",",","if","the","dataset","size","is","not","divisible","by","the","batch","size",".","If","False","and","the","size","of","dataset","is","not","divisible","by","the","batch","size",",","then","the","last","batch","will","be","smaller",".","Defaults","to","False",".","local_shuffle_buffer_size",":","If","non-None",",","the","data","will","be","randomly","shuffled","using","a","local","in-memory","shuffle","buffer",",","and","this","value","will","serve","as","the","minimum","number","of","rows","that","must","be","in","the","local","in-memory","shuffle","buffer","in","order","to","yield","a","batch",".","When","there","are","no","more","rows","to","add","to","the","buffer",",","the","remaining","rows","in","the","buffer","will","be","drained",".","This","buffer","size","must","be","greater","than","or","equal","to","``","batch_size","``",",","and","therefore","``","batch_size","``","must","also","be","specified","when","using","local","shuffling",".","local_shuffle_seed",":","The","seed","to","use","for","the","local","random","shuffle",".","Returns",":","A","``","tf.data.Dataset","``","that","yields","inputs","and","targets","."]
code: def to_tf(
      self,
      feature_columns: Union[str, List[str]],
      label_columns: Union[str, List[str]],
      *,
      prefetch_batches: int = 1,
      batch_size: int = 1,
      drop_last: bool = False,
      local_shuffle_buffer_size: Optional[int] = None,
      local_shuffle_seed: Optional[int] = None,
      # Deprecated.
      prefetch_blocks: int = 0,
  ) -> "tf.data.Dataset":
      """Return a TF Dataset over this dataset.
  
      .. warning::
          If your dataset contains ragged tensors, this method errors. To prevent
          errors, :ref:`resize your tensors <transforming_tensors>`.
  
      Examples:
          >>> import ray
          >>> ds = ray.data.read_csv(
          ...     "s3://anonymous@air-example-data/iris.csv"
          ... )
          >>> it = ds.iterator(); it
          DataIterator(Dataset(
             num_blocks=...,
             num_rows=150,
             schema={
                sepal length (cm): double,
                sepal width (cm): double,
                petal length (cm): double,
                petal width (cm): double,
                target: int64
             }
          ))
  
          If your model accepts a single tensor as input, specify a single feature column.
  
          >>> it.to_tf(feature_columns="sepal length (cm)", label_columns="target")  # doctest: +SKIP
          <_OptionsDataset element_spec=(TensorSpec(shape=(None,), dtype=tf.float64, name='sepal length (cm)'), TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
          If your model accepts a dictionary as input, specify a list of feature columns.
  
          >>> it.to_tf(["sepal length (cm)", "sepal width (cm)"], "target")  # doctest: +SKIP
          <_OptionsDataset element_spec=({'sepal length (cm)': TensorSpec(shape=(None,), dtype=tf.float64, name='sepal length (cm)'), 'sepal width (cm)': TensorSpec(shape=(None,), dtype=tf.float64, name='sepal width (cm)')}, TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
          If your dataset contains multiple features but your model accepts a single
          tensor as input, combine features with
          :class:`~ray.data.preprocessors.Concatenator`.
  
          >>> from ray.data.preprocessors import Concatenator
          >>> preprocessor = Concatenator(output_column_name="features", exclude="target")
          >>> it = preprocessor.transform(ds).iterator()
          >>> it
          DataIterator(Concatenator
          +- Dataset(
                num_blocks=...,
                num_rows=150,
                schema={
                   sepal length (cm): double,
                   sepal width (cm): double,
                   petal length (cm): double,
                   petal width (cm): double,
                   target: int64
                }
             ))
          >>> it.to_tf("features", "target")  # doctest: +SKIP
          <_OptionsDataset element_spec=(TensorSpec(shape=(None, 4), dtype=tf.float64, name='features'), TensorSpec(shape=(None,), dtype=tf.int64, name='target'))>
  
      Args:
          feature_columns: Columns that correspond to model inputs. If this is a
              string, the input data is a tensor. If this is a list, the input data
              is a ``dict`` that maps column names to their tensor representation.
          label_column: Columns that correspond to model targets. If this is a
              string, the target data is a tensor. If this is a list, the target data
              is a ``dict`` that maps column names to their tensor representation.
          prefetch_batches: The number of batches to fetch ahead of the current batch
              to fetch. If set to greater than 0, a separate threadpool will be used
              to fetch the objects to the local node, format the batches, and apply
              the collate_fn. Defaults to 1.
          batch_size: Record batch size. Defaults to 1.
          drop_last: Set to True to drop the last incomplete batch,
              if the dataset size is not divisible by the batch size. If
              False and the size of dataset is not divisible by the batch
              size, then the last batch will be smaller. Defaults to False.
          local_shuffle_buffer_size: If non-None, the data will be randomly shuffled
              using a local in-memory shuffle buffer, and this value will serve as the
              minimum number of rows that must be in the local in-memory shuffle
              buffer in order to yield a batch. When there are no more rows to add to
              the buffer, the remaining rows in the buffer will be drained. This
              buffer size must be greater than or equal to ``batch_size``, and
              therefore ``batch_size`` must also be specified when using local
              shuffling.
          local_shuffle_seed: The seed to use for the local random shuffle.
  
      Returns:
          A ``tf.data.Dataset`` that yields inputs and targets.
      """  # noqa: E501
  
      from ray.air._internal.tensorflow_utils import (
          convert_ndarray_to_tf_tensor,
          get_type_spec,
      )
  
      try:
          import tensorflow as tf
      except ImportError:
          raise ValueError("tensorflow must be installed!")
  
      schema = self.schema()
      valid_columns = schema.names
  
      def validate_column(column: str) -> None:
          if column not in valid_columns:
              raise ValueError(
                  f"You specified '{column}' in `feature_columns` or "
                  f"`label_columns`, but there's no column named '{column}' in the "
                  f"dataset. Valid column names are: {valid_columns}."
              )
  
      def validate_columns(columns: Union[str, List]) -> None:
          if isinstance(columns, list):
              for column in columns:
                  validate_column(column)
          else:
              validate_column(columns)
  
      validate_columns(feature_columns)
      validate_columns(label_columns)
  
      def convert_batch_to_tensors(
          batch: Dict[str, np.ndarray],
          *,
          columns: Union[str, List[str]],
          type_spec: Union[tf.TypeSpec, Dict[str, tf.TypeSpec]],
      ) -> Union[tf.Tensor, Dict[str, tf.Tensor]]:
          if isinstance(columns, str):
              return convert_ndarray_to_tf_tensor(
                  batch[columns], type_spec=type_spec
              )
          return {
              column: convert_ndarray_to_tf_tensor(
                  batch[column], type_spec=type_spec[column]
              )
              for column in columns
          }
  
      def generator():
          for batch in self.iter_batches(
              prefetch_batches=prefetch_batches,
              prefetch_blocks=prefetch_blocks,
              batch_size=batch_size,
              drop_last=drop_last,
              local_shuffle_buffer_size=local_shuffle_buffer_size,
              local_shuffle_seed=local_shuffle_seed,
          ):
              assert isinstance(batch, dict)
              features = convert_batch_to_tensors(
                  batch,
                  columns=feature_columns,
                  type_spec=feature_type_spec,
              )
              labels = convert_batch_to_tensors(
                  batch,
                  columns=label_columns,
                  type_spec=label_type_spec,
              )
              yield features, labels
  
      feature_type_spec = get_type_spec(schema, columns=feature_columns)
      label_type_spec = get_type_spec(schema, columns=label_columns)
      output_signature = (feature_type_spec, label_type_spec)
  
      dataset = tf.data.Dataset.from_generator(
          generator, output_signature=output_signature
      )
  
      options = tf.data.Options()
      options.experimental_distribute.auto_shard_policy = (
          tf.data.experimental.AutoShardPolicy.OFF
      )
      return dataset.with_options(options)
  
imports: import abc
  import time
  from typing import TYPE_CHECKING, Any, Callable, Dict, Iterable, Iterator, List, Optional, Tuple, TypeVar, Union
  import numpy
  from ray.data._internal.block_batching.iter_batches import iter_batches
  from ray.data._internal.stats import DatasetStats, StatsManager
  from ray.data.block import Block, BlockAccessor, BlockMetadata, DataBatch, _apply_strict_mode_batch_format
  from ray.types import ObjectRef
  from ray.util.annotations import PublicAPI
label_desc: Use image node_id 10 for calling the DataIterator obj's underlying member method code with example usage: obj.to_tf(feature_columns, label_columns) and returns: dataset, unknown, convert_ndarray_to_tf_tensor