class_docstr: Parquet datasource, for reading and writing Parquet files.
  
  The primary difference from ParquetBaseDatasource is that this uses
  PyArrow's `ParquetDataset` abstraction for dataset reads, and thus offers
  automatic Arrow dataset schema inference and row count collection at the
  cost of some potential performance and/or compatibility penalties.
class_docstr_tok: ["Parquet","datasource",",","for","reading","and","writing","Parquet","files",".","The","primary","difference","from","ParquetBaseDatasource","is","that","this","uses","PyArrow","'s","`","ParquetDataset","`","abstraction","for","dataset","reads",",","and","thus","offers","automatic","Arrow","dataset","schema","inference","and","row","count","collection","at","the","cost","of","some","potential","performance","and\/or","compatibility","penalties","."]
docstr: Return an estimate of the Parquet files encoding ratio.
  
  To avoid OOMs, it is safer to return an over-estimate than an underestimate.
docstr_tok: ["Return","an","estimate","of","the","Parquet","files","encoding","ratio",".","To","avoid","OOMs",",","it","is","safer","to","return","an","over-estimate","than","an","underestimate","."]
code: def _estimate_files_encoding_ratio(self) -> float:
      """Return an estimate of the Parquet files encoding ratio.
  
      To avoid OOMs, it is safer to return an over-estimate than an underestimate.
      """
      if not DataContext.get_current().decoding_size_estimation:
          return PARQUET_ENCODING_RATIO_ESTIMATE_DEFAULT
  
      # Sample a few rows from Parquet files to estimate the encoding ratio.
      # Launch tasks to sample multiple files remotely in parallel.
      # Evenly distributed to sample N rows in i-th row group in i-th file.
      # TODO(ekl/cheng) take into account column pruning.
      num_files = len(self._pq_fragments)
      num_samples = int(
          num_files * PARQUET_ENCODING_RATIO_ESTIMATE_SAMPLING_RATIO
      )
      min_num_samples = min(
          PARQUET_ENCODING_RATIO_ESTIMATE_MIN_NUM_SAMPLES, num_files
      )
      max_num_samples = min(
          PARQUET_ENCODING_RATIO_ESTIMATE_MAX_NUM_SAMPLES, num_files
      )
      num_samples = max(
          min(num_samples, max_num_samples), min_num_samples
      )
  
      # Evenly distributed to choose which file to sample, to avoid biased prediction
      # if data is skewed.
      file_samples = [
          self._pq_fragments[idx]
          for idx in np.linspace(0, num_files - 1, num_samples)
          .astype(int)
          .tolist()
      ]
  
      sample_fragment = cached_remote_fn(_sample_fragment)
      futures = []
      scheduling = self._local_scheduling or "SPREAD"
      for sample in file_samples:
          # Sample the first rows batch in i-th file.
          # Use SPREAD scheduling strategy to avoid packing many sampling tasks on
          # same machine to cause OOM issue, as sampling can be memory-intensive.
          futures.append(
              sample_fragment.options(
                  scheduling_strategy=scheduling
              ).remote(
                  self._to_batches_kwargs,
                  self._columns,
                  self._schema,
                  sample,
              )
          )
      sample_bar = ProgressBar("Parquet Files Sample", len(futures))
      sample_ratios = sample_bar.fetch_until_complete(futures)
      sample_bar.close()
      ratio = np.mean(sample_ratios)
      logger.debug(
          f"Estimated Parquet encoding ratio from sampling is {ratio}."
      )
      return max(ratio, PARQUET_ENCODING_RATIO_ESTIMATE_LOWER_BOUND)
  
imports: import logging
  from typing import TYPE_CHECKING, Any, Callable, Dict, Iterator, List, Literal, Optional, Union
  import numpy
  import ray
  import ray.cloudpickle
  from ray.data._internal.progress_bar import ProgressBar
  from ray.data._internal.remote_fn import cached_remote_fn
  from ray.data._internal.util import _check_pyarrow_version, _is_local_scheme
  from ray.data.block import Block
  from ray.data.context import DataContext
  from ray.data.datasource import Datasource
  from ray.data.datasource._default_metadata_providers import get_generic_metadata_provider
  from ray.data.datasource.datasource import ReadTask
  from ray.data.datasource.file_meta_provider import DefaultParquetMetadataProvider, ParquetMetadataProvider, _handle_read_os_error
  from ray.data.datasource.partitioning import PathPartitionFilter
  from ray.data.datasource.path_util import _has_file_extension, _resolve_paths_and_filesystem
  from ray.util.annotations import PublicAPI
label_desc: Use image node_id 4 for calling the ParquetDatasource obj's underlying member method code with example usage: obj._estimate_files_encoding_ratio() and returns: max, PARQUET_ENCODING_RATIO_ESTIMATE_DEFAULT