class_docstr: A reference to data persisted as a directory in local or remote storage.
  
  Access the checkpoint contents locally using ``checkpoint.to_directory()``
  or ``checkpoint.as_directory``.
  
  Example creating a checkpoint using ``Checkpoint.from_directory``:
  
      >>> from ray.train import Checkpoint
      >>> checkpoint = Checkpoint.from_directory("/tmp/example_checkpoint_dir")
      >>> checkpoint.filesystem  # doctest: +ELLIPSIS
      <pyarrow._fs.LocalFileSystem object...
      >>> checkpoint.path
      '/tmp/example_checkpoint_dir'
  
  Example creating a checkpoint from a remote URI:
  
      >>> checkpoint = Checkpoint("s3://bucket/path/to/checkpoint")
      >>> checkpoint.filesystem  # doctest: +ELLIPSIS
      <pyarrow._s3fs.S3FileSystem object...
      >>> checkpoint.path
      'bucket/path/to/checkpoint'
  
  Example creating a checkpoint with a custom filesystem:
  
      >>> checkpoint = Checkpoint(
      ...     path="bucket/path/to/checkpoint",
      ...     filesystem=pyarrow.fs.S3FileSystem(),
      ... )
      >>> checkpoint.filesystem  # doctest: +ELLIPSIS
      <pyarrow._s3fs.S3FileSystem object...
      >>> checkpoint.path
      'bucket/path/to/checkpoint'
  
  Attributes:
      path: A path on the filesystem containing the checkpoint contents.
      filesystem: PyArrow FileSystem that can be used to access data at the `path`.
class_docstr_tok: ["A","reference","to","data","persisted","as","a","directory","in","local","or","remote","storage",".","Access","the","checkpoint","contents","locally","using","``","checkpoint.to_directory","(",")","``","or","``","checkpoint.as_directory","``",".","Example","creating","a","checkpoint","using","``","Checkpoint.from_directory","``",":",">",">",">","from","ray.train","import","Checkpoint",">",">",">","checkpoint","=","Checkpoint.from_directory","(","``","\/tmp\/example_checkpoint_dir","''",")",">",">",">","checkpoint.filesystem","#","doctest",":","+ELLIPSIS","<","pyarrow._fs.LocalFileSystem","object","...",">",">",">","checkpoint.path","'\/tmp\/example_checkpoint_dir'","Example","creating","a","checkpoint","from","a","remote","URI",":",">",">",">","checkpoint","=","Checkpoint","(","``","s3",":","\/\/bucket\/path\/to\/checkpoint","''",")",">",">",">","checkpoint.filesystem","#","doctest",":","+ELLIPSIS","<","pyarrow._s3fs.S3FileSystem","object","...",">",">",">","checkpoint.path","'bucket\/path\/to\/checkpoint'","Example","creating","a","checkpoint","with","a","custom","filesystem",":",">",">",">","checkpoint","=","Checkpoint","(","...","path=","''","bucket\/path\/to\/checkpoint","''",",","...","filesystem=pyarrow.fs.S3FileSystem","(",")",",","...",")",">",">",">","checkpoint.filesystem","#","doctest",":","+ELLIPSIS","<","pyarrow._s3fs.S3FileSystem","object","...",">",">",">","checkpoint.path","'bucket\/path\/to\/checkpoint'","Attributes",":","path",":","A","path","on","the","filesystem","containing","the","checkpoint","contents",".","filesystem",":","PyArrow","FileSystem","that","can","be","used","to","access","data","at","the","`","path","`","."]
docstr: Returns checkpoint contents in a local directory as a context.
  
  This function makes checkpoint data available as a directory while avoiding
  unnecessary copies and left-over temporary data.
  
  *If the checkpoint points to a local directory*, this method just returns the
  local directory path without making a copy, and nothing will be cleaned up
  after exiting the context.
  
  *If the checkpoint points to a remote directory*, this method will download the
  checkpoint to a local temporary directory and return the path
  to the temporary directory.
  
  *If multiple processes on the same node call this method simultaneously,*
  only a single process will perform the download, while the others
  wait for the download to finish. Once the download finishes, all processes
  receive the same local (temporary) directory to read from.
  
  Once all processes have finished working with the checkpoint,
  the temporary directory is cleaned up.
  
  Users should treat the returned checkpoint directory as read-only and avoid
  changing any data within it, as it may be deleted when exiting the context.
  
  Example:
  
  .. testcode::
      :hide:
  
      from pathlib import Path
      import tempfile
  
      from ray.train import Checkpoint
  
      temp_dir = tempfile.mkdtemp()
      (Path(temp_dir) / "example.txt").write_text("example checkpoint data")
      checkpoint = Checkpoint.from_directory(temp_dir)
  
  .. testcode::
  
      with checkpoint.as_directory() as checkpoint_dir:
          # Do some read-only processing of files within checkpoint_dir
          pass
  
      # At this point, if a temporary directory was created, it will have
      # been deleted.
docstr_tok: ["Returns","checkpoint","contents","in","a","local","directory","as","a","context",".","This","function","makes","checkpoint","data","available","as","a","directory","while","avoiding","unnecessary","copies","and","left-over","temporary","data",".","*","If","the","checkpoint","points","to","a","local","directory","*",",","this","method","just","returns","the","local","directory","path","without","making","a","copy",",","and","nothing","will","be","cleaned","up","after","exiting","the","context",".","*","If","the","checkpoint","points","to","a","remote","directory","*",",","this","method","will","download","the","checkpoint","to","a","local","temporary","directory","and","return","the","path","to","the","temporary","directory",".","*","If","multiple","processes","on","the","same","node","call","this","method","simultaneously",",","*","only","a","single","process","will","perform","the","download",",","while","the","others","wait","for","the","download","to","finish",".","Once","the","download","finishes",",","all","processes","receive","the","same","local","(","temporary",")","directory","to","read","from",".","Once","all","processes","have","finished","working","with","the","checkpoint",",","the","temporary","directory","is","cleaned","up",".","Users","should","treat","the","returned","checkpoint","directory","as","read-only","and","avoid","changing","any","data","within","it",",","as","it","may","be","deleted","when","exiting","the","context",".","Example",":","..","testcode",":",":",":","hide",":","from","pathlib","import","Path","import","tempfile","from","ray.train","import","Checkpoint","temp_dir","=","tempfile.mkdtemp","(",")","(","Path","(","temp_dir",")","\/","``","example.txt","''",")",".write_text","(","``","example","checkpoint","data","''",")","checkpoint","=","Checkpoint.from_directory","(","temp_dir",")","..","testcode",":",":","with","checkpoint.as_directory","(",")","as","checkpoint_dir",":","#","Do","some","read-only","processing","of","files","within","checkpoint_dir","pass","#","At","this","point",",","if","a","temporary","directory","was","created",",","it","will","have","#","been","deleted","."]
code: def as_directory(self) -> Iterator[str]:
      """Returns checkpoint contents in a local directory as a context.
  
      This function makes checkpoint data available as a directory while avoiding
      unnecessary copies and left-over temporary data.
  
      *If the checkpoint points to a local directory*, this method just returns the
      local directory path without making a copy, and nothing will be cleaned up
      after exiting the context.
  
      *If the checkpoint points to a remote directory*, this method will download the
      checkpoint to a local temporary directory and return the path
      to the temporary directory.
  
      *If multiple processes on the same node call this method simultaneously,*
      only a single process will perform the download, while the others
      wait for the download to finish. Once the download finishes, all processes
      receive the same local (temporary) directory to read from.
  
      Once all processes have finished working with the checkpoint,
      the temporary directory is cleaned up.
  
      Users should treat the returned checkpoint directory as read-only and avoid
      changing any data within it, as it may be deleted when exiting the context.
  
      Example:
  
      .. testcode::
          :hide:
  
          from pathlib import Path
          import tempfile
  
          from ray.train import Checkpoint
  
          temp_dir = tempfile.mkdtemp()
          (Path(temp_dir) / "example.txt").write_text("example checkpoint data")
          checkpoint = Checkpoint.from_directory(temp_dir)
  
      .. testcode::
  
          with checkpoint.as_directory() as checkpoint_dir:
              # Do some read-only processing of files within checkpoint_dir
              pass
  
          # At this point, if a temporary directory was created, it will have
          # been deleted.
  
      """
      if isinstance(self.filesystem, pyarrow.fs.LocalFileSystem):
          yield self.path
      else:
          del_lock_path = _get_del_lock_path(
              self._get_temporary_checkpoint_dir()
          )
          open(del_lock_path, "a").close()
  
          temp_dir = self.to_directory()
          try:
              yield temp_dir
          finally:
              # Always cleanup the del lock after we're done with the directory.
              # This avoids leaving a lock file behind in the case of an exception
              # in the user code.
              try:
                  os.remove(del_lock_path)
              except Exception:
                  logger.warning(
                      f"Could not remove {del_lock_path} deletion file lock. "
                      f"Traceback:\n{traceback.format_exc()}"
                  )
  
              # If there are no more lock files, that means there are no more
              # readers of this directory, and we can safely delete it.
              # In the edge case (process crash before del lock file is removed),
              # we do not remove the directory at all.
              # Since it's in /tmp, this is not that big of a deal.
              # check if any lock files are remaining
              remaining_locks = _list_existing_del_locks(temp_dir)
              if not remaining_locks:
                  try:
                      # Timeout 0 means there will be only one attempt to acquire
                      # the file lock. If it cannot be acquired, a TimeoutError
                      # will be thrown.
                      with TempFileLock(temp_dir, timeout=0):
                          shutil.rmtree(temp_dir, ignore_errors=True)
                  except TimeoutError:
                      pass
  
imports: import contextlib
  import glob
  import json
  import logging
  import os
  import platform
  import shutil
  import tempfile
  import traceback
  import uuid
  from typing import Any, Dict, Iterator, List, Optional, Union
  import pyarrow.fs
  from ray.air._internal.filelock import TempFileLock
  from ray.train._internal.storage import _download_from_fs_path, _exists_at_fs_path
  from ray.util.annotations import PublicAPI
label_desc: Use image node_id 8 for calling the Checkpoint obj's underlying member method code with example usage: obj.as_directory() without return types