class_docstr: Orchestrates pretraining
  
  First collects parameter file symlinks into the given directory. Then
  calls load hooks for each of those parameter files.
  
  Arguments
  ---------
  collect_in : str or Path
      Path to directory where the parameter file symlinks are collected.
  loadables : mapping
      Mapping from loadable key to object. This connects the keys to
      the actual object instances.
  paths : mapping
      Mapping from loadable key to filepath. The last part
      of the path is treated as file name, the rest of it
      is treated as a "source" which can be either a directory
      path or a magic source like Huggingface hub ID.
      e.g. sb/asr-crdnn-libri/lm.ckpt
      -> source=sb/asr-crdnn-libri, file=lm.ckpt
      Note that when collecting, you can specify a default source,
      which is used for all loadables that don't have a path specified.
  custom_hooks : mapping
      Mapping from loadable key to parameter transfer hook function. If you
      want to use a custom loading function, specify it here.
  conditions: mapping
      An optional mapping from loadable keys to condition values,
      useful for loading certain elements only if a flag is turned on
class_docstr_tok: ["Orchestrates","pretraining","First","collects","parameter","file","symlinks","into","the","given","directory",".","Then","calls","load","hooks","for","each","of","those","parameter","files",".","Arguments","--","--","--","--","-","collect_in",":","str","or","Path","Path","to","directory","where","the","parameter","file","symlinks","are","collected",".","loadables",":","mapping","Mapping","from","loadable","key","to","object",".","This","connects","the","keys","to","the","actual","object","instances",".","paths",":","mapping","Mapping","from","loadable","key","to","filepath",".","The","last","part","of","the","path","is","treated","as","file","name",",","the","rest","of","it","is","treated","as","a","``","source","''","which","can","be","either","a","directory","path","or","a","magic","source","like","Huggingface","hub","ID",".","e.g",".","sb\/asr-crdnn-libri\/lm.ckpt","-",">","source=sb\/asr-crdnn-libri",",","file=lm.ckpt","Note","that","when","collecting",",","you","can","specify","a","default","source",",","which","is","used","for","all","loadables","that","do","n't","have","a","path","specified",".","custom_hooks",":","mapping","Mapping","from","loadable","key","to","parameter","transfer","hook","function",".","If","you","want","to","use","a","custom","loading","function",",","specify","it","here",".","conditions",":","mapping","An","optional","mapping","from","loadable","keys","to","condition","values",",","useful","for","loading","certain","elements","only","if","a","flag","is","turned","on"]
docstr: Fetches parameters from known paths with fallback default_source
  
  The actual parameter files may reside elsewhere, but this ensures a
  symlink in the self.collect_in directory. The symlink always uses the
  loadable key in the filename. This standardization makes it easier to
  orchestrate pretraining on e.g. distributed setups.
  
  Use the default_source if you have everything organized neatly into one
  location, like a Huggingface hub repo.
  
  Arguments
  ---------
  default_source : str or Path or FetchSource
      This is used for each loadable which doesn't have a path already
      specified. If the loadable has key "asr", then the file to look for is
      default_source/asr.ckpt
  internal_ddp_handling : bool
      Whether/not the function should handle DDP i.e. `run_on_main`.
      (Default: False)
  
  Returns
  -------
  dict
      Mapping from loadable key to a local path from which loadable's
      parameters can be loaded. This is not used in this class, but
      can possibly be helpful.
docstr_tok: ["Fetches","parameters","from","known","paths","with","fallback","default_source","The","actual","parameter","files","may","reside","elsewhere",",","but","this","ensures","a","symlink","in","the","self.collect_in","directory",".","The","symlink","always","uses","the","loadable","key","in","the","filename",".","This","standardization","makes","it","easier","to","orchestrate","pretraining","on","e.g",".","distributed","setups",".","Use","the","default_source","if","you","have","everything","organized","neatly","into","one","location",",","like","a","Huggingface","hub","repo",".","Arguments","--","--","--","--","-","default_source",":","str","or","Path","or","FetchSource","This","is","used","for","each","loadable","which","does","n't","have","a","path","already","specified",".","If","the","loadable","has","key","``","asr","''",",","then","the","file","to","look","for","is","default_source\/asr.ckpt","internal_ddp_handling",":","bool","Whether\/not","the","function","should","handle","DDP","i.e",".","`","run_on_main","`",".","(","Default",":","False",")","Returns","--","--","--","-","dict","Mapping","from","loadable","key","to","a","local","path","from","which","loadable's","parameters","can","be","loaded",".","This","is","not","used","in","this","class",",","but","can","possibly","be","helpful","."]
code: def collect_files(
      self,
      default_source=None,
      internal_ddp_handling=False,
  ):
      """Fetches parameters from known paths with fallback default_source
  
      The actual parameter files may reside elsewhere, but this ensures a
      symlink in the self.collect_in directory. The symlink always uses the
      loadable key in the filename. This standardization makes it easier to
      orchestrate pretraining on e.g. distributed setups.
  
      Use the default_source if you have everything organized neatly into one
      location, like a Huggingface hub repo.
  
      Arguments
      ---------
      default_source : str or Path or FetchSource
          This is used for each loadable which doesn't have a path already
          specified. If the loadable has key "asr", then the file to look for is
          default_source/asr.ckpt
      internal_ddp_handling : bool
          Whether/not the function should handle DDP i.e. `run_on_main`.
          (Default: False)
  
      Returns
      -------
      dict
          Mapping from loadable key to a local path from which loadable's
          parameters can be loaded. This is not used in this class, but
          can possibly be helpful.
      """
      logger.debug(
          f"Collecting files (or symlinks) for pretraining in {self.collect_in}."
      )
      self.collect_in.mkdir(exist_ok=True)
      loadable_paths = {}
      for name in self.loadables:
          if not self.is_loadable(name):
              continue
          save_filename = name + PARAMFILE_EXT
          if name in self.paths:
              source, filename = self.split_path(self.paths[name])
          elif default_source is not None:
              filename = save_filename
              source = default_source
          else:
              raise ValueError(
                  f"Path not specified for '{name}', "
                  "and no default_source given!"
              )
          if internal_ddp_handling:
              # path needs to be available only if it is a local source w/o symlink
              run_on_main(
                  fetch,
                  kwargs={
                      "filename": filename,
                      "source": source,
                      "overwrite": False,
                      "save_filename": save_filename,
                      "use_auth_token": False,
                      "revision": None,
                  },
              )
  
              # we need the path; regardless of rank
              path = fetch(
                  filename=filename,
                  source=source,
                  savedir=self.collect_in,
                  overwrite=False,
                  save_filename=save_filename,
                  use_auth_token=False,
                  revision=None,
              )
          else:
              # main node is the only one calling this, so path is available
              path = fetch(
                  filename=filename,
                  source=source,
                  savedir=self.collect_in,
                  overwrite=False,
                  save_filename=save_filename,
                  use_auth_token=False,
                  revision=None,
              )
          loadable_paths[name] = path
          fetch_from = None
          if isinstance(source, FetchSource):
              fetch_from, source = source
          if fetch_from is FetchFrom.LOCAL or (
              pathlib.Path(path).resolve()
              == pathlib.Path(source).resolve() / filename
          ):
              logger.info(
                  f"Set local path in self.paths[{name}] = {path}"
              )
              self.paths[name] = str(path)
              self.is_local.append(name)
      return loadable_paths
  
file_path: speechbrain/speechbrain/utils/parameter_transfer.py
imports: import logging
  import pathlib
  from speechbrain.utils.distributed import run_on_main
  from speechbrain.utils.fetching import fetch, FetchFrom, FetchSource
  from speechbrain.utils.checkpoints import DEFAULT_LOAD_HOOKS, DEFAULT_TRANSFER_HOOKS, PARAMFILE_EXT, get_default_hook
label_desc: Use image node_id 8 for calling the Pretrainer obj's underlying member method code with example usage: obj.collect_files(default_source, internal_ddp_handling) and returns: loadable_paths