          to the model.
      model_output_shapes: list
          A list of tuples defining the output shapes from the model, in the order that the outputs
          are returned. The tuples should be in (`height`, `width`, `channels`) format.
      training_opts: dict
          This is a dictionary of model training options as defined in
          :mod:`plugins.train.model._base`. These options will be defined by the user from the
          provided cli options or from the model ``config.ini``. At a minimum this ``dict`` should
          contain the following keys:
  
          * **coverage_ratio** (`float`) - The ratio of the training image to be trained on. \
          Dictates how much of the image will be cropped out. E.G: a coverage ratio of 0.625 \
          will result in cropping a 160px box from a 256px image (256 * 0.625 = 160).
  
          * **augment_color** (`bool`) - ``True`` if color is to be augmented, otherwise ``False`` \
  
          * **no_flip** (`bool`) - ``True`` if the image shouldn't be randomly flipped as part of \
          augmentation, otherwise ``False``
  
          * **warp_to_landmarks** (`bool`) - ``True`` if the random warp method should warp to \
          similar landmarks from the other side, ``False`` if the standard random warp method \
          should be used. If ``True`` then the additional key ``landmarks`` must be provided.
  
          * **landmarks** (`dict`, `optional`). Required if :attr:`warp_to_landmarks` is \
          ``True``. Returning dictionary has a key of **side** (`str`) the value of which is a \
          `dict` of {**filename** (`str`): **68 point landmarks** (`numpy.ndarray`)}.
  
 
</s> remove         cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a "
        "256px image (256 * 0.625 = 160).
 </s> add         cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`) </s> remove         `dict` of {**filename** (`str`): **68 point landmarks** (`numpy.ndarray`)}.
 </s> add         `dict` of {**filename** (`str`): **68 point landmarks** (:class:`numpy.ndarray`)}. </s> remove         The configuration ``dict`` generated from :file:`config.train.ini` containing the trainer \
 </s> add         The configuration `dict` generated from :file:`config.train.ini` containing the trainer </s> remove             * **targets** (`list`) - A list of 4-dimensional ``numpy.ndarray`` s in the order \
            and size of each output of the model as defined in :attr:`output_shapes`. The \
 </s> add             * **targets** (`list`) - A list of 4-dimensional :class:`numpy.ndarray` s in the \
            order and size of each output of the model as defined in :attr:`output_shapes`. The \ </s> remove             * **targets** (`list`) - A list of 4-dimensional ``numpy.ndarray`` s in the order \
            and size of each output of the model as defined in :attr:`model_output_shapes`. the \
            format of these arrays will be (`batchsize`, `height`, `width`, `3`). This is \
            the :attr:`y` parameter for :func:`keras.models.model.train_on_batch` **NB:** \
            masks are not included in the ``targets`` list. If required for feeding into the \
            Keras model, they will need to be added to this list in \
            :mod:`plugins.train.trainer._base` from the ``masks`` key.
 </s> add             * **targets** (`list`) - A list of 4-dimensional :class:`numpy.ndarray` objects in \
            the order and size of each output of the model as defined in \
            :attr:`model_output_shapes`. the format of these arrays will be (`batchsize`, \
            `height`, `width`, `3`). This is the :attr:`y` parameter for \
            :func:`keras.models.model.train_on_batch` **NB:** masks are not included in the \
            `targets` list. If required for feeding into the Keras model, they will need to be \
            added to this list in :mod:`plugins.train.trainer._base` from the `masks` key.
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
          to the model.
      model_output_shapes: list
          A list of tuples defining the output shapes from the model, in the order that the outputs
          are returned. The tuples should be in (`height`, `width`, `channels`) format.
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
    coverage_ratio: float
        The ratio of the training image to be trained on. Dictates how much of the image will be
        cropped out. E.G: a coverage ratio of 0.625 will result in cropping a 160px box from a
        256px image (:math:`256 * 0.625 = 160`).
    augment_color: bool
        ``True`` if color is to be augmented, otherwise ``False``
    no_flip: bool
        ``True`` if the image shouldn't be randomly flipped as part of augmentation, otherwise
        ``False``
    warp_to_landmarks: bool
        ``True`` if the random warp method should warp to similar landmarks from the other side,
        ``False`` if the standard random warp method should be used. If ``True`` then
        the key `landmarks` must be provided in the alignments dictionary.
    alignments: dict
        A dictionary containing landmarks and masks if these are required for training:
  
          * **landmarks** (`dict`, `optional`). Required if :attr:`warp_to_landmarks` is \
          ``True``. Returning dictionary has a key of **side** (`str`) the value of which is a \
          `dict` of {**filename** (`str`): **68 point landmarks** (`numpy.ndarray`)}.
  
keep keep keep keep replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace keep keep keep keep keep
 Faceswap 2.0 (#1045)

* Core Updates
    - Remove lib.utils.keras_backend_quiet and replace with get_backend() where relevant
    - Document lib.gpu_stats and lib.sys_info
    - Remove call to GPUStats.is_plaidml from convert and replace with get_backend()
    - lib.gui.menu - typofix

* Update Dependencies
Bump Tensorflow Version Check

* Port extraction to tf2

* Add custom import finder for loading Keras or tf.keras depending on backend

* Add `tensorflow` to KerasFinder search path

* Basic TF2 training running

* model.initializers - docstring fix

* Fix and pass tests for tf2

* Replace Keras backend tests with faceswap backend tests

* Initial optimizers update

* Monkey patch tf.keras optimizer

* Remove custom Adam Optimizers and Memory Saving Gradients

* Remove multi-gpu option. Add Distribution to cli

* plugins.train.model._base: Add Mirror, Central and Default distribution strategies

* Update tensorboard kwargs for tf2

* Penalized Loss - Fix for TF2 and AMD

* Fix syntax for tf2.1

* requirements typo fix

* Explicit None for clipnorm if using a distribution strategy

* Fix penalized loss for distribution strategies

* Update Dlight

* typo fix

* Pin to TF2.2

* setup.py - Install tensorflow from pip if not available in Conda

* Add reduction options and set default for mirrored distribution strategy

* Explicitly use default strategy rather than nullcontext

* lib.model.backup_restore documentation

* Remove mirrored strategy reduction method and default based on OS

* Initial restructure - training

* Remove PingPong
Start model.base refactor

* Model saving and resuming enabled

* More tidying up of model.base

* Enable backup and snapshotting

* Re-enable state file
Remove loss names from state file
Fix print loss function
Set snapshot iterations correctly

* Revert original model to Keras Model structure rather than custom layer
Output full model and sub model summary
Change NNBlocks to callables rather than custom keras layers

* Apply custom Conv2D layer

* Finalize NNBlock restructure
Update Dfaker blocks

* Fix reloading model under a different distribution strategy

* Pass command line arguments through to trainer

* Remove training_opts from model and reference params directly

* Tidy up model __init__

* Re-enable tensorboard logging
Suppress "Model Not Compiled" warning

* Fix timelapse

* lib.model.nnblocks - Bugfix residual block
Port dfaker
bugfix original

* dfl-h128 ported

* DFL SAE ported

* IAE Ported

* dlight ported

* port lightweight

* realface ported

* unbalanced ported

* villain ported

* lib.cli.args - Update Batchsize + move allow_growth to config

* Remove output shape definition
Get image sizes per side rather than globally

* Strip mask input from encoder

* Fix learn mask and output learned mask to preview

* Trigger Allow Growth prior to setting strategy

* Fix GUI Graphing

* GUI - Display batchsize correctly + fix training graphs

* Fix penalized loss

* Enable mixed precision training

* Update analysis displayed batch to match input

* Penalized Loss - Multi-GPU Fix

* Fix all losses for TF2

* Fix Reflect Padding

* Allow different input size for each side of the model

* Fix conv-aware initialization on reload

* Switch allow_growth order

* Move mixed_precision to cli

* Remove distrubution strategies

* Compile penalized loss sub-function into LossContainer

* Bump default save interval to 250
Generate preview on first iteration but don't save
Fix iterations to start at 1 instead of 0
Remove training deprecation warnings
Bump some scripts.train loglevels

* Add ability to refresh preview on demand on pop-up window

* Enable refresh of training preview from GUI

* Fix Convert
Debug logging in Initializers

* Fix Preview Tool

* Update Legacy TF1 weights to TF2
Catch stats error on loading stats with missing logs

* lib.gui.popup_configure - Make more responsive + document

* Multiple Outputs supported in trainer
Original Model - Mask output bugfix

* Make universal inference model for convert
Remove scaling from penalized mask loss (now handled at input to y_true)

* Fix inference model to work properly with all models

* Fix multi-scale output for convert

* Fix clipnorm issue with distribution strategies
Edit error message on OOM

* Update plaidml losses

* Add missing file

* Disable gmsd loss for plaidnl

* PlaidML - Basic training working

* clipnorm rewriting for mixed-precision

* Inference model creation bugfixes

* Remove debug code

* Bugfix: Default clipnorm to 1.0

* Remove all mask inputs from training code

* Remove mask inputs from convert

* GUI - Analysis Tab - Docstrings

* Fix rate in totals row

* lib.gui - Only update display pages if they have focus

* Save the model on first iteration

* plaidml - Fix SSIM loss with penalized loss

* tools.alignments - Remove manual and fix jobs

* GUI - Remove case formatting on help text

* gui MultiSelect custom widget - Set default values on init

* vgg_face2 - Move to plugins.extract.recognition and use plugins._base base class
cli - Add global GPU Exclude Option
tools.sort - Use global GPU Exlude option for backend
lib.model.session - Exclude all GPUs when running in CPU mode
lib.cli.launcher - Set backend to CPU mode when all GPUs excluded

* Cascade excluded devices to GPU Stats

* Explicit GPU selection for Train and Convert

* Reduce Tensorflow Min GPU Multiprocessor Count to 4

* remove compat.v1 code from extract

* Force TF to skip mixed precision compatibility check if GPUs have been filtered

* Add notes to config for non-working AMD losses

* Rasie error if forcing extract to CPU mode

* Fix loading of legace dfl-sae weights + dfl-sae typo fix

* Remove unused requirements
Update sphinx requirements
Fix broken rst file locations

* docs: lib.gui.display

* clipnorm amd condition check

* documentation - gui.display_analysis

* Documentation - gui.popup_configure

* Documentation - lib.logger

* Documentation - lib.model.initializers

* Documentation - lib.model.layers

* Documentation - lib.model.losses

* Documentation - lib.model.nn_blocks

* Documetation - lib.model.normalization

* Documentation - lib.model.session

* Documentation - lib.plaidml_stats

* Documentation: lib.training_data

* Documentation: lib.utils

* Documentation: plugins.train.model._base

* GUI Stats: prevent stats from using GPU

* Documentation - Original Model

* Documentation: plugins.model.trainer._base

* linting

* unit tests: initializers + losses

* unit tests: nn_blocks

* bugfix - Exclude gpu devices in train, not include

* Enable Exclude-Gpus in Extract

* Enable exclude gpus in tools

* Disallow multiple plugin types in a single model folder

* Automatically add exclude_gpus argument in for cpu backends

* Cpu backend fixes

* Relax optimizer test threshold

* Default Train settings - Set mask to Extended

* Update Extractor cli help text
Update to Python 3.8

* Fix FAN to run on CPU

* lib.plaidml_tools - typofix

* Linux installer - check for curl

* linux installer - typo fix 
lib/training_data.py