          is_timelapse: bool optional
              Indicates whether this iterator is generating time-lapse images. If ``True``, then
              certain augmentations will not be performed. Default: ``False``
  
          Yields
          ------
          dict
              The following items are contained in each `dict` yielded from this iterator:
  
              * **feed** (:class:`numpy.ndarray`) - The feed for the model. The array returned is \
              in the format (`batchsize`, `height`, `width`, `channels`). This is the :attr:`x` \
              parameter for :func:`keras.models.model.train_on_batch`.
  
              * **targets** (`list`) - A list of 4-dimensional :class:`numpy.ndarray` objects in \
              the order and size of each output of the model as defined in \
              :attr:`model_output_shapes`. the format of these arrays will be (`batchsize`, \
              `height`, `width`, `3`). This is the :attr:`y` parameter for \
              :func:`keras.models.model.train_on_batch` **NB:** masks are not included in the \
              `targets` list. If required for feeding into the Keras model, they will need to be \
              added to this list in :mod:`plugins.train.trainer._base` from the `masks` key.
  
              * **masks** (:class:`numpy.ndarray`) - A 4-dimensional array containing the target \
              masks in the format (`batchsize`, `height`, `width`, `1`).
  
              * **samples** (:class:`numpy.ndarray`) - A 4-dimensional array containing the samples \
              for feeding to the model's predict function for generating preview and time-lapse \
              samples. The array will be in the format (`batchsize`, `height`, `width`, \
              `channels`). **NB:** This item will only exist in the `dict` if :attr:`is_preview` \
              or :attr:`is_timelapse` is ``True``
          """
          logger.debug("Queue batches: (image_count: %s, batchsize: %s, side: '%s', do_shuffle: %s, "
                       "is_preview, %s, is_timelapse: %s)", len(images), batchsize, side, do_shuffle,
                       is_preview, is_timelapse)
          self._batchsize = batchsize
 
</s> remove         grids = np.mgrid[0:p_mx:complex(self._training_size), 0:p_mx:complex(self._training_size)]

        self._constants = dict(clahe_base_contrast=clahe_base_contrast,
                               tgt_slices=tgt_slices,
                               warp_mapx=warp_mapx,
                               warp_mapy=warp_mapy,
                               warp_pad=warp_pad,
                               warp_slices=warp_slices,
                               warp_lm_edge_anchors=edge_anchors,
                               warp_lm_grids=grids)
        self.initialized = True
        logger.debug("Initialized constants: %s", {k: str(v) if isinstance(v, np.ndarray) else v
                                                   for k, v in self._constants.items()})

    # <<< TARGET IMAGES >>> #
    def get_targets(self, batch):
        """ Returns the target images, and masks, if required.

        Parameters
        ----------
        batch: :class:`numpy.ndarray`
            This should be a 4+-dimensional array of training images in the format (`batchsize`,
            `height`, `width`, `channels`). Targets should be requested after performing image
            transformations but prior to performing warps.

            The 4th channel should be the mask. Any channels above the 4th should be any additional
            masks that are requested.

        Returns
        -------
        dict
            The following keys will be within the returned dictionary:

            * **targets** (`list`) - A list of 4-dimensional :class:`numpy.ndarray` s in the \
            order and size of each output of the model as defined in :attr:`output_shapes`. The \
            format of these arrays will be (`batchsize`, `height`, `width`, `3`). **NB:** \
            masks are not included in the `targets` list. If masks are to be included in the \
            output they will be returned as their own item from the `masks` key.

            * **masks** (:class:`numpy.ndarray`) - A 4-dimensional array containing the target \
            masks in the format (`batchsize`, `height`, `width`, `1`).
        """
        logger.trace("Compiling targets: batch shape: %s", batch.shape)
        slices = self._constants["tgt_slices"]
        target_batch = [np.array([cv2.resize(image[slices, slices, :],
                                             (size, size),
                                             cv2.INTER_AREA)
                                  for image in batch], dtype='float32') / 255.
                        for size in self._output_sizes]
        logger.trace("Target image shapes: %s",
                     [tgt_images.shape for tgt_images in target_batch])

        retval = self._separate_target_mask(target_batch)
        logger.trace("Final targets: %s",
                     {k: v.shape if isinstance(v, np.ndarray) else [img.shape for img in v]
                      for k, v in retval.items()})
        return retval

    @staticmethod
    def _separate_target_mask(target_batch):
        """ Return the batch and the batch of final masks

        Parameters
        ----------
        target_batch: list
            List of 4 dimension :class:`numpy.ndarray` objects resized the model outputs.
            The 4th channel of the array contains the face mask, any additional channels after
            this are additional masks (e.g. eye mask and mouth mask)

        Returns
        -------
        dict:
            The targets and the masks separated into their own items. The targets are a list of
            3 channel, 4 dimensional :class:`numpy.ndarray` objects sized for each output from the
            model. The masks are a :class:`numpy.ndarray` of the final output size. Any additional
            masks(e.g. eye and mouth masks) will be collated together into a :class:`numpy.ndarray`
            of the final output size. The number of channels will be the number of additional
            masks available
        """
        logger.trace("target_batch shapes: %s", [tgt.shape for tgt in target_batch])
        retval = dict(targets=[batch[..., :3] for batch in target_batch],
                      masks=target_batch[-1][..., 3][..., None])
        if target_batch[-1].shape[-1] > 4:
            retval["additional_masks"] = target_batch[-1][..., 4:]
        logger.trace("returning: %s", {k: v.shape if isinstance(v, np.ndarray) else [tgt.shape
                                                                                     for tgt in v]
                                       for k, v in retval.items()})
 </s> add         grids = np.mgrid[0: p_mx: complex(self._processing_size),  # type: ignore
                         0: p_mx: complex(self._processing_size)]  # type: ignore
        retval = AugConstants(clahe_base_contrast=max(2, self._processing_size // 128),
                              clahe_chance=int(self._config.get("color_clahe_chance", 50)) / 100,
                              clahe_max_size=int(self._config.get("color_clahe_max_size", 4)),
                              lab_adjust=lab_adjust,
                              transform_rotation=int(self._config.get("rotation_range", 10)),
                              transform_zoom=int(self._config.get("zoom_amount", 5)) / 100,
                              transform_shift=tform_shift,
                              warp_maps=np.stack((warp_mapx, warp_mapy), axis=1),
                              warp_pad=(warp_pad, warp_pad),
                              warp_slices=slice(warp_pad // 10, -warp_pad // 10),
                              warp_lm_edge_anchors=edge_anchors,
                              warp_lm_grids=grids)
        logger.debug("Initialized constants: %s", retval) </s> remove         images: list
            A list of image paths that will be used to compile the final augmented data from.
        batchsize: int
            The batchsize for this iterator. Images will be returned in :class:`numpy.ndarray`
            objects of this size from the iterator.
        side: {'a' or 'b'}
            The side of the model that this iterator is for.
        do_shuffle: bool, optional
            Whether data should be shuffled prior to loading from disk. If true, each time the full
            list of filenames are processed, the data will be reshuffled to make sure they are not
            returned in the same order. Default: ``True``
        is_preview: bool, optional
            Indicates whether this iterator is generating preview images. If ``True`` then certain
            augmentations will not be performed. Default: ``False``
        is_timelapse: bool optional
            Indicates whether this iterator is generating time-lapse images. If ``True``, then
            certain augmentations will not be performed. Default: ``False``
 </s> add         batch: :class:`numpy.ndarray`
            This should be a 4-dimensional array of training images in the format (`batch size`,
            `height`, `width`, `channels`). Targets should be requested after performing image
            transformations but prior to performing warps. The 4th channel should be the mask.
            Any channels above the 4th should be any additional area masks (e.g. eye/mouth) that
            are required. </s> remove         Returns
        -------
        list
            List of dictionaries containing the cached metadata. The list returns in the same order
            as the filenames received
 </s> add         Yields
        ------
        feed: list
            4-dimensional array of faces to feed the training the model (:attr:`x` parameter for
            :func:`keras.models.model.train_on_batch`.). The array returned is in the format
            (`batch size`, `height`, `width`, `channels`).
        targets: list
            List of 4-dimensional :class:`numpy.ndarray` objects in the order and size of each
            output of the model. The format of these arrays will be (`batch size`, `height`,
            `width`, `x`). This is the :attr:`y` parameter for
            :func:`keras.models.model.train_on_batch`. The number of channels here will vary.
            The first 3 channels are (rgb/bgr). The 4th channel is the face mask. Any subsequent
            channels are area masks (e.g. eye/mouth masks) </s> remove         logger.debug("Queue batches: (image_count: %s, batchsize: %s, side: '%s', do_shuffle: %s, "
                     "is_preview, %s, is_timelapse: %s)", len(images), batchsize, side, do_shuffle,
                     is_preview, is_timelapse)
        self._batchsize = batchsize
        self._face_cache = _get_cache(side, images, self._config)
        self._processing = ImageAugmentation(batchsize,
                                             is_preview or is_timelapse,
                                             self._model_input_size,
                                             self._model_output_shapes,
                                             self._coverage_ratio,
                                             self._config)

        if self._warp_to_landmarks and not self._face_cache.partially_loaded:
            self._face_cache.pre_fill(images, side)

        args = (images, side, do_shuffle, batchsize)
        batcher = BackgroundGenerator(self._minibatch, thread_count=2, args=args)
        return batcher.iterator()

    # << INTERNAL METHODS >> #
    def _validate_samples(self, data):
        """ Ensures that the total number of images within :attr:`images` is greater or equal to
        the selected :attr:`batchsize`. Raises an exception if this is not the case. """
        length = len(data)
        msg = ("Number of images is lower than batch-size (Note that too few "
               "images may lead to bad training). # images: {}, "
               "batch-size: {}".format(length, self._batchsize))
        try:
            assert length >= self._batchsize, msg
        except AssertionError as err:
            msg += ("\nYou should increase the number of images in your training set or lower "
                    "your batch-size.")
            raise FaceswapError(msg) from err

    def _minibatch(self, images, side, do_shuffle, batchsize):
        """ A generator function that yields the augmented, target and sample images.
        see :func:`minibatch_ab` for more details on the output. """
        logger.debug("Loading minibatch generator: (image_count: %s, side: '%s', do_shuffle: %s)",
                     len(images), side, do_shuffle)
        self._validate_samples(images)

        def _img_iter(imgs):
            while True:
                if do_shuffle:
                    shuffle(imgs)
                for img in imgs:
                    yield img

        img_iter = _img_iter(images)
        while True:
            img_paths = [next(img_iter) for _ in range(batchsize)]
            yield self._process_batch(img_paths, side)

        logger.debug("Finished minibatch generator: (side: '%s')", side)
 </s> add         logger.trace("Compiling targets: batch shape: %s", batch.shape)  # type: ignore
        if len(self._output_sizes) == 1 and self._output_sizes[0] == self._process_size:
            # Rolling buffer here makes next to no difference, so just create array on the fly
            retval = [self._to_float32(batch)]
        else:
            retval = [self._to_float32(np.array([cv2.resize(image, (size, size), cv2.INTER_AREA)
                                                 for image in batch]))
                      for size in self._output_sizes]
        logger.trace("Processed targets: %s", [t.shape for t in retval])  # type: ignore
        return retval </s> remove         # Get Landmarks prior to manipulating the image
        if self._warp_to_landmarks:
            batch_dst_pts = self._get_closest_match(filenames, side, landmarks)
            warp_kwargs = dict(batch_src_points=landmarks, batch_dst_points=batch_dst_pts)
        else:
            warp_kwargs = dict()
 </s> add         Returns
        -------
        feed: list
            4-dimensional array of faces to feed the training the model (:attr:`x` parameter for
            :func:`keras.models.model.train_on_batch`.). The array returned is in the format
            (`batch size`, `height`, `width`, `channels`).
        targets: list
            List of 4-dimensional :class:`numpy.ndarray` objects in the order and size of each
            output of the model. The format of these arrays will be (`batch size`, `height`,
            `width`, `x`). This is the :attr:`y` parameter for
            :func:`keras.models.model.train_on_batch`. The number of channels here will vary.
            The first 3 channels are (rgb/bgr). The 4th channel is the face mask. Any subsequent
            channels are area masks (e.g. eye/mouth masks)
        """
        logger.trace("Process training: (side: '%s', filenames: '%s', images: %s, "  # type:ignore
                     "batch: %s, detected_faces: %s)", self._side, filenames, images.shape,
                     batch.shape, len(detected_faces))
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
          is_timelapse: bool optional
              Indicates whether this iterator is generating time-lapse images. If ``True``, then
              certain augmentations will not be performed. Default: ``False``
  
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
        Returns
        -------
        list
            List of 4-dimensional target images, at all model output sizes, with masks compiled
            into channels 4+ for each output size
          """
          logger.debug("Queue batches: (image_count: %s, batchsize: %s, side: '%s', do_shuffle: %s, "
                       "is_preview, %s, is_timelapse: %s)", len(images), batchsize, side, do_shuffle,
                       is_preview, is_timelapse)
          self._batchsize = batchsize
keep keep keep keep replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace replace keep keep keep keep keep
 Data Augmentation update (#1263)

- lib.detected_face
    - Subclass Masks for Landmark based masks
    - Add training mask propery + methods to DetectedFace
  - lib.training_training
    - subclass TrainingDataGenerator for training and preview data
    - Split cache into own module
    - Reduce thread count to 1 to prevent image corruption + data re-use
    - Process on largest model input/output size rather than stored image size
    - Size and crop masks during caching stage
    - Implement ring buffer for data flow
    - Fix preview reload bug
  - augmentation
    - typing
    - switch color aug order
    - better initialization
    - Fix warp + landmark warp to correctly apply at different image scales
    - Slightly improved warp caching
    - Don't store whether image is_preview. Handle all data as training images implicitly
  - plugins.trainer: Typing and fixes to work with trainingdata refactor 
lib/training/generator.py