mmaction/models/recognizers/base_semi.py
def train_step(self, data_batch_labeled, data_batch_unlabeled, optimizer, **kwargs):
        """The iteration step during training.

        This method defines an iteration step during training, except for the
        back propagation and optimizer updating, which are done in an optimizer
        hook. Note that in some complicated cases or models, the whole process
        including back propagation and optimizer updating is also defined in
        this method, such as GAN.

        Args:
            data_batch_labeled (dict): The output of dataloader (labeled).
            data_batch_unlabeled (dict): The output of dataloader (unlabeled).
            optimizer (:obj:`torch.optim.Optimizer` | dict): The optimizer of
                runner is passed to ``train_step()``. This argument is unused
                and reserved.

        Returns:
            dict: It should contain at least 3 keys: ``loss``, ``log_vars``,
                ``num_samples``.
                ``loss`` is a tensor for back propagation, which can be a
                weighted sum of multiple losses.
                ``log_vars`` contains all the variables to be sent to the
                logger.
                ``num_samples`` indicates the batch size (when the model is
                DDP, it means the batch size on each GPU), which is used for
                averaging the logs.
        """
        imgs = data_batch_labeled['imgs']
        label = data_batch_labeled['label']

        imgs_weak = data_batch_unlabeled['imgs_weak']
        imgs_strong = data_batch_unlabeled['imgs_strong']

        # NOTE: ActorCutMix (replacing backgrounds)
        # Manually changed the ACM prob from 0.3 to 0.5 by Jinwoo, suggested by Yuliang
        if self.actor_cutmix and np.random.rand() < 0.5:
            human_mask = data_batch_unlabeled['human_mask']

            # NOTE: Only support num_clip = 1 case!
            human_mask = human_mask.permute(0, 4, 1, 2, 3).unsqueeze(1)
            # Select clips with valid ActorCutMix augmentation
            full_size = human_mask[0].numel()
            mask_size = human_mask.sum((1, 2, 3, 4, 5))
            invalid = torch.logical_or(mask_size == full_size, mask_size == 0)
            # If batch size is an odd number, then the central one is always invalid
            if imgs_strong.shape[0] % 2 != 0:
                invalid[imgs_strong.shape[0]//2] = True
            # For invalid clips, set the whole clip as foreground so that we won't mess up the label
            human_mask[invalid] = 1

            invalid = invalid.float().view(-1, 1, 1, 1, 1, 1)

            # When do ActorCutMix, no other aug
            fg = imgs_weak * human_mask
            bg = imgs_weak * (1-human_mask)
            bg = torch.flip(bg, dims=[0]) * (1-human_mask)
            temp = fg + bg

            imgs_strong = invalid * imgs_strong + (1-invalid) * temp

        else:
            human_mask = None

        if self.knowledge_distill:
            imagenet_prob = data_batch_unlabeled['imagenet_prob']
        else:
            imagenet_prob = None

        label_unlabeled = data_batch_unlabeled['label_unlabeled']

        losses = self(imgs, label, imgs_weak=imgs_weak, imgs_strong=imgs_strong, label_unlabeled=label_unlabeled, human_mask=human_mask, imagenet_prob=imagenet_prob)

        loss, log_vars = self._parse_losses(losses)

        outputs = dict(
            loss=loss,
            log_vars=log_vars,
            num_samples=len(next(iter(data_batch_labeled.values()))))

        return outputs
The iteration step during training.

        This method defines an iteration step during training, except for the
        back propagation and optimizer updating, which are done in an optimizer
        hook. Note that in some complicated cases or models, the whole process
        including back propagation and optimizer updating is also defined in
        this method, such as GAN.

        Args:
            data_batch_labeled (dict): The output of dataloader (labeled).
            data_batch_unlabeled (dict): The output of dataloader (unlabeled).
            optimizer (:obj:`torch.optim.Optimizer` | dict): The optimizer of
                runner is passed to ``train_step()``. This argument is unused
                and reserved.

        Returns:
            dict: It should contain at least 3 keys: ``loss``, ``log_vars``,
                ``num_samples``.
                ``loss`` is a tensor for back propagation, which can be a
                weighted sum of multiple losses.
                ``log_vars`` contains all the variables to be sent to the
                logger.
                ``num_samples`` indicates the batch size (when the model is
                DDP, it means the batch size on each GPU), which is used for
                averaging the logs.
        
The iteration step during training.
This method defines an iteration step during training, except for the
back propagation and optimizer updating, which are done in an optimizer
hook. Note that in some complicated cases or models, the whole process
including back propagation and optimizer updating is also defined in
this method, such as GAN.
The iteration step during training . This method defines an iteration step during training except for the back propagation and optimizer updating which are done in an optimizer hook . Note that in some complicated cases or models the whole process including back propagation and optimizer updating is also defined in this method such as GAN .
def train_step(self, data_batch_labeled, data_batch_unlabeled, optimizer, **kwargs):
        imgs = data_batch_labeled['imgs']
        label = data_batch_labeled['label']
        imgs_weak = data_batch_unlabeled['imgs_weak']
        imgs_strong = data_batch_unlabeled['imgs_strong']
        if self.actor_cutmix and np.random.rand() < 0.5:
            human_mask = data_batch_unlabeled['human_mask']
            human_mask = human_mask.permute(0, 4, 1, 2, 3).unsqueeze(1)
            full_size = human_mask[0].numel()
            mask_size = human_mask.sum((1, 2, 3, 4, 5))
            invalid = torch.logical_or(mask_size == full_size, mask_size == 0)
            if imgs_strong.shape[0] % 2 != 0:
                invalid[imgs_strong.shape[0]//2] = True
            human_mask[invalid] = 1
            invalid = invalid.float().view(-1, 1, 1, 1, 1, 1)
            fg = imgs_weak * human_mask
            bg = imgs_weak * (1-human_mask)
            bg = torch.flip(bg, dims=[0]) * (1-human_mask)
            temp = fg + bg
            imgs_strong = invalid * imgs_strong + (1-invalid) * temp
        else:
            human_mask = None
        if self.knowledge_distill:
            imagenet_prob = data_batch_unlabeled['imagenet_prob']
        else:
            imagenet_prob = None
        label_unlabeled = data_batch_unlabeled['label_unlabeled']
        losses = self(imgs, label, imgs_weak=imgs_weak, imgs_strong=imgs_strong, label_unlabeled=label_unlabeled, human_mask=human_mask, imagenet_prob=imagenet_prob)
        loss, log_vars = self._parse_losses(losses)
        outputs = dict(
            loss=loss,
            log_vars=log_vars,
            num_samples=len(next(iter(data_batch_labeled.values()))))
        return outputs
def train_step ( self , data_batch_labeled , data_batch_unlabeled , optimizer , ** kwargs ) : imgs = data_batch_labeled [ 'imgs' ] label = data_batch_labeled [ 'label' ] imgs_weak = data_batch_unlabeled [ 'imgs_weak' ] imgs_strong = data_batch_unlabeled [ 'imgs_strong' ] if self . actor_cutmix and np . random . rand ( ) < 0.5 : human_mask = data_batch_unlabeled [ 'human_mask' ] human_mask = human_mask . permute ( 0 , 4 , 1 , 2 , 3 ) . unsqueeze ( 1 ) full_size = human_mask [ 0 ] . numel ( ) mask_size = human_mask . sum ( ( 1 , 2 , 3 , 4 , 5 ) ) invalid = torch . logical_or ( mask_size == full_size , mask_size == 0 ) if imgs_strong . shape [ 0 ] % 2 != 0 : invalid [ imgs_strong . shape [ 0 ] // 2 ] = True human_mask [ invalid ] = 1 invalid = invalid . float ( ) . view ( - 1 , 1 , 1 , 1 , 1 , 1 ) fg = imgs_weak * human_mask bg = imgs_weak * ( 1 - human_mask ) bg = torch . flip ( bg , dims = [ 0 ] ) * ( 1 - human_mask ) temp = fg + bg imgs_strong = invalid * imgs_strong + ( 1 - invalid ) * temp else : human_mask = None if self . knowledge_distill : imagenet_prob = data_batch_unlabeled [ 'imagenet_prob' ] else : imagenet_prob = None label_unlabeled = data_batch_unlabeled [ 'label_unlabeled' ] losses = self ( imgs , label , imgs_weak = imgs_weak , imgs_strong = imgs_strong , label_unlabeled = label_unlabeled , human_mask = human_mask , imagenet_prob = imagenet_prob ) loss , log_vars = self . _parse_losses ( losses ) outputs = dict ( loss = loss , log_vars = log_vars , num_samples = len ( next ( iter ( data_batch_labeled . values ( ) ) ) ) ) return outputs
The iteration step during training.
The iteration step during training .
"""The iteration step during training.

        This method defines an iteration step during training, except for the
        back propagation and optimizer updating, which are done in an optimizer
        hook. Note that in some complicated cases or models, the whole process
        including back propagation and optimizer updating is also defined in
        this method, such as GAN.

        Args:
            data_batch_labeled (dict): The output of dataloader (labeled).
            data_batch_unlabeled (dict): The output of dataloader (unlabeled).
            optimizer (:obj:`torch.optim.Optimizer` | dict): The optimizer of
                runner is passed to ``train_step()``. This argument is unused
                and reserved.

        Returns:
            dict: It should contain at least 3 keys: ``loss``, ``log_vars``,
                ``num_samples``.
                ``loss`` is a tensor for back propagation, which can be a
                weighted sum of multiple losses.
                ``log_vars`` contains all the variables to be sent to the
                logger.
                ``num_samples`` indicates the batch size (when the model is
                DDP, it means the batch size on each GPU), which is used for
                averaging the logs.
        """ # NOTE: ActorCutMix (replacing backgrounds) # Manually changed the ACM prob from 0.3 to 0.5 by Jinwoo, suggested by Yuliang # NOTE: Only support num_clip = 1 case! # Select clips with valid ActorCutMix augmentation # If batch size is an odd number, then the central one is always invalid # For invalid clips, set the whole clip as foreground so that we won't mess up the label # When do ActorCutMix, no other aug
It should contain at least 3 keys: ``loss``, ``log_vars``,
``num_samples``.
``loss`` is a tensor for back propagation, which can be a
weighted sum of multiple losses.
``log_vars`` contains all the variables to be sent to the
logger.
``num_samples`` indicates the batch size (when the model is
DDP, it means the batch size on each GPU), which is used for
averaging the logs.
It should contain at least 3 keys : ` ` loss ` ` ` ` log_vars ` ` ` ` num_samples ` ` . ` ` loss ` ` is a tensor for back propagation which can be a weighted sum of multiple losses . ` ` log_vars ` ` contains all the variables to be sent to the logger . ` ` num_samples ` ` indicates the batch size ( when the model is DDP it means the batch size on each GPU ) which is used for averaging the logs .
dict
self
data_batch_labeled
The output of dataloader (labeled).
The output of dataloader ( labeled ) .
data_batch_unlabeled
The output of dataloader (unlabeled).
The output of dataloader ( unlabeled ) .
optimizer
optimizer (
`torch.optim.Optimizer` | dict): The optimizer of
runner is passed to ``train_step()``. This argument is unused
and reserved.
` torch . optim . Optimizer ` | dict ) : The optimizer of runner is passed to ` ` train_step () ` ` . This argument is unused and reserved .