BERT_rationale_benchmark/models/pipeline/pipeline_utils.py
def make_token_preds_epoch(classifier: nn.Module,
                           data: List[SentenceEvidence],
                           token_mapping: Dict[str, List[List[Tuple[int, int]]]],
                           batch_size: int,
                           device=None,
                           criterion: nn.Module = None,
                           tensorize_model_inputs: bool = True):
    """Predictions for more than one batch.

    Args:
        classifier: a module that looks like an AttentiveClassifier
        data: a list of elements to make predictions over. These must be SentenceEvidence objects.
        batch_size: the biggest chunk we can fit in one batch.
        device: Optional; what compute device this should run on
        criterion: Optional; a loss function
        tensorize_model_inputs: should we convert our data to tensors before passing it to the model? Useful if we have a model that performs its own tokenization
    """
    epoch_loss = 0
    epoch_soft_pred = []
    epoch_hard_pred = []
    epoch_truth = []
    batches = _grouper(data, batch_size)
    classifier.eval()
    for batch in batches:
        loss, soft_preds, hard_preds, targets = make_token_preds_batch(classifier,
                                                                       batch,
                                                                       token_mapping,
                                                                       device,
                                                                       criterion=criterion,
                                                                       tensorize_model_inputs=tensorize_model_inputs)
        if loss is not None:
            epoch_loss += loss.sum().item()
        epoch_hard_pred.extend(hard_preds)
        epoch_soft_pred.extend(soft_preds.cpu().tolist())
        epoch_truth.extend(targets)
    epoch_loss /= len(data)
    return epoch_loss, epoch_soft_pred, epoch_hard_pred, epoch_truth
Predictions for more than one batch.

    Args:
        classifier: a module that looks like an AttentiveClassifier
        data: a list of elements to make predictions over. These must be SentenceEvidence objects.
        batch_size: the biggest chunk we can fit in one batch.
        device: Optional; what compute device this should run on
        criterion: Optional; a loss function
        tensorize_model_inputs: should we convert our data to tensors before passing it to the model? Useful if we have a model that performs its own tokenization
    
Predictions for more than one batch.
Predictions for more than one batch .
def make_token_preds_epoch(classifier: nn.Module,
                           data: List[SentenceEvidence],
                           token_mapping: Dict[str, List[List[Tuple[int, int]]]],
                           batch_size: int,
                           device=None,
                           criterion: nn.Module = None,
                           tensorize_model_inputs: bool = True):
    epoch_loss = 0
    epoch_soft_pred = []
    epoch_hard_pred = []
    epoch_truth = []
    batches = _grouper(data, batch_size)
    classifier.eval()
    for batch in batches:
        loss, soft_preds, hard_preds, targets = make_token_preds_batch(classifier,
                                                                       batch,
                                                                       token_mapping,
                                                                       device,
                                                                       criterion=criterion,
                                                                       tensorize_model_inputs=tensorize_model_inputs)
        if loss is not None:
            epoch_loss += loss.sum().item()
        epoch_hard_pred.extend(hard_preds)
        epoch_soft_pred.extend(soft_preds.cpu().tolist())
        epoch_truth.extend(targets)
    epoch_loss /= len(data)
    return epoch_loss, epoch_soft_pred, epoch_hard_pred, epoch_truth
def make_token_preds_epoch ( classifier : nn . Module , data : List [ SentenceEvidence ] , token_mapping : Dict [ str , List [ List [ Tuple [ int , int ] ] ] ] , batch_size : int , device = None , criterion : nn . Module = None , tensorize_model_inputs : bool = True ) : epoch_loss = 0 epoch_soft_pred = [ ] epoch_hard_pred = [ ] epoch_truth = [ ] batches = _grouper ( data , batch_size ) classifier . eval ( ) for batch in batches : loss , soft_preds , hard_preds , targets = make_token_preds_batch ( classifier , batch , token_mapping , device , criterion = criterion , tensorize_model_inputs = tensorize_model_inputs ) if loss is not None : epoch_loss += loss . sum ( ) . item ( ) epoch_hard_pred . extend ( hard_preds ) epoch_soft_pred . extend ( soft_preds . cpu ( ) . tolist ( ) ) epoch_truth . extend ( targets ) epoch_loss /= len ( data ) return epoch_loss , epoch_soft_pred , epoch_hard_pred , epoch_truth
Predictions for more than one batch.
Predictions for more than one batch .
"""Predictions for more than one batch.

    Args:
        classifier: a module that looks like an AttentiveClassifier
        data: a list of elements to make predictions over. These must be SentenceEvidence objects.
        batch_size: the biggest chunk we can fit in one batch.
        device: Optional; what compute device this should run on
        criterion: Optional; a loss function
        tensorize_model_inputs: should we convert our data to tensors before passing it to the model? Useful if we have a model that performs its own tokenization
    """
classifier
nn.Module
a module that looks like an AttentiveClassifier
a module that looks like an AttentiveClassifier
data
List[SentenceEvidence]
a list of elements to make predictions over. These must be SentenceEvidence objects.
a list of elements to make predictions over . These must be SentenceEvidence objects .
token_mapping
Dict[str, List[List[Tuple[int, int]]]]
batch_size
int
the biggest chunk we can fit in one batch.
the biggest chunk we can fit in one batch .
device
Optional; what compute device this should run on
Optional ; what compute device this should run on
criterion
nn.Module
Optional; a loss function
Optional ; a loss function
tensorize_model_inputs
bool
should we convert our data to tensors before passing it to the model. Useful if we have a model that performs its own tokenization
should we convert our data to tensors before passing it to the model . Useful if we have a model that performs its own tokenization