algo/transformers/run_model.py
def eval_model(self, eval_df, multi_label=False, output_dir=None, verbose=True, silent=False, **kwargs):
        """
        Evaluates the model on eval_df. Saves results to output_dir.

        Args:
            eval_df: Pandas Dataframe containing at least two columns. If the Dataframe has a header, it should contain a 'text' and a 'labels' column. If no header is present,
            the Dataframe should contain at least two columns, with the first column containing the text, and the second column containing the label. The model will be evaluated on this Dataframe.
            output_dir: The directory where model files will be saved. If not given, self.args['output_dir'] will be used.
            verbose: If verbose, results will be printed to the console on completion of evaluation.
            silent: If silent, tqdm progress bars will be hidden.
            **kwargs: Additional metrics that should be used. Pass in the metrics as keyword arguments (name of metric: function to use). E.g. f1=sklearn.metrics.f1_score.
                        A metric function should take in two parameters. The first parameter will be the true labels, and the second parameter will be the predictions.

        Returns:
            result: Dictionary containing evaluation results. (Matthews correlation coefficient, tp, tn, fp, fn)
            model_outputs: List of model outputs for each row in eval_df
            wrong_preds: List of InputExample objects corresponding to each incorrect prediction by the model
        """  # noqa: ignore flake8"

        if not output_dir:
            output_dir = self.args["output_dir"]

        self._move_model_to_device()

        result, model_outputs, wrong_preds = self.evaluate(
            eval_df, output_dir, multi_label=multi_label, verbose=verbose, silent=silent, **kwargs
        )
        self.results.update(result)

        if verbose:
            print(self.results)

        return result, model_outputs, wrong_preds

        Evaluates the model on eval_df. Saves results to output_dir.

        Args:
            eval_df: Pandas Dataframe containing at least two columns. If the Dataframe has a header, it should contain a 'text' and a 'labels' column. If no header is present,
            the Dataframe should contain at least two columns, with the first column containing the text, and the second column containing the label. The model will be evaluated on this Dataframe.
            output_dir: The directory where model files will be saved. If not given, self.args['output_dir'] will be used.
            verbose: If verbose, results will be printed to the console on completion of evaluation.
            silent: If silent, tqdm progress bars will be hidden.
            **kwargs: Additional metrics that should be used. Pass in the metrics as keyword arguments (name of metric: function to use). E.g. f1=sklearn.metrics.f1_score.
                        A metric function should take in two parameters. The first parameter will be the true labels, and the second parameter will be the predictions.

        Returns:
            result: Dictionary containing evaluation results. (Matthews correlation coefficient, tp, tn, fp, fn)
            model_outputs: List of model outputs for each row in eval_df
            wrong_preds: List of InputExample objects corresponding to each incorrect prediction by the model
        
Evaluates the model on eval_df. Saves results to output_dir.
Args:
eval_df: Pandas Dataframe containing at least two columns. If the Dataframe has a header, it should contain a 'text' and a 'labels' column. If no header is present,
the Dataframe should contain at least two columns, with the first column containing the text, and the second column containing the label. The model will be evaluated on this Dataframe.
output_dir: The directory where model files will be saved. If not given, self.args['output_dir'] will be used.
verbose: If verbose, results will be printed to the console on completion of evaluation.
silent: If silent, tqdm progress bars will be hidden.
kwargs: Additional metrics that should be used. Pass in the metrics as keyword arguments (name of metric: function to use).

Dictionary containing evaluation results. (Matthews correlation coefficient, tp, tn, fp, fn)
model_outputs: List of model outputs for each row in eval_df
wrong_preds: List of InputExample objects corresponding to each incorrect prediction by the model
Evaluates the model on eval_df . Saves results to output_dir . Args : eval_df : Pandas Dataframe containing at least two columns . If the Dataframe has a header it should contain a ' text ' and a ' labels ' column . If no header is present the Dataframe should contain at least two columns with the first column containing the text and the second column containing the label . The model will be evaluated on this Dataframe . output_dir : The directory where model files will be saved . If not given self . args [ ' output_dir ' ] will be used . verbose : If verbose results will be printed to the console on completion of evaluation . silent : If silent tqdm progress bars will be hidden . kwargs : Additional metrics that should be used . Pass in the metrics as keyword arguments ( name of metric : function to use ) . Dictionary containing evaluation results . ( Matthews correlation coefficient tp tn fp fn ) model_outputs : List of model outputs for each row in eval_df wrong_preds : List of InputExample objects corresponding to each incorrect prediction by the model
def eval_model(self, eval_df, multi_label=False, output_dir=None, verbose=True, silent=False, **kwargs):
        if not output_dir:
            output_dir = self.args["output_dir"]
        self._move_model_to_device()
        result, model_outputs, wrong_preds = self.evaluate(
            eval_df, output_dir, multi_label=multi_label, verbose=verbose, silent=silent, **kwargs
        )
        self.results.update(result)
        if verbose:
            print(self.results)
        return result, model_outputs, wrong_preds
def eval_model ( self , eval_df , multi_label = False , output_dir = None , verbose = True , silent = False , ** kwargs ) : if not output_dir : output_dir = self . args [ "output_dir" ] self . _move_model_to_device ( ) result , model_outputs , wrong_preds = self . evaluate ( eval_df , output_dir , multi_label = multi_label , verbose = verbose , silent = silent , ** kwargs ) self . results . update ( result ) if verbose : print ( self . results ) return result , model_outputs , wrong_preds
Evaluates the model on eval_df.
Evaluates the model on eval_df .
"""
        Evaluates the model on eval_df. Saves results to output_dir.

        Args:
            eval_df: Pandas Dataframe containing at least two columns. If the Dataframe has a header, it should contain a 'text' and a 'labels' column. If no header is present,
            the Dataframe should contain at least two columns, with the first column containing the text, and the second column containing the label. The model will be evaluated on this Dataframe.
            output_dir: The directory where model files will be saved. If not given, self.args['output_dir'] will be used.
            verbose: If verbose, results will be printed to the console on completion of evaluation.
            silent: If silent, tqdm progress bars will be hidden.
            **kwargs: Additional metrics that should be used. Pass in the metrics as keyword arguments (name of metric: function to use). E.g. f1=sklearn.metrics.f1_score.
                        A metric function should take in two parameters. The first parameter will be the true labels, and the second parameter will be the predictions.

        Returns:
            result: Dictionary containing evaluation results. (Matthews correlation coefficient, tp, tn, fp, fn)
            model_outputs: List of model outputs for each row in eval_df
            wrong_preds: List of InputExample objects corresponding to each incorrect prediction by the model
        """ # noqa: ignore flake8"
self
eval_df
multi_label
output_dir
verbose
silent