Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsCustom Pairwise EvaluatorPairwise Embedding DistancePairwise String ComparisonTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationComparison EvaluatorsCustom Pairwise EvaluatorOn this pageCustom Pairwise EvaluatorYou can make your own pairwise string evaluators by inheriting from PairwiseStringEvaluator class and overwriting the _evaluate_string_pairs method (and the _aevaluate_string_pairs method if you want to use the evaluator asynchronously).In this example, you will make a simple custom evaluator that just returns whether the first prediction has more whitespace tokenized 'words' than the second.You can check out the reference docs for the PairwiseStringEvaluator interface for more info.from typing import Optional, Anyfrom langchain.evaluation import PairwiseStringEvaluatorclass LengthComparisonPairwiseEvalutor(PairwiseStringEvaluator):    """    Custom evaluator to compare two strings.    """    def _evaluate_string_pairs(        self,        *,        prediction: str,        prediction_b: str,        reference: Optional[str] = None,        input: Optional[str] = None,        **kwargs: Any,    ) -> dict:        score = int(len(prediction.split()) > len(prediction_b.split()))        return {"score": score}evaluator =