Comparison Evaluators | ğŸ¦œï¸�ğŸ”— Langchain





Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsCustom Pairwise EvaluatorPairwise Embedding DistancePairwise String ComparisonTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationComparison EvaluatorsComparison EvaluatorsğŸ“„ï¸� Custom Pairwise EvaluatorYou can make your own pairwise string evaluators by inheriting from PairwiseStringEvaluator class and overwriting the evaluatestringpairs method (and the aevaluatestringpairs method if you want to use the evaluator asynchronously).ğŸ“„ï¸� Pairwise Embedding DistanceOne way to measure the similarity (or dissimilarity) between two predictions on a shared or similar input is to embed the predictions and compute a vector distance between the two embeddings.[1]ğŸ“„ï¸� Pairwise String ComparisonOften you will want to compare predictions of an LLM, Chain, or Agent for a given input. The StringComparison evaluators facilitate this so you can answer questions like:PreviousString DistanceNextCustom Pairwise EvaluatorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.