Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsEvaluating Custom CriteriaCustom String EvaluatorEmbedding DistanceQA CorrectnessString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationString EvaluatorsQA CorrectnessOn this pageQA CorrectnessWhen thinking about a QA system, one of the most important questions to ask is whether the final generated result is correct. The "qa" evaluator compares a question-answering model's response to a reference answer to provide this level of information. If you are able to annotate a test dataset, this evaluator will be useful.For more details, check out the reference docs for the QAEvalChain's class definition.from langchain.chat_models import ChatOpenAIfrom langchain.evaluation import load_evaluatorllm = ChatOpenAI(model="gpt-4", temperature=0)# Note: the eval_llm is optional. A gpt-4 model will be provided by default if not specifiedevaluator = load_evaluator("qa", eval_llm=llm)evaluator.evaluate_strings(    input="What's last quarter's sales numbers?",    prediction="Last quarter we sold 600,000 total units of product.",    reference="Last quarter we sold 100,000 units of product A, 210,000 units of product B, and 300,000 units of product C.",)    {'reasoning': None, 'value': 'CORRECT', 'score': 1}SQL Correctnessâ€‹You can use an LLM to check the equivalence of a SQL query against a reference SQL query using the sql prompt.from