Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsExamplesAgent VectorDB Question Answering BenchmarkingComparing Chain OutputsData Augmented Question AnsweringEvaluating an OpenAPI ChainQuestion Answering Benchmarking: Paul Graham EssayQuestion Answering Benchmarking: State of the Union AddressQA GenerationQuestion AnsweringSQL Question Answering Benchmarking: ChinookDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationExamplesEvaluating an OpenAPI ChainOn this pageEvaluating an OpenAPI ChainThis notebook goes over ways to semantically evaluate an OpenAPI Chain, which calls an endpoint defined by the OpenAPI specification using purely natural language.from langchain.tools import OpenAPISpec, APIOperationfrom langchain.chains import OpenAPIEndpointChain, LLMChainfrom langchain.requests import Requestsfrom langchain.llms import OpenAILoad the API Chainâ€‹Load a wrapper of the spec (so we can work with it more easily). You can load from a url or from a local file.# Load and parse the OpenAPI Specspec = OpenAPISpec.from_url(    "https://www.klarna.com/us/shopping/public/openai/v0/api-docs/")# Load a single endpoint operationoperation = APIOperation.from_openapi_spec(spec, "/public/openai/v0/products", "get")verbose = False# Select any LangChain LLMllm = OpenAI(temperature=0, max_tokens=1000)# Create the endpoint chainapi_chain = OpenAPIEndpointChain.from_api_operation(    operation,    llm,    requests=Requests(),    verbose=verbose,