ProvidersMoreJavelin AI GatewayOn this pageJavelin AI GatewayThe Javelin AI Gateway service is a high-performance, enterprise grade API Gateway for AI applications.
It is designed to streamline the usage and access of various large language model (LLM) providers,
such as OpenAI, Cohere, Anthropic and custom large language models within an organization by incorporating
robust access security for all interactions with LLMs. Javelin offers a high-level interface that simplifies the interaction with LLMs by providing a unified endpoint
to handle specific LLM related requests. See the Javelin AI Gateway documentation for more details.
Javelin Python SDK is an easy to use client library meant to be embedded into AI ApplicationsInstallation and Setup​Install javelin_sdk to interact with Javelin AI Gateway:pip install 'javelin_sdk'Set the Javelin's API key as an environment variable:export JAVELIN_API_KEY=...Completions Example​from langchain.chains import LLMChainfrom langchain.llms import JavelinAIGatewayfrom langchain.prompts import PromptTemplateroute_completions = "eng_dept03"gateway = JavelinAIGateway(    gateway_uri="http://localhost:8000",    route=route_completions,    model_name="text-davinci-003",)llmchain = LLMChain(llm=gateway, prompt=prompt)result = llmchain.run("podcast player")print(result)Embeddings Example​from langchain.embeddings import JavelinAIGatewayEmbeddingsfrom langchain.embeddings.openai import OpenAIEmbeddingsembeddings = JavelinAIGatewayEmbeddings(    gateway_uri="http://localhost:8000",    route="embeddings",)print(embeddings.embed_query("hello"))print(embeddings.embed_documents(["hello"]))Chat Example​from langchain.chat_models import ChatJavelinAIGatewayfrom langchain.schema import HumanMessage, SystemMessagemessages = [    SystemMessage(        content="You are a helpful assistant that translates English to French."    ),    HumanMessage(        content="Artificial Intelligence has the power to transform humanity and make the world a better place"    ),]chat = ChatJavelinAIGateway(    gateway_uri="http://localhost:8000",    route="mychatbot_route",    model_name="gpt-3.5-turbo"    params={        "temperature": 0.1    })print(chat(messages))PreviousInfinoNextJinaInstallation and SetupCompletions ExampleEmbeddings ExampleChat Example