!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.1/docs/integrations/platforms/)
    
    *   [Providers](/v0.1/docs/integrations/platforms/)
    *   [Anthropic](/v0.1/docs/integrations/platforms/anthropic/)
    *   [AWS](/v0.1/docs/integrations/platforms/aws/)
    *   [Google](/v0.1/docs/integrations/platforms/google/)
    *   [Microsoft](/v0.1/docs/integrations/platforms/microsoft/)
    *   [OpenAI](/v0.1/docs/integrations/platforms/openai/)
*   [Components](/v0.1/docs/integrations/components/)
    
    *   [LLMs](/v0.1/docs/integrations/llms/)
        
    *   [Chat models](/v0.1/docs/integrations/chat/)
        
        *   [Chat models](/v0.1/docs/integrations/chat/)
        *   [Alibaba Tongyi](/v0.1/docs/integrations/chat/alibaba_tongyi/)
        *   [Anthropic](/v0.1/docs/integrations/chat/anthropic/)
        *   [Anthropic Tools](/v0.1/docs/integrations/chat/anthropic_tools/)
        *   [Azure OpenAI](/v0.1/docs/integrations/chat/azure/)
        *   [Baidu Wenxin](/v0.1/docs/integrations/chat/baidu_wenxin/)
        *   [Bedrock](/v0.1/docs/integrations/chat/bedrock/)
        *   [Cloudflare Workers AI](/v0.1/docs/integrations/chat/cloudflare_workersai/)
        *   [Cohere](/v0.1/docs/integrations/chat/cohere/)
        *   [Fake LLM](/v0.1/docs/integrations/chat/fake/)
        *   [Fireworks](/v0.1/docs/integrations/chat/fireworks/)
        *   [Friendli](/v0.1/docs/integrations/chat/friendli/)
        *   [Google GenAI](/v0.1/docs/integrations/chat/google_generativeai/)
        *   [(Legacy) Google PaLM/VertexAI](/v0.1/docs/integrations/chat/google_palm/)
        *   [Google Vertex AI](/v0.1/docs/integrations/chat/google_vertex_ai/)
        *   [Groq](/v0.1/docs/integrations/chat/groq/)
        *   [Llama CPP](/v0.1/docs/integrations/chat/llama_cpp/)
        *   [Minimax](/v0.1/docs/integrations/chat/minimax/)
        *   [Mistral AI](/v0.1/docs/integrations/chat/mistral/)
        *   [NIBittensorChatModel](/v0.1/docs/integrations/chat/ni_bittensor/)
        *   [Ollama](/v0.1/docs/integrations/chat/ollama/)
        *   [Ollama Functions](/v0.1/docs/integrations/chat/ollama_functions/)
        *   [OpenAI](/v0.1/docs/integrations/chat/openai/)
        *   [PremAI](/v0.1/docs/integrations/chat/premai/)
        *   [PromptLayer OpenAI](/v0.1/docs/integrations/chat/prompt_layer_openai/)
        *   [TogetherAI](/v0.1/docs/integrations/chat/togetherai/)
        *   [WebLLM](/v0.1/docs/integrations/chat/web_llm/)
        *   [YandexGPT](/v0.1/docs/integrations/chat/yandex/)
        *   [ZhipuAI](/v0.1/docs/integrations/chat/zhipuai/)
    *   [Document loaders](/v0.1/docs/integrations/document_loaders/)
        
    *   [Document transformers](/v0.1/docs/integrations/document_transformers/)
        
    *   [Document compressors](/v0.1/docs/integrations/document_compressors/)
        
    *   [Text embedding models](/v0.1/docs/integrations/text_embedding/)
        
    *   [Vector stores](/v0.1/docs/integrations/vectorstores/)
        
    *   [Retrievers](/v0.1/docs/integrations/retrievers/)
        
    *   [Tools](/v0.1/docs/integrations/tools/)
        
    *   [Agents and toolkits](/v0.1/docs/integrations/toolkits/)
        
    *   [Chat Memory](/v0.1/docs/integrations/chat_memory/)
        
    *   [Stores](/v0.1/docs/integrations/stores/)
        

*   [](/v0.1/)
*   [Components](/v0.1/docs/integrations/components/)
*   [Chat models](/v0.1/docs/integrations/chat/)
*   TogetherAI

On this page

ChatTogetherAI
==============

Setup[​](#setup "Direct link to Setup")
---------------------------------------

1.  Create a TogetherAI account and get your API key [here](https://api.together.xyz/).
2.  Export or set your API key inline. The ChatTogetherAI class defaults to `process.env.TOGETHER_AI_API_KEY`.

    export TOGETHER_AI_API_KEY=your-api-key

You can use models provided by TogetherAI as follows:

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/community

    yarn add @langchain/community

    pnpm add @langchain/community

tip

We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.

    import { ChatTogetherAI } from "@langchain/community/chat_models/togetherai";import { HumanMessage } from "@langchain/core/messages";const model = new ChatTogetherAI({  temperature: 0.9,  // In Node.js defaults to process.env.TOGETHER_AI_API_KEY  apiKey: "YOUR-API-KEY",});console.log(await model.invoke([new HumanMessage("Hello there!")]));

#### API Reference:

*   [ChatTogetherAI](https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html) from `@langchain/community/chat_models/togetherai`
*   [HumanMessage](https://api.js.langchain.com/classes/langchain_core_messages.HumanMessage.html) from `@langchain/core/messages`

Tool calling & JSON mode[​](#tool-calling--json-mode "Direct link to Tool calling & JSON mode")
-----------------------------------------------------------------------------------------------

The TogetherAI chat supports JSON mode and calling tools.

### Tool calling[​](#tool-calling "Direct link to Tool calling")

    import { ChatTogetherAI } from "@langchain/community/chat_models/togetherai";import { ChatPromptTemplate } from "@langchain/core/prompts";import { convertToOpenAITool } from "@langchain/core/utils/function_calling";import { Calculator } from "@langchain/community/tools/calculator";// Use a pre-built toolconst calculatorTool = convertToOpenAITool(new Calculator());const modelWithCalculator = new ChatTogetherAI({  temperature: 0,  // This is the default env variable name it will look for if none is passed.  apiKey: process.env.TOGETHER_AI_API_KEY,  // Together JSON mode/tool calling only supports a select number of models  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",}).bind({  // Bind the tool to the model.  tools: [calculatorTool],  tool_choice: calculatorTool, // Specify what tool the model should use});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a super not-so-smart mathmatician."],  ["human", "Help me out, how can I add {math}?"],]);// Use LCEL to chain the prompt to the model.const response = await prompt.pipe(modelWithCalculator).invoke({  math: "2 plus 3",});console.log(JSON.stringify(response.additional_kwargs.tool_calls));/**[  {    "id": "call_f4lzeeuho939vs4dilwd7267",    "type":"function",    "function": {      "name":"calculator",      "arguments": "{\"input\":\"2 + 3\"}"    }  }] */

#### API Reference:

*   [ChatTogetherAI](https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html) from `@langchain/community/chat_models/togetherai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`
*   [convertToOpenAITool](https://api.js.langchain.com/functions/langchain_core_utils_function_calling.convertToOpenAITool.html) from `@langchain/core/utils/function_calling`
*   [Calculator](https://api.js.langchain.com/classes/langchain_community_tools_calculator.Calculator.html) from `@langchain/community/tools/calculator`

tip

See a LangSmith trace of the above example [here](https://smith.langchain.com/public/5082ea20-c2de-410f-80e2-dbdfbf4d8adb/r).

### JSON mode[​](#json-mode "Direct link to JSON mode")

To use JSON mode you must include the string "JSON" inside the prompt. Typical conventions include telling the model to use JSON, eg: `Respond to the user in JSON format`.

    import { ChatTogetherAI } from "@langchain/community/chat_models/togetherai";import { ChatPromptTemplate } from "@langchain/core/prompts";// Define a JSON schema for the responseconst responseSchema = {  type: "object",  properties: {    orderedArray: {      type: "array",      items: {        type: "number",      },    },  },  required: ["orderedArray"],};const modelWithJsonSchema = new ChatTogetherAI({  temperature: 0,  apiKey: process.env.TOGETHER_AI_API_KEY,  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",}).bind({  response_format: {    type: "json_object", // Define the response format as a JSON object    schema: responseSchema, // Pass in the schema for the model's response  },});const prompt = ChatPromptTemplate.fromMessages([  ["system", "You are a helpful assistant who responds in JSON."],  ["human", "Please list this output in order of DESC {unorderedList}."],]);// Use LCEL to chain the prompt to the model.const response = await prompt.pipe(modelWithJsonSchema).invoke({  unorderedList: "[1, 4, 2, 8]",});console.log(JSON.parse(response.content as string));/**{ orderedArray: [ 8, 4, 2, 1 ] } */

#### API Reference:

*   [ChatTogetherAI](https://api.js.langchain.com/classes/langchain_community_chat_models_togetherai.ChatTogetherAI.html) from `@langchain/community/chat_models/togetherai`
*   [ChatPromptTemplate](https://api.js.langchain.com/classes/langchain_core_prompts.ChatPromptTemplate.html) from `@langchain/core/prompts`

tip

See a LangSmith trace of the above example [here](https://smith.langchain.com/public/3864aebb-5096-4b5f-b096-e54ddd1ec3d2/r).

Behind the scenes, TogetherAI uses the OpenAI SDK and OpenAI compatible API, with some caveats:

*   Certain properties are not supported by the TogetherAI API, see [here](https://docs.together.ai/reference/chat-completions).

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

PromptLayer OpenAI

](/v0.1/docs/integrations/chat/prompt_layer_openai/)[

Next

WebLLM

](/v0.1/docs/integrations/chat/web_llm/)

*   [Setup](#setup)
*   [Tool calling & JSON mode](#tool-calling--json-mode)
    *   [Tool calling](#tool-calling)
    *   [JSON mode](#json-mode)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.