!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Get started](/v0.1/docs/get_started/)
    
    *   [Introduction](/v0.1/docs/get_started/introduction/)
    *   [Installation](/v0.1/docs/get_started/installation/)
    *   [Quickstart](/v0.1/docs/get_started/quickstart/)
*   [LangChain Expression Language](/v0.1/docs/expression_language/)
    
    *   [Get started](/v0.1/docs/expression_language/get_started/)
    *   [Why use LCEL?](/v0.1/docs/expression_language/why/)
    *   [Interface](/v0.1/docs/expression_language/interface/)
    *   [Streaming](/v0.1/docs/expression_language/streaming/)
    *   [How to](/v0.1/docs/expression_language/how_to/routing/)
        
    *   [Cookbook](/v0.1/docs/expression_language/cookbook/)
        
    *   [LangChain Expression Language (LCEL)](/v0.1/docs/expression_language/)
*   [Modules](/v0.1/docs/modules/)
    
    *   [Model I/O](/v0.1/docs/modules/model_io/)
        
        *   [Quickstart](/v0.1/docs/modules/model_io/quick_start/)
        *   [Concepts](/v0.1/docs/modules/model_io/concepts/)
        *   [Prompts](/v0.1/docs/modules/model_io/prompts/)
            
        *   [LLMs](/v0.1/docs/modules/model_io/llms/)
            
        *   [Chat Models](/v0.1/docs/modules/model_io/chat/)
            
            *   [Quick Start](/v0.1/docs/modules/model_io/chat/quick_start/)
            *   [Streaming](/v0.1/docs/modules/model_io/chat/streaming/)
            *   [Caching](/v0.1/docs/modules/model_io/chat/caching/)
            *   [Custom chat models](/v0.1/docs/modules/model_io/chat/custom_chat/)
            *   [Tracking token usage](/v0.1/docs/modules/model_io/chat/token_usage_tracking/)
            *   [Cancelling requests](/v0.1/docs/modules/model_io/chat/cancelling_requests/)
            *   [Dealing with API Errors](/v0.1/docs/modules/model_io/chat/dealing_with_api_errors/)
            *   [Dealing with rate limits](/v0.1/docs/modules/model_io/chat/dealing_with_rate_limits/)
            *   [Tool/function calling](/v0.1/docs/modules/model_io/chat/function_calling/)
            *   [Response metadata](/v0.1/docs/modules/model_io/chat/response_metadata/)
            *   [Subscribing to events](/v0.1/docs/modules/model_io/chat/subscribing_events/)
            *   [Adding a timeout](/v0.1/docs/modules/model_io/chat/timeouts/)
        *   [Model I/O](/v0.1/docs/modules/model_io/)
        *   [Output Parsers](/v0.1/docs/modules/model_io/output_parsers/)
            
    *   [Retrieval](/v0.1/docs/modules/data_connection/)
        
    *   [Chains](/v0.1/docs/modules/chains/)
    *   [Agents](/v0.1/docs/modules/agents/)
        
    *   [More](/v0.1/docs/modules/memory/)
        
*   [Security](/v0.1/docs/security/)
*   [Guides](/v0.1/docs/guides/)
    
*   [Ecosystem](/v0.1/docs/ecosystem/)
    
*   [LangGraph](/v0.1/docs/langgraph/)
*   * * *
    

*   [](/v0.1/)
*   [Modules](/v0.1/docs/modules/)
*   [Model I/O](/v0.1/docs/modules/model_io/)
*   [Chat Models](/v0.1/docs/modules/model_io/chat/)
*   Streaming

On this page

Streaming
=========

Some Chat models provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.

Using `.stream()`[​](#using-stream "Direct link to using-stream")
-----------------------------------------------------------------

The easiest way to stream is to use the `.stream()` method. This returns an readable stream that you can also iterate over:

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/openai

    yarn add @langchain/openai

    pnpm add @langchain/openai

tip

We're unifying model params across all packages. We now suggest using `model` instead of `modelName`, and `apiKey` for API keys.

    import { ChatOpenAI } from "@langchain/openai";const chat = new ChatOpenAI({  maxTokens: 25,});// Pass in a human message. Also accepts a raw string, which is automatically// inferred to be a human message.const stream = await chat.stream([["human", "Tell me a joke about bears."]]);for await (const chunk of stream) {  console.log(chunk);}/*AIMessageChunk {  content: '',  additional_kwargs: {}}AIMessageChunk {  content: 'Why',  additional_kwargs: {}}AIMessageChunk {  content: ' did',  additional_kwargs: {}}AIMessageChunk {  content: ' the',  additional_kwargs: {}}AIMessageChunk {  content: ' bear',  additional_kwargs: {}}AIMessageChunk {  content: ' bring',  additional_kwargs: {}}AIMessageChunk {  content: ' a',  additional_kwargs: {}}...*/

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`

For models that do not support streaming, the entire response will be returned as a single chunk.

For convenience, you can also pipe a chat model into a [StringOutputParser](/v0.1/docs/modules/model_io/output_parsers/types/string/) to extract just the raw string values from each chunk:

    import { ChatOpenAI } from "@langchain/openai";import { StringOutputParser } from "@langchain/core/output_parsers";const parser = new StringOutputParser();const model = new ChatOpenAI({ temperature: 0 });const stream = await model.pipe(parser).stream("Hello there!");for await (const chunk of stream) {  console.log(chunk);}/*  Hello  !  How  can  I  assist  you  today  ?*/

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [StringOutputParser](https://api.js.langchain.com/classes/langchain_core_output_parsers.StringOutputParser.html) from `@langchain/core/output_parsers`

You can also do something similar to stream bytes directly (e.g. for returning a stream in an HTTP response) using the [HttpResponseOutputParser](/v0.1/docs/modules/model_io/output_parsers/types/http_response/):

    import { ChatOpenAI } from "@langchain/openai";import { HttpResponseOutputParser } from "langchain/output_parsers";const handler = async () => {  const parser = new HttpResponseOutputParser();  const model = new ChatOpenAI({ temperature: 0 });  const stream = await model.pipe(parser).stream("Hello there!");  const httpResponse = new Response(stream, {    headers: {      "Content-Type": "text/plain; charset=utf-8",    },  });  return httpResponse;};await handler();

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [HttpResponseOutputParser](https://api.js.langchain.com/classes/langchain_output_parsers.HttpResponseOutputParser.html) from `langchain/output_parsers`

Using a callback handler[​](#using-a-callback-handler "Direct link to Using a callback handler")
------------------------------------------------------------------------------------------------

You can also use a [`CallbackHandler`](https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/callbacks/base.ts) like so:

    import { ChatOpenAI } from "@langchain/openai";import { HumanMessage } from "@langchain/core/messages";const chat = new ChatOpenAI({  maxTokens: 25,  streaming: true,});const response = await chat.invoke([new HumanMessage("Tell me a joke.")], {  callbacks: [    {      handleLLMNewToken(token: string) {        console.log({ token });      },    },  ],});console.log(response);// { token: '' }// { token: '\n\n' }// { token: 'Why' }// { token: ' don' }// { token: "'t" }// { token: ' scientists' }// { token: ' trust' }// { token: ' atoms' }// { token: '?\n\n' }// { token: 'Because' }// { token: ' they' }// { token: ' make' }// { token: ' up' }// { token: ' everything' }// { token: '.' }// { token: '' }// AIMessage {//   text: "\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything."// }

#### API Reference:

*   [ChatOpenAI](https://api.js.langchain.com/classes/langchain_openai.ChatOpenAI.html) from `@langchain/openai`
*   [HumanMessage](https://api.js.langchain.com/classes/langchain_core_messages.HumanMessage.html) from `@langchain/core/messages`

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Quick Start

](/v0.1/docs/modules/model_io/chat/quick_start/)[

Next

Caching

](/v0.1/docs/modules/model_io/chat/caching/)

*   [Using `.stream()`](#using-stream)
*   [Using a callback handler](#using-a-callback-handler)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.