!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [Tutorials](/v0.2/docs/tutorials/)
*   Conversational RAG

On this page

Conversational RAG
==================

In many Q&A applications we want to allow the user to have a back-and-forth conversation, meaning the application needs some sort of “memory” of past questions and answers, and some logic for incorporating those into its current thinking.

In this guide we focus on **adding logic for incorporating historical messages.** Further details on chat history management is [covered here](/v0.2/docs/how_to/message_history).

We will cover two approaches:

1.  Chains, in which we always execute a retrieval step;
2.  Agents, in which we give an LLM discretion over whether and how to execute a retrieval step (or multiple steps).

For the external knowledge source, we will use the same [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng from the [RAG tutorial](/v0.2/docs/tutorials/rag).

Setup[​](#setup "Direct link to Setup")
---------------------------------------

### Dependencies[​](#dependencies "Direct link to Dependencies")

We’ll use an OpenAI chat model and embeddings and a Memory vector store in this walkthrough, but everything shown here works with any [ChatModel](/v0.2/docs/concepts/#chat-models) or [LLM](/v0.2/docs/concepts#llms), [Embeddings](/v0.2/docs/concepts#embedding-models), and [VectorStore](/v0.2/docs/concepts#vectorstores) or [Retriever](/v0.2/docs/concepts#retrievers).

We’ll use the following packages:

    npm install --save langchain @langchain/openai cheerio

We need to set environment variable `OPENAI_API_KEY`:

    export OPENAI_API_KEY=YOUR_KEY

### LangSmith[​](#langsmith "Direct link to LangSmith")

Many of the applications you build with LangChain will contain multiple steps with multiple invocations of LLM calls. As these applications get more and more complex, it becomes crucial to be able to inspect what exactly is going on inside your chain or agent. The best way to do this is with [LangSmith](https://smith.langchain.com/).

Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to set your environment variables to start logging traces:

    export LANGCHAIN_TRACING_V2=trueexport LANGCHAIN_API_KEY=YOUR_KEY

### Initial setup[​](#initial-setup "Direct link to Initial setup")

    import "cheerio";import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";import { MemoryVectorStore } from "langchain/vectorstores/memory";import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai";import { pull } from "langchain/hub";import { ChatPromptTemplate } from "@langchain/core/prompts";import {  RunnableSequence,  RunnablePassthrough,} from "@langchain/core/runnables";import { StringOutputParser } from "@langchain/core/output_parsers";

    import { createStuffDocumentsChain } from "langchain/chains/combine_documents";const loader = new CheerioWebBaseLoader(  "https://lilianweng.github.io/posts/2023-06-23-agent/");const docs = await loader.load();const textSplitter = new RecursiveCharacterTextSplitter({  chunkSize: 1000,  chunkOverlap: 200,});const splits = await textSplitter.splitDocuments(docs);const vectorStore = await MemoryVectorStore.fromDocuments(  splits,  new OpenAIEmbeddings());// Retrieve and generate using the relevant snippets of the blog.const retriever = vectorStore.asRetriever();const prompt = await pull<ChatPromptTemplate>("rlm/rag-prompt");const llm = new ChatOpenAI({ model: "gpt-3.5-turbo", temperature: 0 });const ragChain = await createStuffDocumentsChain({  llm,  prompt,  outputParser: new StringOutputParser(),});

Let’s see what this prompt actually looks like:

    console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join("\n"));

    You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.Question: {question}Context: {context}Answer:

    await ragChain.invoke({  context: await retriever.invoke("What is Task Decomposition?"),  question: "What is Task Decomposition?",});

    "Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. I"... 208 more characters

Contextualizing the question[​](#contextualizing-the-question "Direct link to Contextualizing the question")
------------------------------------------------------------------------------------------------------------

First we’ll need to define a sub-chain that takes historical messages and the latest user question, and reformulates the question if it makes reference to any information in the historical information.

We’ll use a prompt that includes a `MessagesPlaceholder` variable under the name “chat\_history”. This allows us to pass in a list of Messages to the prompt using the “chat\_history” input key, and these messages will be inserted after the system message and before the human message containing the latest question.

    import {  ChatPromptTemplate,  MessagesPlaceholder,} from "@langchain/core/prompts";const contextualizeQSystemPrompt = `Given a chat history and the latest user questionwhich might reference context in the chat history, formulate a standalone questionwhich can be understood without the chat history. Do NOT answer the question,just reformulate it if needed and otherwise return it as is.`;const contextualizeQPrompt = ChatPromptTemplate.fromMessages([  ["system", contextualizeQSystemPrompt],  new MessagesPlaceholder("chat_history"),  ["human", "{question}"],]);const contextualizeQChain = contextualizeQPrompt  .pipe(llm)  .pipe(new StringOutputParser());

Using this chain we can ask follow-up questions that reference past messages and have them reformulated into standalone questions:

    import { AIMessage, HumanMessage } from "@langchain/core/messages";await contextualizeQChain.invoke({  chat_history: [    new HumanMessage("What does LLM stand for?"),    new AIMessage("Large language model"),  ],  question: "What is meant by large",});

    'What is the definition of "large" in the context of a language model?'

Chain with chat history[​](#chain-with-chat-history "Direct link to Chain with chat history")
---------------------------------------------------------------------------------------------

And now we can build our full QA chain.

Notice we add some routing functionality to only run the “condense question chain” when our chat history isn’t empty. Here we’re taking advantage of the fact that if a function in an LCEL chain returns another chain, that chain will itself be invoked.

    import {  ChatPromptTemplate,  MessagesPlaceholder,} from "@langchain/core/prompts";import {  RunnablePassthrough,  RunnableSequence,} from "@langchain/core/runnables";import { formatDocumentsAsString } from "langchain/util/document";const qaSystemPrompt = `You are an assistant for question-answering tasks.Use the following pieces of retrieved context to answer the question.If you don't know the answer, just say that you don't know.Use three sentences maximum and keep the answer concise.{context}`;const qaPrompt = ChatPromptTemplate.fromMessages([  ["system", qaSystemPrompt],  new MessagesPlaceholder("chat_history"),  ["human", "{question}"],]);const contextualizedQuestion = (input: Record<string, unknown>) => {  if ("chat_history" in input) {    return contextualizeQChain;  }  return input.question;};const ragChain = RunnableSequence.from([  RunnablePassthrough.assign({    context: (input: Record<string, unknown>) => {      if ("chat_history" in input) {        const chain = contextualizedQuestion(input);        return chain.pipe(retriever).pipe(formatDocumentsAsString);      }      return "";    },  }),  qaPrompt,  llm,]);

    let chat_history = [];const question = "What is task decomposition?";const aiMsg = await ragChain.invoke({ question, chat_history });console.log(aiMsg);chat_history = chat_history.concat(aiMsg);const secondQuestion = "What are common ways of doing it?";await ragChain.invoke({ question: secondQuestion, chat_history });

    AIMessage {  lc_serializable: true,  lc_kwargs: {    content: "Task decomposition is a technique used to break down complex tasks into smaller and more manageable "... 278 more characters,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  },  lc_namespace: [ "langchain_core", "messages" ],  content: "Task decomposition is a technique used to break down complex tasks into smaller and more manageable "... 278 more characters,  name: undefined,  additional_kwargs: { function_call: undefined, tool_calls: undefined }}

    AIMessage {  lc_serializable: true,  lc_kwargs: {    content: "Common ways of task decomposition include using prompting techniques like Chain of Thought (CoT) or "... 332 more characters,    additional_kwargs: { function_call: undefined, tool_calls: undefined }  },  lc_namespace: [ "langchain_core", "messages" ],  content: "Common ways of task decomposition include using prompting techniques like Chain of Thought (CoT) or "... 332 more characters,  name: undefined,  additional_kwargs: { function_call: undefined, tool_calls: undefined }}

See the first [LastSmith trace here](https://smith.langchain.com/public/527981c6-5018-4b68-a11a-ebcde77843e7/r) and the [second trace here](https://smith.langchain.com/public/7b97994a-ab9f-4bf3-a2e4-abb609e5610a/r)

Here we’ve gone over how to add application logic for incorporating historical outputs, but we’re still manually updating the chat history and inserting it into each input. In a real Q&A application we’ll want some way of persisting chat history and some way of automatically inserting and updating it.

For this we can use:

*   [BaseChatMessageHistory](https://v02.api.js.langchain.com/classes/langchain_core_chat_history.BaseChatMessageHistory.html): Store chat history.
*   [RunnableWithMessageHistory](/v0.2/docs/how_to/message_history/): Wrapper for an LCEL chain and a `BaseChatMessageHistory` that handles injecting chat history into inputs and updating it after each invocation.

For a detailed walkthrough of how to use these classes together to create a stateful conversational chain, head to the [How to add message history (memory)](/v0.2/docs/how_to/message_history/) LCEL page.

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

Build a Local RAG Application

](/v0.2/docs/tutorials/local_rag)[

Next

Build a Retrieval Augmented Generation (RAG) App

](/v0.2/docs/tutorials/rag)

*   [Setup](#setup)
    *   [Dependencies](#dependencies)
    *   [LangSmith](#langsmith)
    *   [Initial setup](#initial-setup)
*   [Contextualizing the question](#contextualizing-the-question)
*   [Chain with chat history](#chain-with-chat-history)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.