!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Use cases](/v0.1/docs/use_cases/)
    
    *   [SQL](/v0.1/docs/use_cases/sql/)
        
    *   [Chatbots](/v0.1/docs/use_cases/chatbots/)
        
    *   [Extraction](/v0.1/docs/use_cases/extraction/)
        
    *   [Query Analysis](/v0.1/docs/use_cases/query_analysis/)
        
        *   [Quickstart](/v0.1/docs/use_cases/query_analysis/quickstart/)
        *   [Techniques](/v0.1/docs/use_cases/query_analysis/techniques/decomposition/)
            
            *   [Decomposition](/v0.1/docs/use_cases/query_analysis/techniques/decomposition/)
            *   [Expansion](/v0.1/docs/use_cases/query_analysis/techniques/expansion/)
            *   [Hypothetical Document Embeddings](/v0.1/docs/use_cases/query_analysis/techniques/hyde/)
            *   [Routing](/v0.1/docs/use_cases/query_analysis/techniques/routing/)
            *   [Step Back Prompting](/v0.1/docs/use_cases/query_analysis/techniques/step_back/)
            *   [Structuring](/v0.1/docs/use_cases/query_analysis/techniques/structuring/)
        *   [How-To Guides](/v0.1/docs/use_cases/query_analysis/how_to/few_shot/)
            
    *   [Q&A with RAG](/v0.1/docs/use_cases/question_answering/)
        
    *   [Tool use](/v0.1/docs/use_cases/tool_use/)
        
    *   [Interacting with APIs](/v0.1/docs/use_cases/api/)
    *   [Tabular Question Answering](/v0.1/docs/use_cases/tabular/)
    *   [Graphs](/v0.1/docs/use_cases/graph/)
        
    *   [Summarization](/v0.1/docs/use_cases/summarization/)
    *   [Agent Simulations](/v0.1/docs/use_cases/agent_simulations/)
        
    *   [Autonomous Agents](/v0.1/docs/use_cases/autonomous_agents/)
        
    *   [Code Understanding](/v0.1/docs/use_cases/code_understanding/)
    *   [Audio/Video Structured Extraction](/v0.1/docs/use_cases/media/)

*   [](/v0.1/)
*   [Use cases](/v0.1/docs/use_cases/)
*   [Query Analysis](/v0.1/docs/use_cases/query_analysis/)
*   Techniques
*   Decomposition

On this page

Decomposition
=============

When a user asks a question there is no guarantee that the relevant results can be returned with a single query. Sometimes to answer a question we need to split it into distinct sub-questions, retrieve results for each sub-question, and then answer using the cumulative context.

For example if a user asks: “How is Web Voyager different from reflection agents”, and we have one document that explains Web Voyager and one that explains reflection agents but no document that compares the two, then we’d likely get better results by retrieving for both “What is Web Voyager” and “What are reflection agents” and combining the retrieved documents than by retrieving based on the user question directly.

This process of splitting an input into multiple distinct sub-queries is what we refer to as **query decomposition**. It is also sometimes referred to as sub-query generation. In this guide we’ll walk through an example of how to do decomposition, using our example of a Q&A bot over the LangChain YouTube videos from the [Quickstart](/v0.1/docs/use_cases/query_analysis/quickstart/).

Setup[​](#setup "Direct link to Setup")
---------------------------------------

#### Install dependencies[​](#install-dependencies "Direct link to Install dependencies")

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/core zod uuid

    yarn add @langchain/core zod uuid

    pnpm add @langchain/core zod uuid

#### Set environment variables[​](#set-environment-variables "Direct link to Set environment variables")

    # Optional, use LangSmith for best-in-class observabilityLANGSMITH_API_KEY=your-api-keyLANGCHAIN_TRACING_V2=true

Query generation[​](#query-generation "Direct link to Query generation")
------------------------------------------------------------------------

To convert user questions to a list of sub questions we’ll use a LLM function-calling API, which can return multiple functions each turn:

### Pick your chat model:

*   OpenAI
*   Anthropic
*   FireworksAI
*   MistralAI

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai 

    yarn add @langchain/openai 

    pnpm add @langchain/openai 

#### Add environment variables

    OPENAI_API_KEY=your-api-key

#### Instantiate the model

    import { ChatOpenAI } from "@langchain/openai";const llm = new ChatOpenAI({  model: "gpt-3.5-turbo-0125",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/anthropic 

    yarn add @langchain/anthropic 

    pnpm add @langchain/anthropic 

#### Add environment variables

    ANTHROPIC_API_KEY=your-api-key

#### Instantiate the model

    import { ChatAnthropic } from "@langchain/anthropic";const llm = new ChatAnthropic({  model: "claude-3-sonnet-20240229",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/community 

    yarn add @langchain/community 

    pnpm add @langchain/community 

#### Add environment variables

    FIREWORKS_API_KEY=your-api-key

#### Instantiate the model

    import { ChatFireworks } from "@langchain/community/chat_models/fireworks";const llm = new ChatFireworks({  model: "accounts/fireworks/models/firefunction-v1",  temperature: 0});

#### Install dependencies

tip

See [this section for general instructions on installing integration packages](/docs/get_started/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/mistralai 

    yarn add @langchain/mistralai 

    pnpm add @langchain/mistralai 

#### Add environment variables

    MISTRAL_API_KEY=your-api-key

#### Instantiate the model

    import { ChatMistralAI } from "@langchain/mistralai";const llm = new ChatMistralAI({  model: "mistral-large-latest",  temperature: 0});

    import { z } from "zod";const subQuerySchema = z  .object({    subQuery: z.array(      z.string().describe("A very specific query against the database")    ),  })  .describe(    "Search over a database of tutorial videos about a software library"  );

    import {  ChatPromptTemplate,  MessagesPlaceholder,} from "@langchain/core/prompts";const system = `You are an expert at converting user questions into database queries.You have access to a database of tutorial videos about a software library for building LLM-powered applications.Perform query decomposition. Given a user question, break it down into distinct sub questions thatyou need to answer in order to answer the original question.If there are acronyms or words you are not familiar with, do not try to rephrase them.If the query is already well formed, do not try to decompose it further.`;const prompt = ChatPromptTemplate.fromMessages([  ["system", system],  new MessagesPlaceholder({    variableName: "examples",    optional: true,  }),  ["human", "{question}"],]);const llmWithTools = llm.withStructuredOutput(subQuerySchema, {  name: "SubQuery",});const queryAnalyzer = prompt.pipe(llmWithTools);

Let’s try it out with a simple question:

    await queryAnalyzer.invoke({ question: "how to do rag" });

    { subQuery: [ "How to do rag" ] }

Now with two slightly more involved questions:

    await queryAnalyzer.invoke({  question:    "how to use multi-modal models in a chain and turn chain into a rest api",});

    {  subQuery: [    "How to use multi-modal models in a chain",    "How to turn a chain into a REST API"  ]}

    await queryAnalyzer.invoke({  question:    "what's the difference between web voyager and reflection agents? do they use langgraph?",});

    {  subQuery: [    "Difference between Web Voyager and Reflection Agents",    "Do Web Voyager and Reflection Agents use LangGraph?"  ]}

Adding examples and tuning the prompt[​](#adding-examples-and-tuning-the-prompt "Direct link to Adding examples and tuning the prompt")
---------------------------------------------------------------------------------------------------------------------------------------

This works pretty well, but we probably want it to decompose the last question even further to separate the queries about Web Voyager and Reflection Agents. If we aren’t sure up front what types of queries will do best with our index, we can also intentionally include some redundancy in our queries, so that we return both sub queries and higher level queries.

To tune our query generation results, we can add some examples of inputs questions and gold standard output queries to our prompt. We can also try to improve our system message.

    const examples: Array<Record<string, any>> = [];

    const question = "What's chat langchain, is it a langchain template?";const query = {  query: "What's chat langchain, is it a langchain template?",  subQueries: [    "What is chat langchain",    "Is chat langchain a langchain template",  ],};examples.push({ input: question, toolCalls: [query] });

    1

    const question = "How would I use LangGraph to build an automaton";const query = {  query: "How would I use LangGraph to build an automaton",  subQueries: ["How to build automaton with LangGraph"],};examples.push({ input: question, toolCalls: [query] });

    2

    const question =  "How to build multi-agent system and stream intermediate steps from it";const query = {  query:    "How to build multi-agent system and stream intermediate steps from it",  subQueries: [    "How to build multi-agent system",    "How to stream intermediate steps",    "How to stream intermediate steps from multi-agent system",  ],};examples.push({ input: question, toolCalls: [query] });

    3

    const question =  "What's the difference between LangChain agents and LangGraph?";const query = {  query: "What's the difference between LangChain agents and LangGraph?",  subQueries: [    "What's the difference between LangChain agents and LangGraph?",    "What are LangChain agents",    "What is LangGraph",  ],};examples.push({ input: question, toolCalls: [query] });

    4

Now we need to update our prompt template and chain so that the examples are included in each prompt. Since we’re working with LLM model function-calling, we’ll need to do a bit of extra structuring to send example inputs and outputs to the model. We’ll create a `toolExampleToMessages` helper function to handle this for us:

    import { v4 as uuidV4 } from "uuid";import {  AIMessage,  BaseMessage,  HumanMessage,  SystemMessage,  ToolMessage,} from "@langchain/core/messages";const toolExampleToMessages = (  example: Record<string, any>): Array<BaseMessage> => {  const messages: Array<BaseMessage> = [    new HumanMessage({ content: example.input }),  ];  const openaiToolCalls = example.toolCalls.map((toolCall) => {    return {      id: uuidV4(),      type: "function" as const,      function: {        name: "SubQuery",        arguments: JSON.stringify(toolCall),      },    };  });  messages.push(    new AIMessage({      content: "",      additional_kwargs: { tool_calls: openaiToolCalls },    })  );  const toolOutputs =    "toolOutputs" in example      ? example.toolOutputs      : Array(openaiToolCalls.length).fill(          "This is an example of a correct usage of this tool. Make sure to continue using the tool this way."        );  toolOutputs.forEach((output, index) => {    messages.push(      new ToolMessage({        content: output,        tool_call_id: openaiToolCalls[index].id,      })    );  });  return messages;};const exampleMessages = examples.map((ex) => toolExampleToMessages(ex)).flat();

    import { MessagesPlaceholder } from "@langchain/core/prompts";import {  RunnablePassthrough,  RunnableSequence,} from "@langchain/core/runnables";const system = `You are an expert at converting user questions into database queries.You have access to a database of tutorial videos about a software library for building LLM-powered applications.Perform query decomposition. Given a user question, break it down into the most specific sub questions you canwhich will help you answer the original question. Each sub question should be about a single concept/fact/idea.If there are acronyms or words you are not familiar with, do not try to rephrase them.`;const prompt = ChatPromptTemplate.fromMessages([  ["system", system],  new MessagesPlaceholder({ variableName: "examples", optional: true }),  ["human", "{question}"],]);const queryAnalyzerWithExamples = RunnableSequence.from([  {    question: new RunnablePassthrough(),    examples: () => exampleMessages,  },  prompt,  llmWithTools,]);

    await queryAnalyzerWithExamples.invoke(  "what's the difference between web voyager and reflection agents? do they use langgraph?");

    {  query: "what's the difference between web voyager and reflection agents? do they use langgraph?",  subQueries: [    "What's the difference between web voyager and reflection agents",    "Do web voyager and reflection agents use LangGraph"  ]}

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Quickstart

](/v0.1/docs/use_cases/query_analysis/quickstart/)[

Next

Expansion

](/v0.1/docs/use_cases/query_analysis/techniques/expansion/)

*   [Setup](#setup)
*   [Query generation](#query-generation)
*   [Adding examples and tuning the prompt](#adding-examples-and-tuning-the-prompt)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.