Migrating from LLMChain

# Migrating from LLMChain

[`LLMChain`](https://python.langchain.com/v0.2/api_reference/langchain/chains/langchain.chains.llm.LLMChain.html) combined a prompt template, LLM, and output parser into a class.

Some advantages of switching to the LCEL implementation are:

- Clarity around contents and parameters. The legacy `LLMChain` contains a default output parser and other options.
- Easier streaming. `LLMChain` only supports streaming via callbacks.
- Easier access to raw message outputs if desired. `LLMChain` only exposes these via a parameter or via callback.

```python
%pip install --upgrade --quiet langchain-openai
```

```python
import os
from getpass import getpass

if "OPENAI_API_KEY" not in os.environ:
    os.environ["OPENAI_API_KEY"] = getpass()
```

## Legacy

<details open>

```python
from langchain.chains import LLMChain
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages(
    [("user", "Tell me a {adjective} joke")],
)

legacy_chain = LLMChain(llm=ChatOpenAI(), prompt=prompt)

legacy_result = legacy_chain({"adjective": "funny"})
legacy_result
```

    {'adjective': 'funny',
     'text': "Why couldn't the bicycle stand up by itself?\n\nBecause it was two tired!"}

Note that `LLMChain` by default returned a `dict` containing both the input and the output from `StrOutputParser`, so to extract the output, you need to access the `"text"` key.

```python
legacy_result["text"]
```

    "Why couldn't the bicycle stand up by itself?\n\nBecause it was two tired!"

</details>

## LCEL

<details open>

```python
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages(
    [("user", "Tell me a {adjective} joke")],
)

chain = prompt | ChatOpenAI() | StrOutputParser()

chain.invoke({"adjective": "funny"})
```

    'Why was the math book sad?\n\nBecause it had too many problems.'

If you'd like to mimic the `dict` packaging of input and output in `LLMChain`, you can use a [`RunnablePassthrough.assign`](https://python.langchain.com/v0.2/api_reference/core/runnables/langchain_core.runnables.passthrough.RunnablePassthrough.html) like:

```python
from langchain_core.runnables import RunnablePassthrough

outer_chain = RunnablePassthrough().assign(text=chain)

outer_chain.invoke({"adjective": "funny"})
```

    {'adjective': 'funny',
     'text': 'Why did the scarecrow win an award? Because he was outstanding in his field!'}

</details>

## Next steps

See [this tutorial](/docs/tutorials/llm_chain) for more detail on building with prompt templates, LLMs, and output parsers.

Check out the [LCEL conceptual docs](/docs/concepts/#langchain-expression-language-lcel) for more background information.


# How to use prompting alone (no tool calling) to do extraction

Tool calling features are not required for generating structured output from LLMs. LLMs that are able to follow prompt instructions well can be tasked with outputting information in a given format.

This approach relies on designing good prompts and then parsing the output of the LLMs to make them extract information well.

To extract data without tool-calling features:

1. Instruct the LLM to generate text following an expected format (e.g., JSON with a certain schema);
2. Use [output parsers](/docs/concepts#output-parsers) to structure the model response into a desired Python object.

First we select a LLM:

```{=mdx}
import ChatModelTabs from "@theme/ChatModelTabs";

<ChatModelTabs customVarName="model" />
```

```python
# | output: false
# | echo: false

from langchain_anthropic.chat_models import ChatAnthropic

model = ChatAnthropic(model_name="claude-3-sonnet-20240229", temperature=0)
```

:::{.callout-tip}
This tutorial is meant to be simple, but generally should really include reference examples to squeeze out performance!
:::

## Using PydanticOutputParser

The following example uses the built-in `PydanticOutputParser` to parse the output of a chat model.

```python
from typing import List, Optional

from langchain_core.output_parsers import PydanticOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field, validator

class Person(BaseModel):
    """Information about a person."""

    name: str = Field(..., description="The name of the person")
    height_in_meters: float = Field(
        ..., description="The height of the person expressed in meters."
    )

class People(BaseModel):
    """Identifying information about all people in a text."""

    people: List[Person]

# Set up a parser
parser = PydanticOutputParser(pydantic_object=People)

# Prompt
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "Answer the user query. Wrap the output in `json` tags\n{format_instructions}",
        ),
        ("human", "{query}"),
    ]
).partial(format_instructions=parser.get_format_instructions())
```

Let's take a look at what information is sent to the model

```python
query = "Anna is 23 years old and she is 6 feet tall"
```

```python
print(prompt.format_prompt(query=query).to_string())
```

    System: Answer the user query. Wrap the output in `json` tags
    The output should be formatted as a JSON instance that conforms to the JSON schema below.

    As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
    the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

    Here is the output schema:
    ```
    {"description": "Identifying information about all people in a text.", "properties": {"people": {"title": "People", "type": "array", "items": {"$ref": "#/definitions/Person"}}}, "required": ["people"], "definitions": {"Person": {"title": "Person", "description": "Information about a person.", "type": "object", "properties": {"name": {"title": "Name", "description": "The name of the person", "type": "string"}, "height_in_meters": {"title": "Height In Meters", "description": "The height of the person expressed in meters.", "type": "number"}}, "required": ["name", "height_in_meters"]}}}
    ```
    Human: Anna is 23 years old and she is 6 feet tall

Having defined our prompt, we simply chain together the prompt, model and output parser:

```python
chain = prompt | model | parser
chain.invoke({"query": query})
```

    People(people=[Person(name='Anna', height_in_meters=1.83)])

Check out the associated [Langsmith trace](https://smith.langchain.com/public/92ed52a3-92b9-45af-a663-0a9c00e5e396/r).

Note that the schema shows up in two places:

1. In the prompt, via `parser.get_format_instructions()`;
2. In the chain, to receive the formatted output and structure it into a Python object (in this case, the Pydantic object `People`).

## Custom Parsing

If desired, it's easy to create a custom prompt and parser with `LangChain` and `LCEL`.

To create a custom parser, define a function to parse the output from the model (typically an [AIMessage](https://python.langchain.com/v0.2/api_reference/core/messages/langchain_core.messages.ai.AIMessage.html)) into an object of your choice.

See below for a simple implementation of a JSON parser.

```python
import json
import re
from typing import List, Optional

from langchain_anthropic.chat_models import ChatAnthropic
from langchain_core.messages import AIMessage
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field, validator

class Person(BaseModel):
    """Information about a person."""

    name: str = Field(..., description="The name of the person")
    height_in_meters: float = Field(
        ..., description="The height of the person expressed in meters."
    )

class People(BaseModel):
    """Identifying information about all people in a text."""

    people: List[Person]

# Prompt
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "Answer the user query. Output your answer as JSON that  "
            "matches the given schema: ```json\n{schema}\n```. "
            "Make sure to wrap the answer in ```json and ``` tags",
        ),
        ("human", "{query}"),
    ]
).partial(schema=People.schema())

# Custom parser
def extract_json(message: AIMessage) -> List[dict]:
    """Extracts JSON content from a string where JSON is embedded between ```json and ``` tags.

    Parameters:
        text (str): The text containing the JSON content.

    Returns:
        list: A list of extracted JSON strings.
    """
    text = message.content
    # Define the regular expression pattern to match JSON blocks
    pattern = r"```json(.*?)```"

    # Find all non-overlapping matches of the pattern in the string
    matches = re.findall(pattern, text, re.DOTALL)

    # Return the list of matched JSON strings, stripping any leading or trailing whitespace
    try:
        return [json.loads(match.strip()) for match in matches]
    except Exception:
        raise ValueError(f"Failed to parse: {message}")
```

```python
query = "Anna is 23 years old and she is 6 feet tall"
print(prompt.format_prompt(query=query).to_string())
```

    System: Answer the user query. Output your answer as JSON that  matches the given schema: ```json
    {'title': 'People', 'description': 'Identifying information about all people in a text.', 'type': 'object', 'properties': {'people': {'title': 'People', 'type': 'array', 'items': {'$ref': '#/definitions/Person'}}}, 'required': ['people'], 'definitions': {'Person': {'title': 'Person', 'description': 'Information about a person.', 'type': 'object', 'properties': {'name': {'title': 'Name', 'description': 'The name of the person', 'type': 'string'}, 'height_in_meters': {'title': 'Height In Meters', 'description': 'The height of the person expressed in meters.', 'type': 'number'}}, 'required': ['name', 'height_in_meters']}}}
    ```. Make sure to wrap the answer in ```json and ``` tags
    Human: Anna is 23 years old and she is 6 feet tall

```python
chain = prompt | model | extract_json
chain.invoke({"query": query})
```

    [{'people': [{'name': 'Anna', 'height_in_meters': 1.83}]}]

## Other Libraries

If you're looking at extracting using a parsing approach, check out the [Kor](https://eyurtsev.github.io/kor/) library. It's written by one of the `LangChain` maintainers and it
helps to craft a prompt that takes examples into account, allows controlling formats (e.g., JSON or CSV) and expresses the schema in TypeScript. It seems to work pretty!


How to pass run time values to tools

# How to pass run time values to tools

import Prerequisites from "@theme/Prerequisites";
import Compatibility from "@theme/Compatibility";

<Prerequisites titlesAndLinks={[
  ["Chat models", "/docs/concepts/#chat-models"],
  ["LangChain Tools", "/docs/concepts/#tools"],
  ["How to create tools", "/docs/how_to/custom_tools"],
  ["How to use a model to call tools", "/docs/how_to/tool_calling"],
]} />

<Compatibility packagesAndVersions={[
  ["langchain-core", "0.2.21"],
]} />

You may need to bind values to a tool that are only known at runtime. For example, the tool logic may require using the ID of the user who made the request.

Most of the time, such values should not be controlled by the LLM. In fact, allowing the LLM to control the user ID may lead to a security risk.

Instead, the LLM should only control the parameters of the tool that are meant to be controlled by the LLM, while other parameters (such as user ID) should be fixed by the application logic.

This how-to guide shows you how to prevent the model from generating certain tool arguments and injecting them in directly at runtime.

:::info Using with LangGraph

If you're using LangGraph, please refer to [this how-to guide](https://langchain-ai.github.io/langgraph/how-tos/pass-run-time-values-to-tools/)
which shows how to create an agent that keeps track of a given user's favorite pets.
:::

We can bind them to chat models as follows:

```{=mdx}
import ChatModelTabs from "@theme/ChatModelTabs";

<ChatModelTabs
  customVarName="llm"
  fireworksParams={`model="accounts/fireworks/models/firefunction-v1", temperature=0`}
/>
```

```python
# | output: false
# | echo: false

# %pip install -qU langchain langchain_openai

import os
from getpass import getpass

from langchain_openai import ChatOpenAI

if "OPENAI_API_KEY" not in os.environ:
    os.environ["OPENAI_API_KEY"] = getpass()

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
```

## Hiding arguments from the model

We can use the InjectedToolArg annotation to mark certain parameters of our Tool, like `user_id` as being injected at runtime, meaning they shouldn't be generated by the model

```python
from typing import List

from langchain_core.tools import InjectedToolArg, tool
from typing_extensions import Annotated

user_to_pets = {}

@tool(parse_docstring=True)
def update_favorite_pets(
    pets: List[str], user_id: Annotated[str, InjectedToolArg]
) -> None:
    """Add the list of favorite pets.

    Args:
        pets: List of favorite pets to set.
        user_id: User's ID.
    """
    user_to_pets[user_id] = pets

@tool(parse_docstring=True)
def delete_favorite_pets(user_id: Annotated[str, InjectedToolArg]) -> None:
    """Delete the list of favorite pets.

    Args:
        user_id: User's ID.
    """
    if user_id in user_to_pets:
        del user_to_pets[user_id]

@tool(parse_docstring=True)
def list_favorite_pets(user_id: Annotated[str, InjectedToolArg]) -> None:
    """List favorite pets if any.

    Args:
        user_id: User's ID.
    """
    return user_to_pets.get(user_id, [])
```

If we look at the input schemas for these tools, we'll see that user_id is still listed:

```python
update_favorite_pets.get_input_schema().schema()
```

    {'title': 'update_favorite_petsSchema',
     'description': 'Add the list of favorite pets.',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}},
      'user_id': {'title': 'User Id',
       'description': "User's ID.",
       'type': 'string'}},
     'required': ['pets', 'user_id']}

But if we look at the tool call schema, which is what is passed to the model for tool-calling, user_id has been removed:

```python
update_favorite_pets.tool_call_schema.schema()
```

    {'title': 'update_favorite_pets',
     'description': 'Add the list of favorite pets.',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}}},
     'required': ['pets']}

So when we invoke our tool, we need to pass in user_id:

```python
user_id = "123"
update_favorite_pets.invoke({"pets": ["lizard", "dog"], "user_id": user_id})
print(user_to_pets)
print(list_favorite_pets.invoke({"user_id": user_id}))
```

    {'123': ['lizard', 'dog']}
    ['lizard', 'dog']

But when the model calls the tool, no user_id argument will be generated:

```python
tools = [
    update_favorite_pets,
    delete_favorite_pets,
    list_favorite_pets,
]
llm_with_tools = llm.bind_tools(tools)
ai_msg = llm_with_tools.invoke("my favorite animals are cats and parrots")
ai_msg.tool_calls
```

    [{'name': 'update_favorite_pets',
      'args': {'pets': ['cats', 'parrots']},
      'id': 'call_W3cn4lZmJlyk8PCrKN4PRwqB',
      'type': 'tool_call'}]

## Injecting arguments at runtime

If we want to actually execute our tools using the model-generated tool call, we'll need to inject the user_id ourselves:

```python
from copy import deepcopy

from langchain_core.runnables import chain

@chain
def inject_user_id(ai_msg):
    tool_calls = []
    for tool_call in ai_msg.tool_calls:
        tool_call_copy = deepcopy(tool_call)
        tool_call_copy["args"]["user_id"] = user_id
        tool_calls.append(tool_call_copy)
    return tool_calls

inject_user_id.invoke(ai_msg)
```

    [{'name': 'update_favorite_pets',
      'args': {'pets': ['cats', 'parrots'], 'user_id': '123'},
      'id': 'call_W3cn4lZmJlyk8PCrKN4PRwqB',
      'type': 'tool_call'}]

And now we can chain together our model, injection code, and the actual tools to create a tool-executing chain:

```python
tool_map = {tool.name: tool for tool in tools}

@chain
def tool_router(tool_call):
    return tool_map[tool_call["name"]]

chain = llm_with_tools | inject_user_id | tool_router.map()
chain.invoke("my favorite animals are cats and parrots")
```

    [ToolMessage(content='null', name='update_favorite_pets', tool_call_id='call_HUyF6AihqANzEYxQnTUKxkXj')]

Looking at the user_to_pets dict, we can see that it's been updated to include cats and parrots:

```python
user_to_pets
```

    {'123': ['cats', 'parrots']}

## Other ways of annotating args

Here are a few other ways of annotating our tool args:

```python
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.tools import BaseTool

class UpdateFavoritePetsSchema(BaseModel):
    """Update list of favorite pets"""

    pets: List[str] = Field(..., description="List of favorite pets to set.")
    user_id: Annotated[str, InjectedToolArg] = Field(..., description="User's ID.")

@tool(args_schema=UpdateFavoritePetsSchema)
def update_favorite_pets(pets, user_id):
    user_to_pets[user_id] = pets

update_favorite_pets.get_input_schema().schema()
```

    {'title': 'UpdateFavoritePetsSchema',
     'description': 'Update list of favorite pets',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}},
      'user_id': {'title': 'User Id',
       'description': "User's ID.",
       'type': 'string'}},
     'required': ['pets', 'user_id']}

```python
update_favorite_pets.tool_call_schema.schema()
```

    {'title': 'update_favorite_pets',
     'description': 'Update list of favorite pets',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}}},
     'required': ['pets']}

```python
from typing import Optional, Type

class UpdateFavoritePets(BaseTool):
    name: str = "update_favorite_pets"
    description: str = "Update list of favorite pets"
    args_schema: Optional[Type[BaseModel]] = UpdateFavoritePetsSchema

    def _run(self, pets, user_id):
        user_to_pets[user_id] = pets

UpdateFavoritePets().get_input_schema().schema()
```

    {'title': 'UpdateFavoritePetsSchema',
     'description': 'Update list of favorite pets',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}},
      'user_id': {'title': 'User Id',
       'description': "User's ID.",
       'type': 'string'}},
     'required': ['pets', 'user_id']}

```python
UpdateFavoritePets().tool_call_schema.schema()
```

    {'title': 'update_favorite_pets',
     'description': 'Update list of favorite pets',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'description': 'List of favorite pets to set.',
       'type': 'array',
       'items': {'type': 'string'}}},
     'required': ['pets']}

```python
class UpdateFavoritePets2(BaseTool):
    name: str = "update_favorite_pets"
    description: str = "Update list of favorite pets"

    def _run(self, pets: List[str], user_id: Annotated[str, InjectedToolArg]) -> None:
        user_to_pets[user_id] = pets

UpdateFavoritePets2().get_input_schema().schema()
```

    {'title': 'update_favorite_petsSchema',
     'description': 'Use the tool.\n\nAdd run_manager: Optional[CallbackManagerForToolRun] = None\nto child implementations to enable tracing.',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'type': 'array',
       'items': {'type': 'string'}},
      'user_id': {'title': 'User Id', 'type': 'string'}},
     'required': ['pets', 'user_id']}

```python
UpdateFavoritePets2().tool_call_schema.schema()
```

    {'title': 'update_favorite_pets',
     'description': 'Update list of favorite pets',
     'type': 'object',
     'properties': {'pets': {'title': 'Pets',
       'type': 'array',
       'items': {'type': 'string'}}},
     'required': ['pets']}


LangChain release policy

---
sidebar_position: 2
sidebar_label: Release policy
---

# LangChain release policy

The LangChain ecosystem is composed of different component packages (e.g., `langchain-core`, `langchain`, `langchain-community`, `langgraph`, `langserve`, partner packages etc.)

## Versioning

### `langchain`, `langchain-core`, and integration packages

`langchain`, `langchain-core`, `langchain-text-splitters`, and integration packages (`langchain-openai`, `langchain-anthropic`, etc.) follow [semantic versioning](https://semver.org/) in the format of 0.**Y**.**Z**. The packages are under rapid development, and so are currently versioning the packages with a major version of 0.

Minor version increases will occur for:

- Breaking changes for any public interfaces *not* marked as `beta`.

Patch version increases will occur for:

- Bug fixes,
- New features,
- Any changes to private interfaces,
- Any changes to `beta` features.

When upgrading between minor versions, users should review the list of breaking changes and deprecations.

From time to time, we will version packages as **release candidates**. These are versions that are intended to be released as stable versions, but we want to get feedback from the community before doing so. Release candidates will be versioned as 0.**Y**.**Z**rc**N**. For example, 0.2.0rc1. If no issues are found, the release candidate will be released as a stable version with the same version number. If issues are found, we will release a new release candidate with an incremented `N` value (e.g., 0.2.0rc2).

### `langchain-community`

`langchain-community` is currently on version `0.2.x`.

Minor version increases will occur for:

- Updates to the major/minor versions of required `langchain-x` dependencies. E.g., when updating the required version of `langchain-core` from `^0.2.x` to `0.3.0`.

Patch version increases will occur for:

- Bug fixes,
- New features,
- Any changes to private interfaces,
- Any changes to `beta` features,
- Breaking changes to integrations to reflect breaking changes in the third-party service.

Whenever possible we will avoid making breaking changes in patch versions.
However, if an external API makes a breaking change then breaking changes to the corresponding `langchain-community` integration can occur in a patch version.

### `langchain-experimental`

`langchain-experimental` is currently on version `0.0.x`. All changes will be accompanied with patch version increases.

## Release cadence

We expect to space out **minor** releases (e.g., from 0.2.x to 0.3.0) of `langchain` and `langchain-core` by at least 2-3 months, as such releases may contain breaking changes.

Patch versions are released frequently, up to a few times per week, as they contain bug fixes and new features.

## API stability

The development of LLM applications is a rapidly evolving field, and we are constantly learning from our users and the community. As such, we expect that the APIs in `langchain` and `langchain-core` will continue to evolve to better serve the needs of our users.

Even though both `langchain` and `langchain-core` are currently in a pre-1.0 state, we are committed to maintaining API stability in these packages.

- Breaking changes to the public API will result in a minor version bump (the second digit)
- Any bug fixes or new features will result in a patch version bump (the third digit)

We will generally try to avoid making unnecessary changes, and will provide a deprecation policy for features that are being removed.

### Stability of other packages

The stability of other packages in the LangChain ecosystem may vary:

- `langchain-community` is a community maintained package that contains 3rd party integrations. While we do our best to review and test changes in `langchain-community`, `langchain-community` is expected to experience more breaking changes than `langchain` and `langchain-core` as it contains many community contributions.
- Partner packages may follow different stability and versioning policies, and users should refer to the documentation of those packages for more information; however, in general these packages are expected to be stable.

### What is a "API stability"?

API stability means:

- All the public APIs (everything in this documentation) will not be moved or renamed without providing backwards-compatible aliases.
- If new features are added to these APIs – which is quite possible – they will not break or change the meaning of existing methods. In other words, "stable" does not (necessarily) mean "complete."
- If, for some reason, an API declared stable must be removed or replaced, it will be declared deprecated but will remain in the API for at least two minor releases. Warnings will be issued when the deprecated method is called.

### **APIs marked as internal**

Certain APIs are explicitly marked as “internal” in a couple of ways:

- Some documentation refers to internals and mentions them as such. If the documentation says that something is internal, it may change.
- Functions, methods, and other objects prefixed by a leading underscore (**`_`**). This is the standard Python convention of indicating that something is private; if any method starts with a single **`_`**, it’s an internal API.
    - **Exception:** Certain methods are prefixed with `_` , but do not contain an implementation. These methods are *meant* to be overridden by sub-classes that provide the implementation. Such methods are generally part of the **Public API** of LangChain.

## Deprecation policy

We will generally avoid deprecating features until a better alternative is available.

When a feature is deprecated, it will continue to work in the current and next minor version of `langchain` and `langchain-core`. After that, the feature will be removed.

Since we're expecting to space out minor releases by at least 2-3 months, this means that a feature can be removed within 2-6 months of being deprecated.

In some situations, we may allow the feature to remain in the code base for longer periods of time, if it's not causing issues in the packages, to reduce the burden on users.


How to disable parallel tool calling

# How to disable parallel tool calling

:::info OpenAI-specific

This API is currently only supported by OpenAI.

:::

OpenAI tool calling performs tool calling in parallel by default. That means that if we ask a question like "What is the weather in Tokyo, New York, and Chicago?" and we have a tool for getting the weather, it will call the tool 3 times in parallel. We can force it to call only a single tool once by using the ``parallel_tool_call`` parameter.

First let's set up our tools and model:

```python
from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Adds a and b."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    return a * b

tools = [add, multiply]
```

```python
import os
from getpass import getpass

from langchain_openai import ChatOpenAI

os.environ["OPENAI_API_KEY"] = getpass()

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
```

Now let's show a quick example of how disabling parallel tool calls work:

```python
llm_with_tools = llm.bind_tools(tools, parallel_tool_calls=False)
llm_with_tools.invoke("Please call the first tool two times").tool_calls
```

    [{'name': 'add',
      'args': {'a': 2, 'b': 2},
      'id': 'call_Hh4JOTCDM85Sm9Pr84VKrWu5'}]

As we can see, even though we explicitly told the model to call a tool twice, by disabling parallel tool calls the model was constrained to only calling one.


How to cache chat model responses

# How to cache chat model responses

:::info Prerequisites

This guide assumes familiarity with the following concepts:
- [Chat models](/docs/concepts/#chat-models)
- [LLMs](/docs/concepts/#llms)

:::

LangChain provides an optional caching layer for chat models. This is useful for two main reasons:

- It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. This is especially useful during app development.
- It can speed up your application by reducing the number of API calls you make to the LLM provider.

This guide will walk you through how to enable this in your apps.

```{=mdx}
import ChatModelTabs from "@theme/ChatModelTabs";

<ChatModelTabs customVarName="llm" />
```

```python
# | output: false
# | echo: false

import os
from getpass import getpass

from langchain_openai import ChatOpenAI

os.environ["OPENAI_API_KEY"] = getpass()

llm = ChatOpenAI()
```

```python
# <!-- ruff: noqa: F821 -->
from langchain_core.globals import set_llm_cache
```

## In Memory Cache

This is an ephemeral cache that stores model calls in memory. It will be wiped when your environment restarts, and is not shared across processes.

```python
%%time
from langchain_core.caches import InMemoryCache

set_llm_cache(InMemoryCache())

# The first time, it is not yet in cache, so it should take longer
llm.invoke("Tell me a joke")
```

    CPU times: user 645 ms, sys: 214 ms, total: 859 ms
    Wall time: 829 ms

    AIMessage(content="Why don't scientists trust atoms?\n\nBecause they make up everything!", response_metadata={'token_usage': {'completion_tokens': 13, 'prompt_tokens': 11, 'total_tokens': 24}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'stop', 'logprobs': None}, id='run-b6836bdd-8c30-436b-828f-0ac5fc9ab50e-0')

```python
%%time
# The second time it is, so it goes faster
llm.invoke("Tell me a joke")
```

    CPU times: user 822 µs, sys: 288 µs, total: 1.11 ms
    Wall time: 1.06 ms

    AIMessage(content="Why don't scientists trust atoms?\n\nBecause they make up everything!", response_metadata={'token_usage': {'completion_tokens': 13, 'prompt_tokens': 11, 'total_tokens': 24}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'stop', 'logprobs': None}, id='run-b6836bdd-8c30-436b-828f-0ac5fc9ab50e-0')

## SQLite Cache

This cache implementation uses a `SQLite` database to store responses, and will last across process restarts.

```python
!rm .langchain.db
```

```python
# We can do the same thing with a SQLite cache
from langchain_community.cache import SQLiteCache

set_llm_cache(SQLiteCache(database_path=".langchain.db"))
```

```python
%%time
# The first time, it is not yet in cache, so it should take longer
llm.invoke("Tell me a joke")
```

    CPU times: user 9.91 ms, sys: 7.68 ms, total: 17.6 ms
    Wall time: 657 ms

    AIMessage(content='Why did the scarecrow win an award? Because he was outstanding in his field!', response_metadata={'token_usage': {'completion_tokens': 17, 'prompt_tokens': 11, 'total_tokens': 28}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'stop', 'logprobs': None}, id='run-39d9e1e8-7766-4970-b1d8-f50213fd94c5-0')

```python
%%time
# The second time it is, so it goes faster
llm.invoke("Tell me a joke")
```

    CPU times: user 52.2 ms, sys: 60.5 ms, total: 113 ms
    Wall time: 127 ms

    AIMessage(content='Why did the scarecrow win an award? Because he was outstanding in his field!', id='run-39d9e1e8-7766-4970-b1d8-f50213fd94c5-0')

## Next steps

You've now learned how to cache model responses to save time and money.

Next, check out the other how-to guides chat models in this section, like [how to get a model to return structured output](/docs/how_to/structured_output) or [how to create your own custom chat model](/docs/how_to/custom_chat_model).


How to load PDFs

# How to load PDFs

[Portable Document Format (PDF)](https://en.wikipedia.org/wiki/PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems.

This guide covers how to load `PDF` documents into the LangChain [Document](https://python.langchain.com/v0.2/api_reference/core/documents/langchain_core.documents.base.Document.html#langchain_core.documents.base.Document) format that we use downstream.

LangChain integrates with a host of PDF parsers. Some are simple and relatively low-level; others will support OCR and image-processing, or perform advanced document layout analysis. The right choice will depend on your application. Below we enumerate the possibilities.

## Using PyPDF

Here we load a PDF using `pypdf` into array of documents, where each document contains the page content and metadata with `page` number.

```python
%pip install --upgrade --quiet pypdf
```

```python
from langchain_community.document_loaders import PyPDFLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PyPDFLoader(file_path)
pages = loader.load_and_split()

pages[0]
```

    Document(page_content='LayoutParser : A Uniﬁed Toolkit for Deep\nLearning Based Document Image Analysis\nZejiang Shen1( \x00), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\nLee4, Jacob Carlson3, and Weining Li5\n1Allen Institute for AI\nshannons@allenai.org\n2Brown University\nruochen zhang@brown.edu\n3Harvard University\n{melissadell,jacob carlson }@fas.harvard.edu\n4University of Washington\nbcgl@cs.washington.edu\n5University of Waterloo\nw422li@uwaterloo.ca\nAbstract. Recent advances in document image analysis (DIA) have been\nprimarily driven by the application of neural networks. Ideally, research\noutcomes could be easily deployed in production and extended for further\ninvestigation. However, various factors like loosely organized codebases\nand sophisticated model conﬁgurations complicate the easy reuse of im-\nportant innovations by a wide audience. Though there have been on-going\neﬀorts to improve reusability and simplify deep learning (DL) model\ndevelopment in disciplines like natural language processing and computer\nvision, none of them are optimized for challenges in the domain of DIA.\nThis represents a major gap in the existing toolkit, as DIA is central to\nacademic research across a wide range of disciplines in the social sciences\nand humanities. This paper introduces LayoutParser , an open-source\nlibrary for streamlining the usage of DL in DIA research and applica-\ntions. The core LayoutParser library comes with a set of simple and\nintuitive interfaces for applying and customizing DL models for layout de-\ntection, character recognition, and many other document processing tasks.\nTo promote extensibility, LayoutParser also incorporates a community\nplatform for sharing both pre-trained models and full document digiti-\nzation pipelines. We demonstrate that LayoutParser is helpful for both\nlightweight and large-scale digitization pipelines in real-word use cases.\nThe library is publicly available at https://layout-parser.github.io .\nKeywords: Document Image Analysis ·Deep Learning ·Layout Analysis\n·Character Recognition ·Open Source library ·Toolkit.\n1 Introduction\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\ndocument image analysis (DIA) tasks including document image classiﬁcation [ 11,arXiv:2103.15348v2  [cs.CV]  21 Jun 2021', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'page': 0})

An advantage of this approach is that documents can be retrieved with page numbers.

### Vector search over PDFs

Once we have loaded PDFs into LangChain `Document` objects, we can index them (e.g., a RAG application) in the usual way:

```python
%pip install --upgrade --quiet faiss-cpu
# use `pip install faiss-gpu` for CUDA GPU support
```

```python
import getpass
import os

os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
```

```python
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

faiss_index = FAISS.from_documents(pages, OpenAIEmbeddings())
docs = faiss_index.similarity_search("What is LayoutParser?", k=2)
for doc in docs:
    print(str(doc.metadata["page"]) + ":", doc.page_content[:300])
```

    13: 14 Z. Shen et al.
    6 Conclusion
    LayoutParser provides a comprehensive toolkit for deep learning-based document
    image analysis. The oﬀ-the-shelf library is easy to install, and can be used to
    build ﬂexible and accurate pipelines for processing documents with complicated
    structures. It also supports hi
    0: LayoutParser : A Uniﬁed Toolkit for Deep
    Learning Based Document Image Analysis
    Zejiang Shen1( ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain
    Lee4, Jacob Carlson3, and Weining Li5
    1Allen Institute for AI
    shannons@allenai.org
    2Brown University
    ruochen zhang@brown.edu
    3Harvard University

### Extract text from images

Some PDFs contain images of text -- e.g., within scanned documents, or figures. Using the `rapidocr-onnxruntime` package we can extract images as text as well:

```python
%pip install --upgrade --quiet rapidocr-onnxruntime
```

```python
loader = PyPDFLoader("https://arxiv.org/pdf/2103.15348.pdf", extract_images=True)
pages = loader.load()
pages[4].page_content
```

    'LayoutParser : A Uniﬁed Toolkit for DL-Based DIA 5\nTable 1: Current layout detection models in the LayoutParser model zoo\nDataset Base Model1Large Model Notes\nPubLayNet [38] F / M M Layouts of modern scientiﬁc documents\nPRImA [3] M - Layouts of scanned modern magazines and scientiﬁc reports\nNewspaper [17] F - Layouts of scanned US newspapers from the 20th century\nTableBank [18] F F Table region on modern scientiﬁc and business document\nHJDataset [31] F / M - Layouts of history Japanese documents\n1For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\nvs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\nbackbones [ 13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [ 28] (F) and Mask\nR-CNN [ 12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\nusing the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\nzoo in coming months.\nlayout data structures , which are optimized for eﬃciency and versatility. 3) When\nnecessary, users can employ existing or customized OCR models via the uniﬁed\nAPI provided in the OCR module . 4)LayoutParser comes with a set of utility\nfunctions for the visualization and storage of the layout data. 5) LayoutParser\nis also highly customizable, via its integration with functions for layout data\nannotation and model training . We now provide detailed descriptions for each\ncomponent.\n3.1 Layout Detection Models\nInLayoutParser , a layout model takes a document image as an input and\ngenerates a list of rectangular boxes for the target content regions. Diﬀerent\nfrom traditional methods, it relies on deep convolutional neural networks rather\nthan manually curated rules to identify content regions. It is formulated as an\nobject detection problem and state-of-the-art models like Faster R-CNN [ 28] and\nMask R-CNN [ 12] are used. This yields prediction results of high accuracy and\nmakes it possible to build a concise, generalized interface for layout detection.\nLayoutParser , built upon Detectron2 [ 35], provides a minimal API that can\nperform layout detection with only four lines of code in Python:\n1import layoutparser as lp\n2image = cv2. imread (" image_file ") # load images\n3model = lp. Detectron2LayoutModel (\n4 "lp :// PubLayNet / faster_rcnn_R_50_FPN_3x / config ")\n5layout = model . detect ( image )\nLayoutParser provides a wealth of pre-trained model weights using various\ndatasets covering diﬀerent languages, time periods, and document types. Due to\ndomain shift [ 7], the prediction performance can notably drop when models are ap-\nplied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\ndocument structures and layouts vary greatly in diﬀerent domains, it is important\nto select models trained on a dataset similar to the test samples. A semantic syntax\nis used for initializing the model weights in LayoutParser , using both the dataset\nname and model name lp://<dataset-name>/<model-architecture-name> .'

## Using other PDF loaders

For a list of other PDF loaders to use, please see [this table](https://python.langchain.com/v0.2/docs/integrations/document_loaders/#pdfs)


---
sidebar_position: 3
keywords: [summarize, summarization, refine]
---

# How to summarize text through iterative refinement

LLMs can summarize and otherwise distill desired information from text, including large volumes of text. In many cases, especially when the amount of text is large compared to the size of the model's context window, it can be helpful (or necessary) to break up the summarization task into smaller components.

Iterative refinement represents one strategy for summarizing long texts. The strategy is as follows:

- Split a text into smaller documents;
- Summarize the first document;
- Refine or update the result based on the next document;
- Repeat through the sequence of documents until finished.

Note that this strategy is not parallelized. It is especially effective when understanding of a sub-document depends on prior context-- for instance, when summarizing a novel or body of text with an inherent sequence.

[LangGraph](https://langchain-ai.github.io/langgraph/), built on top of `langchain-core`, is well-suited to this problem:

- LangGraph allows for individual steps (such as successive summarizations) to be streamed, allowing for greater control of execution;
- LangGraph's [checkpointing](https://langchain-ai.github.io/langgraph/how-tos/persistence/) supports error recovery, extending with human-in-the-loop workflows, and easier incorporation into conversational applications.
- Because it is assembled from modular components, it is also simple to extend or modify (e.g., to incorporate [tool calling](/docs/concepts/#functiontool-calling) or other behavior).

Below, we demonstrate how to summarize text via iterative refinement.

## Load chat model

Let's first load a chat model:
```{=mdx}
import ChatModelTabs from "@theme/ChatModelTabs";

<ChatModelTabs
  customVarName="llm"
/>
```

```python
# | output: false
# | echo: false

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
```

## Load documents

Next, we need some documents to summarize. Below, we generate some toy documents for illustrative purposes. See the document loader [how-to guides](/docs/how_to/#document-loaders) and [integration pages](/docs/integrations/document_loaders/) for additional sources of data. The [summarization tutorial](/docs/tutorials/summarization) also includes an example summarizing a blog post.

```python
from langchain_core.documents import Document

documents = [
    Document(page_content="Apples are red", metadata={"title": "apple_book"}),
    Document(page_content="Blueberries are blue", metadata={"title": "blueberry_book"}),
    Document(page_content="Bananas are yelow", metadata={"title": "banana_book"}),
]
```

## Create graph

Below we show a LangGraph implementation of this process:

- We generate a simple chain for the initial summary that plucks out the first document, formats it into a prompt and runs inference with our LLM.
- We generate a second `refine_summary_chain` that operates on each successive document, refining the initial summary.

We will need to install `langgraph`:

```python
pip install -qU langgraph
```

```python
import operator
from typing import List, Literal, TypedDict

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableConfig
from langgraph.constants import Send
from langgraph.graph import END, START, StateGraph

# Initial summary
summarize_prompt = ChatPromptTemplate(
    [
        ("human", "Write a concise summary of the following: {context}"),
    ]
)
initial_summary_chain = summarize_prompt | llm | StrOutputParser()

# Refining the summary with new docs
refine_template = """
Produce a final summary.

Existing summary up to this point:
{existing_answer}

New context:
------------
{context}
------------

Given the new context, refine the original summary.
"""
refine_prompt = ChatPromptTemplate([("human", refine_template)])

refine_summary_chain = refine_prompt | llm | StrOutputParser()

# We will define the state of the graph to hold the document
# contents and summary. We also include an index to keep track
# of our position in the sequence of documents.
class State(TypedDict):
    contents: List[str]
    index: int
    summary: str

# We define functions for each node, including a node that generates
# the initial summary:
async def generate_initial_summary(state: State, config: RunnableConfig):
    summary = await initial_summary_chain.ainvoke(
        state["contents"][0],
        config,
    )
    return {"summary": summary, "index": 1}

# And a node that refines the summary based on the next document
async def refine_summary(state: State, config: RunnableConfig):
    content = state["contents"][state["index"]]
    summary = await refine_summary_chain.ainvoke(
        {"existing_answer": state["summary"], "context": content},
        config,
    )

    return {"summary": summary, "index": state["index"] + 1}

# Here we implement logic to either exit the application or refine
# the summary.
def should_refine(state: State) -> Literal["refine_summary", END]:
    if state["index"] >= len(state["contents"]):
        return END
    else:
        return "refine_summary"

graph = StateGraph(State)
graph.add_node("generate_initial_summary", generate_initial_summary)
graph.add_node("refine_summary", refine_summary)

graph.add_edge(START, "generate_initial_summary")
graph.add_conditional_edges("generate_initial_summary", should_refine)
graph.add_conditional_edges("refine_summary", should_refine)
app = graph.compile()
```

LangGraph allows the graph structure to be plotted to help visualize its function:

```python
from IPython.display import Image

Image(app.get_graph().draw_mermaid_png())
```

![jpeg](output_11_0.jpg)

## Invoke graph

We can step through the execution as follows, printing out the summary as it is refined:

```python
async for step in app.astream(
    {"contents": [doc.page_content for doc in documents]},
    stream_mode="values",
):
    if summary := step.get("summary"):
        print(summary)
```

    Apples are characterized by their red color.
    Apples are characterized by their red color, while blueberries are known for their blue hue.
    Apples are characterized by their red color, blueberries are known for their blue hue, and bananas are recognized for their yellow color.

The final `step` contains the summary as synthesized from the entire set of documents.

## Next steps

Check out the summarization [how-to guides](/docs/how_to/#summarization) for additional summarization strategies, including those designed for larger volumes of text.

See [this tutorial](/docs/tutorials/summarization) for more detail on summarization.

See also the [LangGraph documentation](https://langchain-ai.github.io/langgraph/) for detail on building with LangGraph.


How to split JSON data

# How to split JSON data

This json splitter splits json data while allowing control over chunk sizes. It traverses json data depth first and builds smaller json chunks. It attempts to keep nested json objects whole but will split them if needed to keep chunks between a min_chunk_size and the max_chunk_size.

If the value is not a nested json, but rather a very large string the string will not be split. If you need a hard cap on the chunk size consider composing this with a Recursive Text splitter on those chunks. There is an optional pre-processing step to split lists, by first converting them to json (dict) and then splitting them as such.

1. How the text is split: json value.
2. How the chunk size is measured: by number of characters.

```python
%pip install -qU langchain-text-splitters
```

First we load some json data:

```python
import json

import requests

# This is a large nested json object and will be loaded as a python dict
json_data = requests.get("https://api.smith.langchain.com/openapi.json").json()
```

## Basic usage

Specify `max_chunk_size` to constrain chunk sizes:

```python
from langchain_text_splitters import RecursiveJsonSplitter

splitter = RecursiveJsonSplitter(max_chunk_size=300)
```

To obtain json chunks, use the `.split_json` method:

```python
# Recursively split json data - If you need to access/manipulate the smaller json chunks
json_chunks = splitter.split_json(json_data=json_data)

for chunk in json_chunks[:3]:
    print(chunk)
```

    {'openapi': '3.1.0', 'info': {'title': 'LangSmith', 'version': '0.1.0'}, 'servers': [{'url': 'https://api.smith.langchain.com', 'description': 'LangSmith API endpoint.'}]}
    {'paths': {'/api/v1/sessions/{session_id}': {'get': {'tags': ['tracer-sessions'], 'summary': 'Read Tracer Session', 'description': 'Get a specific session.', 'operationId': 'read_tracer_session_api_v1_sessions__session_id__get'}}}}
    {'paths': {'/api/v1/sessions/{session_id}': {'get': {'security': [{'API Key': []}, {'Tenant ID': []}, {'Bearer Auth': []}]}}}}

To obtain LangChain [Document](https://python.langchain.com/v0.2/api_reference/core/documents/langchain_core.documents.base.Document.html) objects, use the `.create_documents` method:

```python
# The splitter can also output documents
docs = splitter.create_documents(texts=[json_data])

for doc in docs[:3]:
    print(doc)
```

    page_content='{"openapi": "3.1.0", "info": {"title": "LangSmith", "version": "0.1.0"}, "servers": [{"url": "https://api.smith.langchain.com", "description": "LangSmith API endpoint."}]}'
    page_content='{"paths": {"/api/v1/sessions/{session_id}": {"get": {"tags": ["tracer-sessions"], "summary": "Read Tracer Session", "description": "Get a specific session.", "operationId": "read_tracer_session_api_v1_sessions__session_id__get"}}}}'
    page_content='{"paths": {"/api/v1/sessions/{session_id}": {"get": {"security": [{"API Key": []}, {"Tenant ID": []}, {"Bearer Auth": []}]}}}}'

Or use `.split_text` to obtain string content directly:

```python
texts = splitter.split_text(json_data=json_data)

print(texts[0])
print(texts[1])
```

    {"openapi": "3.1.0", "info": {"title": "LangSmith", "version": "0.1.0"}, "servers": [{"url": "https://api.smith.langchain.com", "description": "LangSmith API endpoint."}]}
    {"paths": {"/api/v1/sessions/{session_id}": {"get": {"tags": ["tracer-sessions"], "summary": "Read Tracer Session", "description": "Get a specific session.", "operationId": "read_tracer_session_api_v1_sessions__session_id__get"}}}}

## How to manage chunk sizes from list content

Note that one of the chunks in this example is larger than the specified `max_chunk_size` of 300. Reviewing one of these chunks that was bigger we see there is a list object there:

```python
print([len(text) for text in texts][:10])
print()
print(texts[3])
```

    [171, 231, 126, 469, 210, 213, 237, 271, 191, 232]

    {"paths": {"/api/v1/sessions/{session_id}": {"get": {"parameters": [{"name": "session_id", "in": "path", "required": true, "schema": {"type": "string", "format": "uuid", "title": "Session Id"}}, {"name": "include_stats", "in": "query", "required": false, "schema": {"type": "boolean", "default": false, "title": "Include Stats"}}, {"name": "accept", "in": "header", "required": false, "schema": {"anyOf": [{"type": "string"}, {"type": "null"}], "title": "Accept"}}]}}}}

The json splitter by default does not split lists.

Specify `convert_lists=True` to preprocess the json, converting list content to dicts with `index:item` as `key:val` pairs:

```python
texts = splitter.split_text(json_data=json_data, convert_lists=True)
```

Let's look at the size of the chunks. Now they are all under the max

```python
print([len(text) for text in texts][:10])
```

    [176, 236, 141, 203, 212, 221, 210, 213, 242, 291]

The list has been converted to a dict, but retains all the needed contextual information even if split into many chunks:

```python
print(texts[1])
```

    {"paths": {"/api/v1/sessions/{session_id}": {"get": {"tags": {"0": "tracer-sessions"}, "summary": "Read Tracer Session", "description": "Get a specific session.", "operationId": "read_tracer_session_api_v1_sessions__session_id__get"}}}}

```python
# We can also look at the documents
docs[1]
```

    Document(page_content='{"paths": {"/api/v1/sessions/{session_id}": {"get": {"tags": ["tracer-sessions"], "summary": "Read Tracer Session", "description": "Get a specific session.", "operationId": "read_tracer_session_api_v1_sessions__session_id__get"}}}}')


Deprecations and Breaking Changes

---
sidebar_position: 3
sidebar_label: Changes
keywords: [retrievalqa, llmchain, conversationalretrievalchain]
---

# Deprecations and Breaking Changes

This code contains a list of deprecations and removals in the `langchain` and `langchain-core` packages.

New features and improvements are not listed here. See the [overview](/docs/versions/overview/) for a summary of what's new in this release.

## Breaking changes

As of release 0.2.0, `langchain` is required to be integration-agnostic. This means that code in `langchain`  should not by default instantiate any specific chat models, llms, embedding models, vectorstores etc; instead, the user will be required to specify those explicitly.

The following functions and classes require an explicit LLM to be passed as an argument:

- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreToolkit`
- `langchain.agents.agent_toolkits.vectorstore.toolkit.VectorStoreRouterToolkit`
- `langchain.chains.openai_functions.get_openapi_chain`
- `langchain.chains.router.MultiRetrievalQAChain.from_retrievers`
- `langchain.indexes.VectorStoreIndexWrapper.query`
- `langchain.indexes.VectorStoreIndexWrapper.query_with_sources`
- `langchain.indexes.VectorStoreIndexWrapper.aquery_with_sources`
- `langchain.chains.flare.FlareChain`

The following classes now require passing an explicit Embedding model as an argument:

- `langchain.indexes.VectostoreIndexCreator`

The following code has been removed:

- `langchain.natbot.NatBotChain.from_default` removed in favor of the `from_llm` class method.

Behavior was changed for the following code:

### @tool decorator

`@tool` decorator now assigns the function doc-string as the tool description. Previously, the `@tool` decorator
using to prepend the function signature.

Before 0.2.0:

```python
@tool
def my_tool(x: str) -> str:
    """Some description."""
    return "something"

print(my_tool.description)
```

Would result in: `my_tool: (x: str) -> str - Some description.`

As of 0.2.0:

It will result in: `Some description.`

## Code that moved to another package

Code that was moved from `langchain` into another package (e.g, `langchain-community`)

If you try to import it from `langchain`, the import will keep on working, but will raise a deprecation warning. The warning will provide a replacement import statement.

 ```shell
 python -c "from langchain.document_loaders.markdown import UnstructuredMarkdownLoader"
```

 ```shell
 LangChainDeprecationWarning: Importing UnstructuredMarkdownLoader from langchain.document_loaders is deprecated. Please replace deprecated imports:

 >> from langchain.document_loaders import UnstructuredMarkdownLoader

 with new imports of:

 >> from langchain_community.document_loaders import UnstructuredMarkdownLoader
```

We will continue supporting the imports in `langchain` until release 0.4 as long as the relevant package where the code lives is installed. (e.g., as long as `langchain_community` is installed.)

However, we advise for users to not rely on these imports and instead migrate to the new imports. To help with this process, we’re releasing a migration script via the LangChain CLI. See further instructions in migration guide.

## Code targeted for removal

Code that has better alternatives available and will eventually be removed, so there’s only a single way to do things. (e.g., `predict_messages` method in ChatModels has been deprecated in favor of `invoke`).

### astream events V1

If you are using `astream_events`, please review how to [migrate to astream events v2](/docs/versions/v0_2/migrating_astream_events).

### langchain_core

#### try_load_from_hub

In module: `utils.loading`
Deprecated: 0.1.30
Removal: 0.3.0

Alternative: Using the hwchase17/langchain-hub repo for prompts is deprecated. Please use https://smith.langchain.com/hub instead.

#### BaseLanguageModel.predict

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLanguageModel.predict_messages

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLanguageModel.apredict

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLanguageModel.apredict_messages

In module: `language_models.base`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### RunTypeEnum

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use string instead.

#### TracerSessionV1Base

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionV1Create

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionV1

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSessionBase

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### TracerSession

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### BaseRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### LLMRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### ChainRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### ToolRun

In module: `tracers.schemas`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Run

#### BaseChatModel.__call__

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.call_as_llm

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.predict

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.predict_messages

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseChatModel.apredict

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseChatModel.apredict_messages

In module: `language_models.chat_models`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLLM.__call__

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.predict

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.predict_messages

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: invoke

#### BaseLLM.apredict

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseLLM.apredict_messages

In module: `language_models.llms`
Deprecated: 0.1.7
Removal: 0.3.0

Alternative: ainvoke

#### BaseRetriever.get_relevant_documents

In module: `retrievers`
Deprecated: 0.1.46
Removal: 0.3.0

Alternative: invoke

#### BaseRetriever.aget_relevant_documents

In module: `retrievers`
Deprecated: 0.1.46
Removal: 0.3.0

Alternative: ainvoke

#### ChatPromptTemplate.from_role_strings

In module: `prompts.chat`
Deprecated: 0.0.1
Removal:

Alternative: from_messages classmethod

#### ChatPromptTemplate.from_strings

In module: `prompts.chat`
Deprecated: 0.0.1
Removal:

Alternative: from_messages classmethod

#### BaseTool.__call__

In module: `tools`
Deprecated: 0.1.47
Removal: 0.3.0

Alternative: invoke

#### convert_pydantic_to_openai_function

In module: `utils.function_calling`
Deprecated: 0.1.16
Removal: 0.3.0

Alternative: langchain_core.utils.function_calling.convert_to_openai_function()

#### convert_pydantic_to_openai_tool

In module: `utils.function_calling`
Deprecated: 0.1.16
Removal: 0.3.0

Alternative: langchain_core.utils.function_calling.convert_to_openai_tool()

#### convert_python_function_to_openai_function

In module: `utils.function_calling`
Deprecated: 0.1.16
Removal: 0.3.0

Alternative: langchain_core.utils.function_calling.convert_to_openai_function()

#### format_tool_to_openai_function

In module: `utils.function_calling`
Deprecated: 0.1.16
Removal: 0.3.0

Alternative: langchain_core.utils.function_calling.convert_to_openai_function()

#### format_tool_to_openai_tool

In module: `utils.function_calling`
Deprecated: 0.1.16
Removal: 0.3.0

Alternative: langchain_core.utils.function_calling.convert_to_openai_tool()

### langchain

#### AgentType

In module: `agents.agent_types`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use [LangGraph](/docs/how_to/migrate_agent/) or new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc.

#### Chain.__call__

In module: `chains.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: invoke

#### Chain.acall

In module: `chains.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: ainvoke

#### Chain.run

In module: `chains.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: invoke

#### Chain.arun

In module: `chains.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: ainvoke

#### Chain.apply

In module: `chains.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: batch

#### LLMChain

In module: `chains.llm`
Deprecated: 0.1.17
Removal: 0.3.0

Alternative: [RunnableSequence](/docs/how_to/sequence/), e.g., `prompt | llm`

This [migration guide](/docs/versions/migrating_chains/llm_chain) has a side-by-side comparison.

#### LLMSingleActionAgent

In module: `agents.agent`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use [LangGraph](/docs/how_to/migrate_agent/) or new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc.

#### Agent

In module: `agents.agent`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use [LangGraph](/docs/how_to/migrate_agent/) or new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc.

#### OpenAIFunctionsAgent

In module: `agents.openai_functions_agent.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_openai_functions_agent

#### ZeroShotAgent

In module: `agents.mrkl.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_react_agent

#### MRKLChain

In module: `agents.mrkl.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### ConversationalAgent

In module: `agents.conversational.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_react_agent

#### ConversationalChatAgent

In module: `agents.conversational_chat.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_json_chat_agent

#### ChatAgent

In module: `agents.chat.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_react_agent

#### OpenAIMultiFunctionsAgent

In module: `agents.openai_functions_multi_agent.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_openai_tools_agent

#### ReActDocstoreAgent

In module: `agents.react.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### DocstoreExplorer

In module: `agents.react.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### ReActTextWorldAgent

In module: `agents.react.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### ReActChain

In module: `agents.react.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### SelfAskWithSearchAgent

In module: `agents.self_ask_with_search.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_self_ask_with_search

#### SelfAskWithSearchChain

In module: `agents.self_ask_with_search.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### StructuredChatAgent

In module: `agents.structured_chat.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_structured_chat_agent

#### RetrievalQA

In module: `chains.retrieval_qa.base`
Deprecated: 0.1.17
Removal: 0.3.0

Alternative: [create_retrieval_chain](https://python.langchain.com/v0.2/api_reference/langchain/chains/langchain.chains.retrieval.create_retrieval_chain.html#langchain-chains-retrieval-create-retrieval-chain)
This [migration guide](/docs/versions/migrating_chains/retrieval_qa) has a side-by-side comparison.

#### load_agent_from_config

In module: `agents.loading`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### load_agent

In module: `agents.loading`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative:

#### initialize_agent

In module: `agents.initialize`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: Use [LangGraph](/docs/how_to/migrate_agent/) or new agent constructor methods like create_react_agent, create_json_agent, create_structured_chat_agent, etc.

#### XMLAgent

In module: `agents.xml.base`
Deprecated: 0.1.0
Removal: 0.3.0

Alternative: create_xml_agent

#### CohereRerank

In module: `retrievers.document_compressors.cohere_rerank`
Deprecated: 0.0.30
Removal: 0.3.0

Alternative: langchain_cohere.CohereRerank

#### ConversationalRetrievalChain

In module: `chains.conversational_retrieval.base`
Deprecated: 0.1.17
Removal: 0.3.0

Alternative: [create_history_aware_retriever](https://python.langchain.com/v0.2/api_reference/langchain/chains/langchain.chains.history_aware_retriever.create_history_aware_retriever.html) together with [create_retrieval_chain](https://python.langchain.com/v0.2/api_reference/langchain/chains/langchain.chains.retrieval.create_retrieval_chain.html#langchain-chains-retrieval-create-retrieval-chain) (see example in docstring)
This [migration guide](/docs/versions/migrating_chains/conversation_retrieval_chain) has a side-by-side comparison.

#### create_extraction_chain_pydantic

In module: `chains.openai_tools.extraction`
Deprecated: 0.1.14
Removal: 0.3.0

Alternative: [with_structured_output](/docs/how_to/structured_output/#the-with_structured_output-method) method on chat models that support tool calling.

#### create_openai_fn_runnable

In module: `chains.structured_output.base`
Deprecated: 0.1.14
Removal: 0.3.0

Alternative: [with_structured_output](/docs/how_to/structured_output/#the-with_structured_output-method) method on chat models that support tool calling.

#### create_structured_output_runnable

In module: `chains.structured_output.base`
Deprecated: 0.1.17
Removal: 0.3.0

Alternative: [with_structured_output](/docs/how_to/structured_output/#the-with_structured_output-method) method on chat models that support tool calling.

#### create_openai_fn_chain

In module: `chains.openai_functions.base`
Deprecated: 0.1.1
Removal: 0.3.0

Alternative: create_openai_fn_runnable

#### create_structured_output_chain

In module: `chains.openai_functions.base`
Deprecated: 0.1.1
Removal: 0.3.0

Alternative: ChatOpenAI.with_structured_output

#### create_extraction_chain

In module: `chains.openai_functions.extraction`
Deprecated: 0.1.14
Removal: 0.3.0

Alternative: [with_structured_output](/docs/how_to/structured_output/#the-with_structured_output-method) method on chat models that support tool calling.

#### create_extraction_chain_pydantic

In module: `chains.openai_functions.extraction`
Deprecated: 0.1.14
Removal: 0.3.0

Alternative: [with_structured_output](/docs/how_to/structured_output/#the-with_structured_output-method) method on chat models that support tool calling.


How to force models to call a tool

# How to force models to call a tool

:::info Prerequisites

This guide assumes familiarity with the following concepts:
- [Chat models](/docs/concepts/#chat-models)
- [LangChain Tools](/docs/concepts/#tools)
- [How to use a model to call tools](/docs/how_to/tool_calling)
:::

In order to force our LLM to spelect a specific tool, we can use the `tool_choice` parameter to ensure certain behavior. First, let's define our model and tools:

```python
from langchain_core.tools import tool

@tool
def add(a: int, b: int) -> int:
    """Adds a and b."""
    return a + b

@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    return a * b

tools = [add, multiply]
```

```python
# | output: false
# | echo: false

%pip install -qU langchain langchain_openai

import os
from getpass import getpass

from langchain_openai import ChatOpenAI

os.environ["OPENAI_API_KEY"] = getpass()

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
```

For example, we can force our tool to call the multiply tool by using the following code:

```python
llm_forced_to_multiply = llm.bind_tools(tools, tool_choice="Multiply")
llm_forced_to_multiply.invoke("what is 2 + 4")
```

    AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_9cViskmLvPnHjXk9tbVla5HA', 'function': {'arguments': '{"a":2,"b":4}', 'name': 'Multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 103, 'total_tokens': 112}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-095b827e-2bdd-43bb-8897-c843f4504883-0', tool_calls=[{'name': 'Multiply', 'args': {'a': 2, 'b': 4}, 'id': 'call_9cViskmLvPnHjXk9tbVla5HA'}], usage_metadata={'input_tokens': 103, 'output_tokens': 9, 'total_tokens': 112})

Even if we pass it something that doesn't require multiplcation - it will still call the tool!

We can also just force our tool to select at least one of our tools by passing in the "any" (or "required" which is OpenAI specific) keyword to the `tool_choice` parameter.

```python
llm_forced_to_use_tool = llm.bind_tools(tools, tool_choice="any")
llm_forced_to_use_tool.invoke("What day is today?")
```

    AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W', 'function': {'arguments': '{"a":1,"b":2}', 'name': 'Add'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 94, 'total_tokens': 109}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-28f75260-9900-4bed-8cd3-f1579abb65e5-0', tool_calls=[{'name': 'Add', 'args': {'a': 1, 'b': 2}, 'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W'}], usage_metadata={'input_tokens': 94, 'output_tokens': 15, 'total_tokens': 109})


How to save and load LangChain objects

# How to save and load LangChain objects

LangChain classes implement standard methods for serialization. Serializing LangChain objects using these methods confer some advantages:

- Secrets, such as API keys, are separated from other parameters and can be loaded back to the object on de-serialization;
- De-serialization is kept compatible across package versions, so objects that were serialized with one version of LangChain can be properly de-serialized with another.

To save and load LangChain objects using this system, use the `dumpd`, `dumps`, `load`, and `loads` functions in the [load module](https://python.langchain.com/v0.2/api_reference/core/load.html) of `langchain-core`. These functions support JSON and JSON-serializable objects.

All LangChain objects that inherit from [Serializable](https://python.langchain.com/v0.2/api_reference/core/load/langchain_core.load.serializable.Serializable.html) are JSON-serializable. Examples include [messages](https://python.langchain.com/v0.2/api_reference//python/core_api_reference.html#module-langchain_core.messages), [document objects](https://python.langchain.com/v0.2/api_reference/core/documents/langchain_core.documents.base.Document.html) (e.g., as returned from [retrievers](/docs/concepts/#retrievers)), and most [Runnables](/docs/concepts/#langchain-expression-language-lcel), such as chat models, retrievers, and [chains](/docs/how_to/sequence) implemented with the LangChain Expression Language.

Below we walk through an example with a simple [LLM chain](/docs/tutorials/llm_chain).

:::{.callout-caution}

De-serialization using `load` and `loads` can instantiate any serializable LangChain object. Only use this feature with trusted inputs!

De-serialization is a beta feature and is subject to change.
:::

```python
from langchain_core.load import dumpd, dumps, load, loads
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "Translate the following into {language}:"),
        ("user", "{text}"),
    ],
)

llm = ChatOpenAI(model="gpt-3.5-turbo-0125", api_key="llm-api-key")

chain = prompt | llm
```

## Saving objects

### To json

```python
string_representation = dumps(chain, pretty=True)
print(string_representation[:500])
```

    {
      "lc": 1,
      "type": "constructor",
      "id": [
        "langchain",
        "schema",
        "runnable",
        "RunnableSequence"
      ],
      "kwargs": {
        "first": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "prompts",
            "chat",
            "ChatPromptTemplate"
          ],
          "kwargs": {
            "input_variables": [
              "language",
              "text"
            ],
            "messages": [
              {
                "lc": 1,
                "type": "constructor",

### To a json-serializable Python dict

```python
dict_representation = dumpd(chain)

print(type(dict_representation))
```

    <class 'dict'>

### To disk

```python
import json

with open("/tmp/chain.json", "w") as fp:
    json.dump(string_representation, fp)
```

Note that the API key is withheld from the serialized representations. Parameters that are considered secret are specified by the `.lc_secrets` attribute of the LangChain object:

```python
chain.last.lc_secrets
```

    {'openai_api_key': 'OPENAI_API_KEY'}

## Loading objects

Specifying `secrets_map` in `load` and `loads` will load the corresponding secrets onto the de-serialized LangChain object.

### From string

```python
chain = loads(string_representation, secrets_map={"OPENAI_API_KEY": "llm-api-key"})
```

### From dict

```python
chain = load(dict_representation, secrets_map={"OPENAI_API_KEY": "llm-api-key"})
```

### From disk

```python
with open("/tmp/chain.json", "r") as fp:
    chain = loads(json.load(fp), secrets_map={"OPENAI_API_KEY": "llm-api-key"})
```

Note that we recover the API key specified at the start of the guide:

```python
chain.last.openai_api_key.get_secret_value()
```

    'llm-api-key'

```python

```


# How to use callbacks in async environments

:::info Prerequisites

This guide assumes familiarity with the following concepts:

- [Callbacks](/docs/concepts/#callbacks)
- [Custom callback handlers](/docs/how_to/custom_callbacks)
:::

If you are planning to use the async APIs, it is recommended to use and extend [`AsyncCallbackHandler`](https://python.langchain.com/v0.2/api_reference/core/callbacks/langchain_core.callbacks.base.AsyncCallbackHandler.html) to avoid blocking the event.

:::{.callout-warning}
If you use a sync `CallbackHandler` while using an async method to run your LLM / Chain / Tool / Agent, it will still work. However, under the hood, it will be called with [`run_in_executor`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor) which can cause issues if your `CallbackHandler` is not thread-safe.
:::

:::{.callout-danger}

If you're on `python<=3.10`, you need to remember to propagate `config` or `callbacks` when invoking other `runnable` from within a `RunnableLambda`, `RunnableGenerator` or `@tool`. If you do not do this,
the callbacks will not be propagated to the child runnables being invoked.
:::

```python
# | output: false
# | echo: false

%pip install -qU langchain langchain_anthropic

import getpass
import os

os.environ["ANTHROPIC_API_KEY"] = getpass.getpass()
```

```python
import asyncio
from typing import Any, Dict, List

from langchain_anthropic import ChatAnthropic
from langchain_core.callbacks import AsyncCallbackHandler, BaseCallbackHandler
from langchain_core.messages import HumanMessage
from langchain_core.outputs import LLMResult

class MyCustomSyncHandler(BaseCallbackHandler):
    def on_llm_new_token(self, token: str, **kwargs) -> None:
        print(f"Sync handler being called in a `thread_pool_executor`: token: {token}")

class MyCustomAsyncHandler(AsyncCallbackHandler):
    """Async callback handler that can be used to handle callbacks from langchain."""

    async def on_llm_start(
        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
    ) -> None:
        """Run when chain starts running."""
        print("zzzz....")
        await asyncio.sleep(0.3)
        class_name = serialized["name"]
        print("Hi! I just woke up. Your llm is starting")

    async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
        """Run when chain ends running."""
        print("zzzz....")
        await asyncio.sleep(0.3)
        print("Hi! I just woke up. Your llm is ending")

# To enable streaming, we pass in `streaming=True` to the ChatModel constructor
# Additionally, we pass in a list with our custom handler
chat = ChatAnthropic(
    model="claude-3-sonnet-20240229",
    max_tokens=25,
    streaming=True,
    callbacks=[MyCustomSyncHandler(), MyCustomAsyncHandler()],
)

await chat.agenerate([[HumanMessage(content="Tell me a joke")]])
```

    zzzz....
    Hi! I just woke up. Your llm is starting
    Sync handler being called in a `thread_pool_executor`: token: Here
    Sync handler being called in a `thread_pool_executor`: token: 's
    Sync handler being called in a `thread_pool_executor`: token:  a
    Sync handler being called in a `thread_pool_executor`: token:  little
    Sync handler being called in a `thread_pool_executor`: token:  joke
    Sync handler being called in a `thread_pool_executor`: token:  for
    Sync handler being called in a `thread_pool_executor`: token:  you
    Sync handler being called in a `thread_pool_executor`: token: :
    Sync handler being called in a `thread_pool_executor`: token:

    Why
    Sync handler being called in a `thread_pool_executor`: token:  can
    Sync handler being called in a `thread_pool_executor`: token: 't
    Sync handler being called in a `thread_pool_executor`: token:  a
    Sync handler being called in a `thread_pool_executor`: token:  bicycle
    Sync handler being called in a `thread_pool_executor`: token:  stan
    Sync handler being called in a `thread_pool_executor`: token: d up
    Sync handler being called in a `thread_pool_executor`: token:  by
    Sync handler being called in a `thread_pool_executor`: token:  itself
    Sync handler being called in a `thread_pool_executor`: token: ?
    Sync handler being called in a `thread_pool_executor`: token:  Because
    Sync handler being called in a `thread_pool_executor`: token:  it
    Sync handler being called in a `thread_pool_executor`: token: 's
    Sync handler being called in a `thread_pool_executor`: token:  two
    Sync handler being called in a `thread_pool_executor`: token: -
    Sync handler being called in a `thread_pool_executor`: token: tire
    zzzz....
    Hi! I just woke up. Your llm is ending

    LLMResult(generations=[[ChatGeneration(text="Here's a little joke for you:\n\nWhy can't a bicycle stand up by itself? Because it's two-tire", message=AIMessage(content="Here's a little joke for you:\n\nWhy can't a bicycle stand up by itself? Because it's two-tire", id='run-8afc89e8-02c0-4522-8480-d96977240bd4-0'))]], llm_output={}, run=[RunInfo(run_id=UUID('8afc89e8-02c0-4522-8480-d96977240bd4'))])

## Next steps

You've now learned how to create your own custom callback handlers.

Next, check out the other how-to guides in this section, such as [how to attach callbacks to a runnable](/docs/how_to/callbacks_attach).


Migrating to Astream Events v2

---
sidebar_position: 2
sidebar_label: astream_events v2
---

# Migrating to Astream Events v2

We've added a `v2` of the astream_events API with the release of `0.2.x`. You can see this [PR](https://github.com/langchain-ai/langchain/pull/21638) for more details.

The `v2` version is a re-write of the `v1` version, and should be more efficient, with more consistent output for the events. The `v1` version of the API will be deprecated in favor of the `v2` version and will be removed in  `0.4.0`.

Below is a list of changes between the `v1` and `v2` versions of the API.

### output for `on_chat_model_end`

In `v1`, the outputs associated with `on_chat_model_end` changed depending on whether the
chat model was run as a root level runnable or as part of a chain.

As a root level runnable the output was:

```python
"data": {"output": AIMessageChunk(content="hello world!", id='some id')}
```

As part of a chain the output was:

```
            "data": {
                "output": {
                    "generations": [
                        [
                            {
                                "generation_info": None,
                                "message": AIMessageChunk(
                                    content="hello world!", id=AnyStr()
                                ),
                                "text": "hello world!",
                                "type": "ChatGenerationChunk",
                            }
                        ]
                    ],
                    "llm_output": None,
                }
            },
```

As of `v2`, the output will always be the simpler representation:

```python
"data": {"output": AIMessageChunk(content="hello world!", id='some id')}
```

:::note
Non chat models (i.e., regular LLMs) are will be consistently associated with the more verbose format for now.
:::

### output for `on_retriever_end`

`on_retriever_end` output will always return a list of `Documents`.

Before:
```python
{
    "data": {
        "output": [
            Document(...),
            Document(...),
            ...
        ]
    }
}
```

### Removed `on_retriever_stream`

The `on_retriever_stream` event was an artifact of the implementation and has been removed.

Full information associated with the event is already available in the `on_retriever_end` event.

Please use `on_retriever_end` instead.

### Removed `on_tool_stream`

The `on_tool_stream` event was an artifact of the implementation and has been removed.

Full information associated with the event is already available in the `on_tool_end` event.

Please use `on_tool_end` instead.

### Propagating Names

Names of runnables have been updated to be more consistent.

```python
model = GenericFakeChatModel(messages=infinite_cycle).configurable_fields(
    messages=ConfigurableField(
        id="messages",
        name="Messages",
        description="Messages return by the LLM",
    )
)
```

In `v1`, the event name was `RunnableConfigurableFields`.

In `v2`, the event name is `GenericFakeChatModel`.

If you're filtering by event names, check if you need to update your filters.

### RunnableRetry

Usage of [RunnableRetry](https://python.langchain.com/v0.2/api_reference/core/runnables/langchain_core.runnables.retry.RunnableRetry.html)
within an LCEL chain being streamed generated an incorrect `on_chain_end` event in `v1` corresponding
to the failed runnable invocation that was being retried. This event has been removed in `v2`.

No action is required for this change.


How to create a custom Document Loader

---
title: Custom Document Loader
sidebar_position: 10
---
# How to create a custom Document Loader

## Overview

Applications based on LLMs frequently entail extracting data from databases or files, like PDFs, and converting it into a format that LLMs can utilize. In LangChain, this usually involves creating Document objects, which encapsulate the extracted text (`page_content`) along with metadata—a dictionary containing details about the document, such as the author's name or the date of publication.

`Document` objects are often formatted into prompts that are fed into an LLM, allowing the LLM to use the information in the `Document` to generate a desired response (e.g., summarizing the document).
`Documents` can be either used immediately or indexed into a vectorstore for future retrieval and use.

The main abstractions for Document Loading are:

| Component      | Description                    |
|----------------|--------------------------------|
| Document       | Contains `text` and `metadata` |
| BaseLoader     | Use to convert raw data into `Documents`  |
| Blob           | A representation of binary data that's located either in a file or in memory |
| BaseBlobParser | Logic to parse a `Blob` to yield `Document` objects |

This guide will demonstrate how to write custom document loading and file parsing logic; specifically, we'll see how to:

1. Create a standard document Loader by sub-classing from `BaseLoader`.
2. Create a parser using  `BaseBlobParser` and use it in conjunction with `Blob` and `BlobLoaders`. This is useful primarily when working with files.

## Standard Document Loader

A document loader can be implemented by sub-classing from a `BaseLoader` which provides a standard interface for loading documents.

### Interface

| Method Name | Explanation |
|-------------|-------------|
| lazy_load   | Used to load documents one by one **lazily**. Use for production code. |
| alazy_load  | Async variant of `lazy_load` |
| load        | Used to load all the documents into memory **eagerly**. Use for prototyping or interactive work. |
| aload       | Used to load all the documents into memory **eagerly**. Use for prototyping or interactive work. **Added in 2024-04 to LangChain.** |

* The `load` methods is a convenience method meant solely for prototyping work -- it just invokes `list(self.lazy_load())`.
* The `alazy_load` has a default implementation that will delegate to `lazy_load`. If you're using async, we recommend overriding the default implementation and providing a native async implementation.

:::{.callout-important}
When implementing a document loader do **NOT** provide parameters via the `lazy_load` or `alazy_load` methods.

All configuration is expected to be passed through the initializer (__init__). This was a design choice made by LangChain to make sure that once a document loader has been instantiated it has all the information needed to load documents.
:::

### Implementation

Let's create an example of a standard document loader that loads a file and creates a document from each line in the file.

```python
from typing import AsyncIterator, Iterator

from langchain_core.document_loaders import BaseLoader
from langchain_core.documents import Document

class CustomDocumentLoader(BaseLoader):
    """An example document loader that reads a file line by line."""

    def __init__(self, file_path: str) -> None:
        """Initialize the loader with a file path.

        Args:
            file_path: The path to the file to load.
        """
        self.file_path = file_path

    def lazy_load(self) -> Iterator[Document]:  # <-- Does not take any arguments
        """A lazy loader that reads a file line by line.

        When you're implementing lazy load methods, you should use a generator
        to yield documents one by one.
        """
        with open(self.file_path, encoding="utf-8") as f:
            line_number = 0
            for line in f:
                yield Document(
                    page_content=line,
                    metadata={"line_number": line_number, "source": self.file_path},
                )
                line_number += 1

    # alazy_load is OPTIONAL.
    # If you leave out the implementation, a default implementation which delegates to lazy_load will be used!
    async def alazy_load(
        self,
    ) -> AsyncIterator[Document]:  # <-- Does not take any arguments
        """An async lazy loader that reads a file line by line."""
        # Requires aiofiles
        # Install with `pip install aiofiles`
        # https://github.com/Tinche/aiofiles
        import aiofiles

        async with aiofiles.open(self.file_path, encoding="utf-8") as f:
            line_number = 0
            async for line in f:
                yield Document(
                    page_content=line,
                    metadata={"line_number": line_number, "source": self.file_path},
                )
                line_number += 1
```

### Test 🧪

To test out the document loader, we need a file with some quality content.

```python
with open("./meow.txt", "w", encoding="utf-8") as f:
    quality_content = "meow meow🐱 \n meow meow🐱 \n meow😻😻"
    f.write(quality_content)

loader = CustomDocumentLoader("./meow.txt")
```

```python
## Test out the lazy load interface
for doc in loader.lazy_load():
    print()
    print(type(doc))
    print(doc)
```

    <class 'langchain_core.documents.base.Document'>
    page_content='meow meow🐱 \n' metadata={'line_number': 0, 'source': './meow.txt'}

    <class 'langchain_core.documents.base.Document'>
    page_content=' meow meow🐱 \n' metadata={'line_number': 1, 'source': './meow.txt'}

    <class 'langchain_core.documents.base.Document'>
    page_content=' meow😻😻' metadata={'line_number': 2, 'source': './meow.txt'}

```python
## Test out the async implementation
async for doc in loader.alazy_load():
    print()
    print(type(doc))
    print(doc)
```

    <class 'langchain_core.documents.base.Document'>
    page_content='meow meow🐱 \n' metadata={'line_number': 0, 'source': './meow.txt'}

    <class 'langchain_core.documents.base.Document'>
    page_content=' meow meow🐱 \n' metadata={'line_number': 1, 'source': './meow.txt'}

    <class 'langchain_core.documents.base.Document'>
    page_content=' meow😻😻' metadata={'line_number': 2, 'source': './meow.txt'}

:::{.callout-tip}

`load()` can be helpful in an interactive environment such as a jupyter notebook.

Avoid using it for production code since eager loading assumes that all the content
can fit into memory, which is not always the case, especially for enterprise data.
:::

```python
loader.load()
```

    [Document(page_content='meow meow🐱 \n', metadata={'line_number': 0, 'source': './meow.txt'}),
     Document(page_content=' meow meow🐱 \n', metadata={'line_number': 1, 'source': './meow.txt'}),
     Document(page_content=' meow😻😻', metadata={'line_number': 2, 'source': './meow.txt'})]

## Working with Files

Many document loaders involve parsing files. The difference between such loaders usually stems from how the file is parsed, rather than how the file is loaded. For example, you can use `open` to read the binary content of either a PDF or a markdown file, but you need different parsing logic to convert that binary data into text.

As a result, it can be helpful to decouple the parsing logic from the loading logic, which makes it easier to re-use a given parser regardless of how the data was loaded.

### BaseBlobParser

A `BaseBlobParser` is an interface that accepts a `blob` and outputs a list of `Document` objects. A `blob` is a representation of data that lives either in memory or in a file. LangChain python has a `Blob` primitive which is inspired by the [Blob WebAPI spec](https://developer.mozilla.org/en-US/docs/Web/API/Blob).

```python
from langchain_core.document_loaders import BaseBlobParser, Blob

class MyParser(BaseBlobParser):
    """A simple parser that creates a document from each line."""

    def lazy_parse(self, blob: Blob) -> Iterator[Document]:
        """Parse a blob into a document line by line."""
        line_number = 0
        with blob.as_bytes_io() as f:
            for line in f:
                line_number += 1
                yield Document(
                    page_content=line,
                    metadata={"line_number": line_number, "source": blob.source},
                )
```

```python
blob = Blob.from_path("./meow.txt")
parser = MyParser()
```

```python
list(parser.lazy_parse(blob))
```

    [Document(page_content='meow meow🐱 \n', metadata={'line_number': 1, 'source': './meow.txt'}),
     Document(page_content=' meow meow🐱 \n', metadata={'line_number': 2, 'source': './meow.txt'}),
     Document(page_content=' meow😻😻', metadata={'line_number': 3, 'source': './meow.txt'})]

Using the **blob** API also allows one to load content directly from memory without having to read it from a file!

```python
blob = Blob(data=b"some data from memory\nmeow")
list(parser.lazy_parse(blob))
```

    [Document(page_content='some data from memory\n', metadata={'line_number': 1, 'source': None}),
     Document(page_content='meow', metadata={'line_number': 2, 'source': None})]

### Blob

Let's take a quick look through some of the Blob API.

```python
blob = Blob.from_path("./meow.txt", metadata={"foo": "bar"})
```

```python
blob.encoding
```

    'utf-8'

```python
blob.as_bytes()
```

    b'meow meow\xf0\x9f\x90\xb1 \n meow meow\xf0\x9f\x90\xb1 \n meow\xf0\x9f\x98\xbb\xf0\x9f\x98\xbb'

```python
blob.as_string()
```

    'meow meow🐱 \n meow meow🐱 \n meow😻😻'

```python
blob.as_bytes_io()
```

    <contextlib._GeneratorContextManager at 0x743f34324450>

```python
blob.metadata
```

    {'foo': 'bar'}

```python
blob.source
```

    './meow.txt'

### Blob Loaders

While a parser encapsulates the logic needed to parse binary data into documents, *blob loaders* encapsulate the logic that's necessary to load blobs from a given storage location.

A the moment, `LangChain` only supports `FileSystemBlobLoader`.

You can use the `FileSystemBlobLoader` to load blobs and then use the parser to parse them.

```python
from langchain_community.document_loaders.blob_loaders import FileSystemBlobLoader

blob_loader = FileSystemBlobLoader(path=".", glob="*.mdx", show_progress=True)
```

```python
parser = MyParser()
for blob in blob_loader.yield_blobs():
    for doc in parser.lazy_parse(blob):
        print(doc)
        break
```

      0%|          | 0/8 [00:00<?, ?it/s]

    page_content='# Microsoft Office\n' metadata={'line_number': 1, 'source': 'office_file.mdx'}
    page_content='# Markdown\n' metadata={'line_number': 1, 'source': 'markdown.mdx'}
    page_content='# JSON\n' metadata={'line_number': 1, 'source': 'json.mdx'}
    page_content='---\n' metadata={'line_number': 1, 'source': 'pdf.mdx'}
    page_content='---\n' metadata={'line_number': 1, 'source': 'index.mdx'}
    page_content='# File Directory\n' metadata={'line_number': 1, 'source': 'file_directory.mdx'}
    page_content='# CSV\n' metadata={'line_number': 1, 'source': 'csv.mdx'}
    page_content='# HTML\n' metadata={'line_number': 1, 'source': 'html.mdx'}

### Generic Loader

LangChain has a `GenericLoader` abstraction which composes a `BlobLoader` with a `BaseBlobParser`.

`GenericLoader` is meant to provide standardized classmethods that make it easy to use existing `BlobLoader` implementations. At the moment, only the `FileSystemBlobLoader` is supported.

```python
from langchain_community.document_loaders.generic import GenericLoader

loader = GenericLoader.from_filesystem(
    path=".", glob="*.mdx", show_progress=True, parser=MyParser()
)

for idx, doc in enumerate(loader.lazy_load()):
    if idx < 5:
        print(doc)

print("... output truncated for demo purposes")
```

      0%|          | 0/8 [00:00<?, ?it/s]

    page_content='# Microsoft Office\n' metadata={'line_number': 1, 'source': 'office_file.mdx'}
    page_content='\n' metadata={'line_number': 2, 'source': 'office_file.mdx'}
    page_content='>[The Microsoft Office](https://www.office.com/) suite of productivity software includes Microsoft Word, Microsoft Excel, Microsoft PowerPoint, Microsoft Outlook, and Microsoft OneNote. It is available for Microsoft Windows and macOS operating systems. It is also available on Android and iOS.\n' metadata={'line_number': 3, 'source': 'office_file.mdx'}
    page_content='\n' metadata={'line_number': 4, 'source': 'office_file.mdx'}
    page_content='This covers how to load commonly used file formats including `DOCX`, `XLSX` and `PPTX` documents into a document format that we can use downstream.\n' metadata={'line_number': 5, 'source': 'office_file.mdx'}
    ... output truncated for demo purposes

#### Custom Generic Loader

If you really like creating classes, you can sub-class and create a class to encapsulate the logic together.

You can sub-class from this class to load content using an existing loader.

```python
from typing import Any

class MyCustomLoader(GenericLoader):
    @staticmethod
    def get_parser(**kwargs: Any) -> BaseBlobParser:
        """Override this method to associate a default parser with the class."""
        return MyParser()
```

```python
loader = MyCustomLoader.from_filesystem(path=".", glob="*.mdx", show_progress=True)

for idx, doc in enumerate(loader.lazy_load()):
    if idx < 5:
        print(doc)

print("... output truncated for demo purposes")
```

      0%|          | 0/8 [00:00<?, ?it/s]

    page_content='# Microsoft Office\n' metadata={'line_number': 1, 'source': 'office_file.mdx'}
    page_content='\n' metadata={'line_number': 2, 'source': 'office_file.mdx'}
    page_content='>[The Microsoft Office](https://www.office.com/) suite of productivity software includes Microsoft Word, Microsoft Excel, Microsoft PowerPoint, Microsoft Outlook, and Microsoft OneNote. It is available for Microsoft Windows and macOS operating systems. It is also available on Android and iOS.\n' metadata={'line_number': 3, 'source': 'office_file.mdx'}
    page_content='\n' metadata={'line_number': 4, 'source': 'office_file.mdx'}
    page_content='This covers how to load commonly used file formats including `DOCX`, `XLSX` and `PPTX` documents into a document format that we can use downstream.\n' metadata={'line_number': 5, 'source': 'office_file.mdx'}
    ... output truncated for demo purposes