.ipynb
.pdf
LLM Caching
 Contents 
In Memory Cache
SQLite Cache
Redis Cache
SQLAlchemy Cache
Custom SQLAlchemy Schemas
Optional Caching
Optional Caching in Chains
LLM Caching#
This notebook covers how to cache results of individual LLM calls.
from langchain.llms import OpenAI
In Memory Cache#
import langchain
from langchain.cache import InMemoryCache
langchain.llm_cache = InMemoryCache()
# To make the caching really obvious, lets use a slower model.
llm = OpenAI(model_name="text-davinci-002", n=2, best_of=2)
%%time
# The first time, it is not yet in cache, so it should take longer
llm("Tell me a joke")
CPU times: user 30.7 ms, sys: 18.6 ms, total: 49.3 ms
Wall time: 791 ms
"\n\nWhy couldn't the bicycle stand up by itself? Because it was...two tired!"
%%time
# The second time it is, so it goes faster
llm("Tell me a joke")
CPU times: user 80 µs, sys: 0 ns, total: 80 µs
Wall time: 83.9 µs
"\n\nWhy couldn't the bicycle stand up by itself? Because it was...two tired!"
SQLite Cache#
!rm .langchain.db
# We can do the same thing with a SQLite cache
from langchain.cache import SQLiteCache
langchain.llm_cache = SQLiteCache(database_path=".langchain.db")
%%time
# The first time, it is not yet in cache, so it should take longer
llm("Tell me a joke")