ProvidersMoreVectaraChat Over Documents with VectaraOn this pageChat Over Documents with VectaraThis notebook is based on the chat_vector_db notebook, but using Vectara as the vector database.import osfrom langchain.vectorstores import Vectarafrom langchain.vectorstores.vectara import VectaraRetrieverfrom langchain.llms import OpenAIfrom langchain.chains import ConversationalRetrievalChainLoad in documents. You can replace this with a loader for whatever type of data you wantfrom langchain.document_loaders import TextLoaderloader = TextLoader("../../../modules/state_of_the_union.txt")documents = loader.load()We now split the documents, create embeddings for them, and put them in a vectorstore. This allows us to do semantic search over them.vectorstore = Vectara.from_documents(documents, embedding=None)We can now create a memory object, which is neccessary to track the inputs/outputs and hold a conversation.from langchain.memory import ConversationBufferMemorymemory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)We now initialize the ConversationalRetrievalChainopenai_api_key = os.environ["OPENAI_API_KEY"]llm = OpenAI(openai_api_key=openai_api_key, temperature=0)retriever = vectorstore.as_retriever(lambda_val=0.025, k=5, filter=None)d = retriever.get_relevant_documents(    "What did the president say about Ketanji Brown Jackson")qa = ConversationalRetrievalChain.from_llm(llm, retriever, memory=memory)query = "What did the president say about Ketanji Brown Jackson"result = qa({"question": query})result["answer"]    " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice, and that she will continue Justice Breyer's legacy of excellence."query = "Did he mention who she suceeded"result = qa({"question": query})result["answer"]    ' Ketanji Brown Jackson succeeded Justice Breyer.'Pass in chat history​In the above example, we used a Memory object to track chat history. We can also just pass it in explicitly. In order to do this, we need to initialize a chain without any memory object.qa = ConversationalRetrievalChain.from_llm(    OpenAI(temperature=0), vectorstore.as_retriever())Here's an example of asking a question with no chat historychat_history = []query = "What did the president say about Ketanji Brown Jackson"result = qa({"question": query, "chat_history": chat_history})result["answer"]    " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice, and that she will continue Justice Breyer's legacy of excellence."Here's an example of asking a question with some chat historychat_history = [(query, result["answer"])]query = "Did he mention who she suceeded"result = qa({"question": query, "chat_history": chat_history})result["answer"]    ' Ketanji Brown Jackson succeeded Justice Breyer.'Return Source Documents​You can also easily return source documents from the ConversationalRetrievalChain. This is useful for when you want to inspect what documents were returned.qa = ConversationalRetrievalChain.from_llm(    llm, vectorstore.as_retriever(), return_source_documents=True)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = qa({"question": query, "chat_history": chat_history})result["source_documents"][0]    Document(page_content='Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence. A former top litigator in private practice.', metadata={'source': '../../../modules/state_of_the_union.txt'})ConversationalRetrievalChain with search_distance​If you are using a vector store that supports filtering by search distance, you can add a threshold value parameter.vectordbkwargs = {"search_distance": 0.9}qa = ConversationalRetrievalChain.from_llm(    OpenAI(temperature=0), vectorstore.as_retriever(), return_source_documents=True)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = qa(    {"question": query, "chat_history": chat_history, "vectordbkwargs": vectordbkwargs})print(result["answer"])     The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice, and that she will continue Justice Breyer's legacy of excellence.ConversationalRetrievalChain with map_reduce​We can also use different types of combine document chains with the ConversationalRetrievalChain chain.from langchain.chains import LLMChainfrom langchain.chains.question_answering import load_qa_chainfrom langchain.chains.conversational_retrieval.prompts import CONDENSE_QUESTION_PROMPTquestion_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)doc_chain = load_qa_chain(llm, chain_type="map_reduce")chain = ConversationalRetrievalChain(    retriever=vectorstore.as_retriever(),    question_generator=question_generator,    combine_docs_chain=doc_chain,)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = chain({"question": query, "chat_history": chat_history})result["answer"]    " The president said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson, who is one of the nation's top legal minds and a former top litigator in private practice."ConversationalRetrievalChain with Question Answering with sources​You can also use this chain with the question answering with sources chain.from langchain.chains.qa_with_sources import load_qa_with_sources_chainquestion_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)doc_chain = load_qa_with_sources_chain(llm, chain_type="map_reduce")chain = ConversationalRetrievalChain(    retriever=vectorstore.as_retriever(),    question_generator=question_generator,    combine_docs_chain=doc_chain,)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = chain({"question": query, "chat_history": chat_history})result["answer"]    " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice.\nSOURCES: ../../../modules/state_of_the_union.txt"ConversationalRetrievalChain with streaming to stdout​Output from the chain will be streamed to stdout token by token in this example.from langchain.chains.llm import LLMChainfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerfrom langchain.chains.conversational_retrieval.prompts import (    CONDENSE_QUESTION_PROMPT,    QA_PROMPT,)from langchain.chains.question_answering import load_qa_chain# Construct a ConversationalRetrievalChain with a streaming llm for combine docs# and a separate, non-streaming llm for question generationllm = OpenAI(temperature=0, openai_api_key=openai_api_key)streaming_llm = OpenAI(    streaming=True,    callbacks=[StreamingStdOutCallbackHandler()],    temperature=0,    openai_api_key=openai_api_key,)question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)doc_chain = load_qa_chain(streaming_llm, chain_type="stuff", prompt=QA_PROMPT)qa = ConversationalRetrievalChain(    retriever=vectorstore.as_retriever(),    combine_docs_chain=doc_chain,    question_generator=question_generator,)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = qa({"question": query, "chat_history": chat_history})     The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice, and that she will continue Justice Breyer's legacy of excellence.chat_history = [(query, result["answer"])]query = "Did he mention who she suceeded"result = qa({"question": query, "chat_history": chat_history})     Justice Breyerget_chat_history Function​You can also specify a get_chat_history function, which can be used to format the chat_history string.def get_chat_history(inputs) -> str:    res = []    for human, ai in inputs:        res.append(f"Human:{human}\nAI:{ai}")    return "\n".join(res)qa = ConversationalRetrievalChain.from_llm(    llm, vectorstore.as_retriever(), get_chat_history=get_chat_history)chat_history = []query = "What did the president say about Ketanji Brown Jackson"result = qa({"question": query, "chat_history": chat_history})result["answer"]    " The president said that Ketanji Brown Jackson is one of the nation's top legal minds and a former top litigator in private practice, and that she will continue Justice Breyer's legacy of excellence."PreviousVectaraNextVectara Text GenerationPass in chat historyReturn Source DocumentsConversationalRetrievalChain with search_distanceConversationalRetrievalChain with map_reduceConversationalRetrievalChain with Question Answering with sourcesConversationalRetrievalChain with streaming to stdoutget_chat_history Function