# embedding_metadata

key: chroma:document
string_value:
```python
# Test one article has worked by checking one object
test_article = (
    client.query
    .get("Article", ["title", "url", "content"])
    .with_limit(1)
    .do()
)["data"]["Get"]["Article"][0]

print(test_article['title'])
print(test_article['url'])
print(test_article['content'])
```

### Question Answering on the Data

As above, we'll fire some queries at our new Index and get back results based on the closeness to our existing vectors


```python
def qna(query, collection_name):
    
    properties = [
        "title", "content", "url",
        "_additional { answer { hasAnswer property result startPosition endPosition } distance }"
    ]

    ask = {
        "question": query,
        "properties": ["content"]
    }

    result = (
        client.query
        .get(collection_name, properties)
        .with_ask(ask)
        .with_limit(1)
        .do()
    )
    
    # Check for errors
    if ("errors" in result):
        print ("\033[91mYou probably have run out of OpenAI API calls for the current minute – the limit is set at 60 per minute.")
        raise Exception(result["errors"][0]['message'])
    
    return result["data"]["Get"][collection_name]
```


```python
query_result = qna("Did Alanis Morissette win a Grammy?", "Article")

for i, article in enumerate(query_result):
    print(f"{i+1}. { article['_additional']['answer']['result']} (Distance: {round(article['_additional']['distance'],3) })")
```


```python
query_result = qna("What is the capital of China?", "Article")

for i, article in enumerate(query_result):
    if article['_additional']['answer']['hasAnswer'] == False:
      print('No answer found')
    else:
      print(f"{i+1}. { article['_additional']['answer']['result']} (Distance: {round(article['_additional']['distance'],3) })")
```

Thanks for following along, you're now equipped to set up your own vector databases and use embeddings to do all kinds of cool things - enjoy! For more complex use cases please continue to work through other cookbook examples in this repo.


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb


key: _node_content
string_value:
{"id_": "41a4e310-9812-42df-a834-abe0f1c9e943", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "25d285bb-38bf-4d18-93ff-76ff4068c1fa", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0068fe76cc029a3134d445a268d1a1cac275ad10ae7f1be1b16907c610b4a17b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "fe9b7f04-f1de-4d07-b93b-65a6e0cf9f7c", "node_type": "1", "metadata": {}, "hash": "665fc0e191604fb24e531a9081bb499ee93c0dfe8882853d9cca318743c8a157", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2582, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Question Answering with LangChain, Deep Lake, & OpenAI

This notebook shows how to implement a question answering system with LangChain, [Deep Lake](https://activeloop.ai/) as a vector store and OpenAI embeddings. We will take the following steps to achieve this:

1. Load a Deep Lake text dataset
2. Initialize a [Deep Lake vector store with LangChain](https://docs.activeloop.ai/tutorials/vector-store/deep-lake-vector-store-in-langchain)
3. Add text to the vector store
4. Run queries on the database
5. Done!

You can also follow other tutorials such as question answering over any type of data (PDFs, json, csv, text): [chatting with any data](https://www.activeloop.ai/resources/data-chad-an-ai-app-with-lang-chain-deep-lake-to-chat-with-any-data/) stored in Deep Lake, [code understanding](https://www.activeloop.ai/resources/lang-chain-gpt-4-for-code-understanding-twitter-algorithm/), or [question answering over PDFs](https://www.activeloop.ai/resources/ultimate-guide-to-lang-chain-deep-lake-build-chat-gpt-to-answer-questions-on-your-financial-data/), or [recommending songs](https://www.activeloop.ai/resources/3-ways-to-build-a-recommendation-engine-for-songs-with-lang-chain/).

## Install requirements
Let's install the following packages.


```python
!pip install deeplake langchain openai tiktoken
```

## Authentication
Provide your OpenAI API key here:


```python
import getpass
import os

os.environ['OPENAI_API_KEY'] = getpass.getpass()
```

    ··········


## Load a Deep Lake text dataset
We will use a 20000 sample subset of the [cohere-wikipedia-22](https://app.activeloop.ai/davitbun/cohere-wikipedia-22) dataset for this example.


```python
import deeplake

ds = deeplake.load("hub://activeloop/cohere-wikipedia-22-sample")
ds.summary()
```

    \

    Opening dataset in read-only mode as you don't have write permissions.


    -

    This dataset can be visualized in Jupyter Notebook by ds.visualize() or at https://app.activeloop.ai/activeloop/cohere-wikipedia-22-sample
    


    |

    hub://activeloop/cohere-wikipedia-22-sample loaded successfully.
    
    Dataset(path='hub://activeloop/cohere-wikipedia-22-sample', read_only=True, tensors=['ids', 'metadata', 'text'])
    
      tensor    htype     shape      dtype  compression
     -------   -------   -------    -------  ------- 
       ids      text    (20000, 1)    str     None   
     metadata   json    (20000, 1)    str     None   
       text     text    (20000, 1)    str     None   


    
 



Let's take a look at a few samples:


```python
ds[:3].text.data()["value"]
```


key: _node_content
string_value:
{"id_": "fe9b7f04-f1de-4d07-b93b-65a6e0cf9f7c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "25d285bb-38bf-4d18-93ff-76ff4068c1fa", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0068fe76cc029a3134d445a268d1a1cac275ad10ae7f1be1b16907c610b4a17b", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "41a4e310-9812-42df-a834-abe0f1c9e943", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "cca3223ad65532f05da6e569d1770e6dd998b4edffaf8ef3636720ae06704578", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "838728ed-7783-4530-afb3-8731a0cc4748", "node_type": "1", "metadata": {}, "hash": "1e87b575b3af86e056ade500aac8a05d17c75fc87052e2cc872f7722a79b02aa", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1260, "end_char_idx": 4053, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
!pip install deeplake langchain openai tiktoken
```

## Authentication
Provide your OpenAI API key here:


```python
import getpass
import os

os.environ['OPENAI_API_KEY'] = getpass.getpass()
```

    ··········


## Load a Deep Lake text dataset
We will use a 20000 sample subset of the [cohere-wikipedia-22](https://app.activeloop.ai/davitbun/cohere-wikipedia-22) dataset for this example.


```python
import deeplake

ds = deeplake.load("hub://activeloop/cohere-wikipedia-22-sample")
ds.summary()
```

    \

    Opening dataset in read-only mode as you don't have write permissions.


    -

    This dataset can be visualized in Jupyter Notebook by ds.visualize() or at https://app.activeloop.ai/activeloop/cohere-wikipedia-22-sample
    


    |

    hub://activeloop/cohere-wikipedia-22-sample loaded successfully.
    
    Dataset(path='hub://activeloop/cohere-wikipedia-22-sample', read_only=True, tensors=['ids', 'metadata', 'text'])
    
      tensor    htype     shape      dtype  compression
     -------   -------   -------    -------  ------- 
       ids      text    (20000, 1)    str     None   
     metadata   json    (20000, 1)    str     None   
       text     text    (20000, 1)    str     None   


    
 



Let's take a look at a few samples:


```python
ds[:3].text.data()["value"]
```




    ['The 24-hour clock is a way of telling the time in which the day runs from midnight to midnight and is divided into 24 hours, numbered from 0 to 23. It does not use a.m. or p.m. This system is also referred to (only in the US and the English speaking parts of Canada) as military time or (only in the United Kingdom and now very rarely) as continental time. In some parts of the world, it is called railway time. Also, the international standard notation of time (ISO 8601) is based on this format.',
     'A time in the 24-hour clock is written in the form hours:minutes (for example, 01:23), or hours:minutes:seconds (01:23:45). Numbers under 10 have a zero in front (called a leading zero); e.g. 09:07. Under the 24-hour clock system, the day begins at midnight, 00:00, and the last minute of the day begins at 23:59 and ends at 24:00, which is identical to 00:00 of the following day. 12:00 can only be mid-day. Midnight is called 24:00 and is used to mean the end of the day and 00:00 is used to mean the beginning of the day. For example, you would say "Tuesday at 24:00" and "Wednesday at 00:00" to mean exactly the same time.',
     'However, the US military prefers not to say 24:00 - they do not like to have two names for the same thing, so they always say "23:59", which is one minute before midnight.']



## LangChain's Deep Lake vector store
Let's define a `dataset_path`, this is where your Deep Lake vector store will house the text embeddings.


key: _node_content
string_value:
{"id_": "838728ed-7783-4530-afb3-8731a0cc4748", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "25d285bb-38bf-4d18-93ff-76ff4068c1fa", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0068fe76cc029a3134d445a268d1a1cac275ad10ae7f1be1b16907c610b4a17b", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "fe9b7f04-f1de-4d07-b93b-65a6e0cf9f7c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "f8d9765bbf7634e45665306b71af60b4c8f0cb979c2a423460b3e4434599372d", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "7e1d3477-6d08-4ea5-8216-a7aa38898661", "node_type": "1", "metadata": {}, "hash": "4bdf07fab2d3c01768d1becc4b92455275dd6dfcf47ea16a1d27faff9254bc11", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2591, "end_char_idx": 5106, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
['The 24-hour clock is a way of telling the time in which the day runs from midnight to midnight and is divided into 24 hours, numbered from 0 to 23. It does not use a.m. or p.m. This system is also referred to (only in the US and the English speaking parts of Canada) as military time or (only in the United Kingdom and now very rarely) as continental time. In some parts of the world, it is called railway time. Also, the international standard notation of time (ISO 8601) is based on this format.',
     'A time in the 24-hour clock is written in the form hours:minutes (for example, 01:23), or hours:minutes:seconds (01:23:45). Numbers under 10 have a zero in front (called a leading zero); e.g. 09:07. Under the 24-hour clock system, the day begins at midnight, 00:00, and the last minute of the day begins at 23:59 and ends at 24:00, which is identical to 00:00 of the following day. 12:00 can only be mid-day. Midnight is called 24:00 and is used to mean the end of the day and 00:00 is used to mean the beginning of the day. For example, you would say "Tuesday at 24:00" and "Wednesday at 00:00" to mean exactly the same time.',
     'However, the US military prefers not to say 24:00 - they do not like to have two names for the same thing, so they always say "23:59", which is one minute before midnight.']



## LangChain's Deep Lake vector store
Let's define a `dataset_path`, this is where your Deep Lake vector store will house the text embeddings.


```python
dataset_path = 'wikipedia-embeddings-deeplake'
```

We will setup OpenAI's `text-embedding-3-small` as our embedding function and initialize a Deep Lake vector store at `dataset_path`...


```python
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import DeepLake

embedding = OpenAIEmbeddings(model="text-embedding-3-small")
db = DeepLake(dataset_path, embedding=embedding, overwrite=True)
```

    




... and populate it with samples, one batch at a time, using the `add_texts` method.


```python
from tqdm.auto import tqdm

batch_size = 100

nsamples = 10  # for testing. Replace with len(ds) to append everything
for i in tqdm(range(0, nsamples, batch_size)):
    # find end of batch
    i_end = min(nsamples, i + batch_size)

    batch = ds[i:i_end]
    id_batch = batch.ids.data()["value"]
    text_batch = batch.text.data()["value"]
    meta_batch = batch.metadata.data()["value"]

    db.add_texts(text_batch, metadatas=meta_batch, ids=id_batch)
```


      0%|          | 0/1 [00:00<?, ?it/s]


key: _node_content
string_value:
{"id_": "7e1d3477-6d08-4ea5-8216-a7aa38898661", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "25d285bb-38bf-4d18-93ff-76ff4068c1fa", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0068fe76cc029a3134d445a268d1a1cac275ad10ae7f1be1b16907c610b4a17b", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "838728ed-7783-4530-afb3-8731a0cc4748", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "acdbc1aa2c0299eeb0ede362c9f9359ef9555168c5b8ca3a0aeaa9a70f5e9097", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1ad01a31-0e55-4153-b169-3c81be726e42", "node_type": "1", "metadata": {}, "hash": "01a916c1e98d52a1067a32ef516e8f4040ebcc2d303042aa5d6c85f7c03a37c3", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3911, "end_char_idx": 6573, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## LangChain's Deep Lake vector store
Let's define a `dataset_path`, this is where your Deep Lake vector store will house the text embeddings.


```python
dataset_path = 'wikipedia-embeddings-deeplake'
```

We will setup OpenAI's `text-embedding-3-small` as our embedding function and initialize a Deep Lake vector store at `dataset_path`...


```python
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import DeepLake

embedding = OpenAIEmbeddings(model="text-embedding-3-small")
db = DeepLake(dataset_path, embedding=embedding, overwrite=True)
```

    




... and populate it with samples, one batch at a time, using the `add_texts` method.


```python
from tqdm.auto import tqdm

batch_size = 100

nsamples = 10  # for testing. Replace with len(ds) to append everything
for i in tqdm(range(0, nsamples, batch_size)):
    # find end of batch
    i_end = min(nsamples, i + batch_size)

    batch = ds[i:i_end]
    id_batch = batch.ids.data()["value"]
    text_batch = batch.text.data()["value"]
    meta_batch = batch.metadata.data()["value"]

    db.add_texts(text_batch, metadatas=meta_batch, ids=id_batch)
```


      0%|          | 0/1 [00:00<?, ?it/s]


    
    creating embeddings:   0%|          | 0/1 [00:00<?, ?it/s][A
    creating embeddings: 100%|██████████| 1/1 [00:02<00:00,  2.11s/it]
    
    100%|██████████| 10/10 [00:00<00:00, 462.42it/s]

    Dataset(path='wikipedia-embeddings-deeplake', tensors=['text', 'metadata', 'embedding', 'id'])
    
      tensor      htype      shape      dtype  compression
      -------    -------    -------    -------  ------- 
       text       text      (10, 1)      str     None   
     metadata     json      (10, 1)      str     None   
     embedding  embedding  (10, 1536)  float32   None   
        id        text      (10, 1)      str     None   


    
    


## Run user queries on the database
The underlying Deep Lake dataset object is accessible through `db.vectorstore.dataset`, and the data structure can be summarized using `db.vectorstore.summary()`, which shows 4 tensors with 10 samples:


```python
db.vectorstore.summary()
```

    Dataset(path='wikipedia-embeddings-deeplake', tensors=['text', 'metadata', 'embedding', 'id'])
    
      tensor      htype      shape      dtype  compression
      -------    -------    -------    -------  ------- 
       text       text      (10, 1)      str     None   
     metadata     json      (10, 1)      str     None   
     embedding  embedding  (10, 1536)  float32   None   
        id        text      (10, 1)      str     None   


We will now setup QA on our vector store with GPT-3.5-Turbo as our LLM.


key: _node_content
string_value:
{"id_": "1ad01a31-0e55-4153-b169-3c81be726e42", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "25d285bb-38bf-4d18-93ff-76ff4068c1fa", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0068fe76cc029a3134d445a268d1a1cac275ad10ae7f1be1b16907c610b4a17b", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "7e1d3477-6d08-4ea5-8216-a7aa38898661", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/deeplake/deeplake_langchain_qa.ipynb", "title": "Question Answering with LangChain, Deep Lake, & OpenAI", "tokens": 1929, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "6702c6254addbc70cca9e7982b7e57a4feac6e4592e5cd294920fc935313c581", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5772, "end_char_idx": 7368, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Run user queries on the database
The underlying Deep Lake dataset object is accessible through `db.vectorstore.dataset`, and the data structure can be summarized using `db.vectorstore.summary()`, which shows 4 tensors with 10 samples:


```python
db.vectorstore.summary()
```

    Dataset(path='wikipedia-embeddings-deeplake', tensors=['text', 'metadata', 'embedding', 'id'])
    
      tensor      htype      shape      dtype  compression
      -------    -------    -------    -------  ------- 
       text       text      (10, 1)      str     None   
     metadata     json      (10, 1)      str     None   
     embedding  embedding  (10, 1536)  float32   None   
        id        text      (10, 1)      str     None   


We will now setup QA on our vector store with GPT-3.5-Turbo as our LLM.


```python
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI

# Re-load the vector store in case it's no longer initialized
# db = DeepLake(dataset_path = dataset_path, embedding_function=embedding)

qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(model='gpt-3.5-turbo'), chain_type="stuff", retriever=db.as_retriever())
```

Let's try running a prompt and check the output. Internally, this API performs an embedding search to find the most relevant data to feed into the LLM context.


```python
query = 'Why does the military not say 24:00?'
qa.run(query)
```




    'The military prefers not to say 24:00 because they do not like to have two names for the same thing. Instead, they always say "23:59", which is one minute before midnight.'



Et voila!


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb


key: _node_content
string_value:
{"id_": "0d1e7e99-9a61-4aac-add4-b9d4b55e91e8", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e4160066-ded8-4f66-866f-a97f7eceae06", "node_type": "1", "metadata": {}, "hash": "4499fe894d164c59e09bbf19662d344b0a123d2c8a08848b3eb524c62699e0f4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3394, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Using Tair as a vector database for OpenAI embeddings

This notebook guides you step by step on using Tair as a vector database for OpenAI embeddings.

This notebook presents an end-to-end process of:
1. Using precomputed embeddings created by OpenAI API.
2. Storing the embeddings in a cloud instance of Tair.
3. Converting raw text query to an embedding with OpenAI API.
4. Using Tair to perform the nearest neighbour search in the created collection.

### What is Tair

[Tair](https://www.alibabacloud.com/help/en/tair/latest/what-is-tair) is a cloud native in-memory database service that is developed by Alibaba Cloud. Tair is compatible with open source Redis and provides a variety of data models and enterprise-class capabilities to support your real-time online scenarios. Tair also introduces persistent memory-optimized instances that are based on the new non-volatile memory (NVM) storage medium. These instances can reduce costs by 30%, ensure data persistence, and provide almost the same performance as in-memory databases. Tair has been widely used in areas such as government affairs, finance, manufacturing, healthcare, and pan-Internet to meet their high-speed query and computing requirements.

[Tairvector](https://www.alibabacloud.com/help/en/tair/latest/tairvector) is an in-house data structure that provides high-performance real-time storage and retrieval of vectors. TairVector provides two indexing algorithms: Hierarchical Navigable Small World (HNSW) and Flat Search. Additionally, TairVector supports multiple distance functions, such as Euclidean distance, inner product, and Jaccard distance. Compared with traditional vector retrieval services, TairVector has the following advantages:
- Stores all data in memory and supports real-time index updates to reduce latency of read and write operations.
- Uses an optimized data structure in memory to better utilize storage capacity.
- Functions as an out-of-the-box data structure in a simple and efficient architecture without complex modules or dependencies.

### Deployment options

- Using [Tair Cloud Vector Database](https://www.alibabacloud.com/help/en/tair/latest/getting-started-overview). [Click here](https://www.alibabacloud.com/product/tair) to fast deploy it.


## Prerequisites

For the purposes of this exercise we need to prepare a couple of things:

1. Tair cloud server instance.
2. The 'tair' library to interact with the tair database.
3. An [OpenAI API key](https://beta.openai.com/account/api-keys).



### Install requirements

This notebook obviously requires the `openai` and `tair` packages, but there are also some other additional libraries we will use. The following command installs them all:



```python
! pip install openai redis tair pandas wget
```

    Looking in indexes: http://sg.mirrors.cloud.aliyuncs.com/pypi/simple/
    Requirement already satisfied: openai in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.28.0)
    Requirement already satisfied: redis in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (5.0.0)
    Requirement already satisfied: tair in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (1.3.6)
    Requirement already satisfied: pandas in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (2.1.0)
    Requirement already satisfied: wget in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (3.


key: _node_content
string_value:
{"id_": "e4160066-ded8-4f66-866f-a97f7eceae06", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0d1e7e99-9a61-4aac-add4-b9d4b55e91e8", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d2b04e322f392cdfd04cf264377643233363cdfb182e52205e8679e92b7feabf", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "249333da-d68d-4982-8fe5-f166f4aa83af", "node_type": "1", "metadata": {}, "hash": "5bc63a6e595fbfccde05b3ab22b229885e126f38ab6efc199ae96ecd07995818", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2259, "end_char_idx": 4879, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Prerequisites

For the purposes of this exercise we need to prepare a couple of things:

1. Tair cloud server instance.
2. The 'tair' library to interact with the tair database.
3. An [OpenAI API key](https://beta.openai.com/account/api-keys).



### Install requirements

This notebook obviously requires the `openai` and `tair` packages, but there are also some other additional libraries we will use. The following command installs them all:



```python
! pip install openai redis tair pandas wget
```

    Looking in indexes: http://sg.mirrors.cloud.aliyuncs.com/pypi/simple/
    Requirement already satisfied: openai in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.28.0)
    Requirement already satisfied: redis in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (5.0.0)
    Requirement already satisfied: tair in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (1.3.6)
    Requirement already satisfied: pandas in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (2.1.0)
    Requirement already satisfied: wget in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (3.2)
    Requirement already satisfied: requests>=2.20 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (2.31.0)
    Requirement already satisfied: tqdm in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (4.66.1)
    Requirement already satisfied: aiohttp in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (3.8.5)
    Requirement already satisfied: async-timeout>=4.0.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from redis) (4.0.3)
    Requirement already satisfied: numpy>=1.22.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (1.25.2)
    Requirement already satisfied: python-dateutil>=2.8.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2.8.2)
    Requirement already satisfied: pytz>=2020.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2023.3.post1)
    Requirement already satisfied: tzdata>=2022.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2023.3)
    Requirement already satisfied: six>=1.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
    Requirement already satisfied: charset-normalizer<4,>=2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.2.0)
    Requirement already satisfied: idna<4,>=2.5 in /root/anaconda3/envs/notebook/lib/python3.


key: _node_content
string_value:
{"id_": "249333da-d68d-4982-8fe5-f166f4aa83af", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e4160066-ded8-4f66-866f-a97f7eceae06", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "985b4bc3741616171c8cdc4e661f23ca189c35602c4e94dadb5fdcf00331cbce", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2607ead1-c71f-40cb-9528-4aabf16e7d43", "node_type": "1", "metadata": {}, "hash": "47052296c660883ef719340b963d733c780c9a9f671ff52b75566c53081a9126", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3617, "end_char_idx": 5966, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
10/site-packages (from openai) (4.66.1)
    Requirement already satisfied: aiohttp in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (3.8.5)
    Requirement already satisfied: async-timeout>=4.0.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from redis) (4.0.3)
    Requirement already satisfied: numpy>=1.22.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (1.25.2)
    Requirement already satisfied: python-dateutil>=2.8.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2.8.2)
    Requirement already satisfied: pytz>=2020.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2023.3.post1)
    Requirement already satisfied: tzdata>=2022.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pandas) (2023.3)
    Requirement already satisfied: six>=1.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
    Requirement already satisfied: charset-normalizer<4,>=2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.2.0)
    Requirement already satisfied: idna<4,>=2.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2023.7.22)
    Requirement already satisfied: attrs>=17.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (22.1.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: yarl<2.0,>=1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)
    Requirement already satisfied: frozenlist>=1.1.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)
    Requirement already satisfied: aiosignal>=1.1.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.3.


key: _node_content
string_value:
{"id_": "2607ead1-c71f-40cb-9528-4aabf16e7d43", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "249333da-d68d-4982-8fe5-f166f4aa83af", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c1861349c95d1b57ecd722a36fae8ce34922daca7c90229eadb79d6551ee9762", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "bef7a064-71a8-44ea-a477-433c66a74ffe", "node_type": "1", "metadata": {}, "hash": "ddecbe1e8266fa8287858488e4cd9d0c0e36a1889069b505410949e3a7b967b2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4781, "end_char_idx": 7228, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
2.0)
    Requirement already satisfied: idna<4,>=2.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2023.7.22)
    Requirement already satisfied: attrs>=17.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (22.1.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: yarl<2.0,>=1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)
    Requirement already satisfied: frozenlist>=1.1.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)
    Requirement already satisfied: aiosignal>=1.1.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)
    [33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m[33m
    [0m

### Prepare your OpenAI API key

The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://beta.openai.com/account/api-keys](https://beta.openai.com/account/api-keys).

Once you get your key, please add it by getpass.


```python
import getpass
import openai

openai.api_key = getpass.getpass("Input your OpenAI API key:")
```

    Input your OpenAI API key:········


## Connect to Tair
First add it to your environment variables.

Connecting to a running instance of Tair server is easy with the official Python library.


```python
# The format of url: redis://[[username]:[password]]@localhost:6379/0
TAIR_URL = getpass.getpass("Input your tair url:")
```

    Input your tair url:········



```python
from tair import Tair as TairClient

# connect to tair from url and create a client

url = TAIR_URL
client = TairClient.from_url(url)
```

We can test the connection by ping:


```python
client.ping()
```




    True


key: _node_content
string_value:
{"id_": "bef7a064-71a8-44ea-a477-433c66a74ffe", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2607ead1-c71f-40cb-9528-4aabf16e7d43", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d9804fddfbd87884717711a506a543f0066d819aa35d404747d657580e1ac6ab", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a7056a36-aed5-4408-a22e-86fe02c4b6a3", "node_type": "1", "metadata": {}, "hash": "7861b717487a28cfd6c218373ce41b27783c7a079b179d696215139b7a306359", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5776, "end_char_idx": 8801, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
10/site-packages (from aiohttp->openai) (1.4.0)
    Requirement already satisfied: aiosignal>=1.1.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)
    [33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m[33m
    [0m

### Prepare your OpenAI API key

The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://beta.openai.com/account/api-keys](https://beta.openai.com/account/api-keys).

Once you get your key, please add it by getpass.


```python
import getpass
import openai

openai.api_key = getpass.getpass("Input your OpenAI API key:")
```

    Input your OpenAI API key:········


## Connect to Tair
First add it to your environment variables.

Connecting to a running instance of Tair server is easy with the official Python library.


```python
# The format of url: redis://[[username]:[password]]@localhost:6379/0
TAIR_URL = getpass.getpass("Input your tair url:")
```

    Input your tair url:········



```python
from tair import Tair as TairClient

# connect to tair from url and create a client

url = TAIR_URL
client = TairClient.from_url(url)
```

We can test the connection by ping:


```python
client.ping()
```




    True




```python
import wget

embeddings_url = "https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip"

# The file is ~700 MB so this will take some time
wget.download(embeddings_url)
```

    100% [......................................................................] 698933052 / 698933052




    'vector_database_wikipedia_articles_embedded (1).zip'



The downloaded file has to then be extracted:


```python
import zipfile
import os
import re
import tempfile

current_directory = os.getcwd()
zip_file_path = os.path.join(current_directory, "vector_database_wikipedia_articles_embedded.zip")
output_directory = os.path.join(current_directory, "../../data")

with zipfile.ZipFile(zip_file_path, "r") as zip_ref:
    zip_ref.extractall(output_directory)


# check the csv file exist
file_name = "vector_database_wikipedia_articles_embedded.csv"
data_directory = os.path.join(current_directory, "../../data")
file_path = os.path.join(data_directory, file_name)


if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Create Index

Tair stores data in indexes where each object is described by one key. Each key contains a vector and multiple attribute_keys.

We will start with creating two indexes, one for **title_vector** and one for **content_vector**, and then we will fill it with our precomputed embeddings.


key: _node_content
string_value:
{"id_": "a7056a36-aed5-4408-a22e-86fe02c4b6a3", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "bef7a064-71a8-44ea-a477-433c66a74ffe", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c1fdffcbe5eb6d9182e30c0537f43dbc71a1811519a5122c7ff8d33f1f378cce", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "019d3610-4282-437e-9dfa-8d098c41da62", "node_type": "1", "metadata": {}, "hash": "69f9fcc9835b4402090cf9c0a310a10f1878c514f15472471755c1357d4f6973", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7188, "end_char_idx": 9690, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
client.ping()
```




    True




```python
import wget

embeddings_url = "https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip"

# The file is ~700 MB so this will take some time
wget.download(embeddings_url)
```

    100% [......................................................................] 698933052 / 698933052




    'vector_database_wikipedia_articles_embedded (1).zip'



The downloaded file has to then be extracted:


```python
import zipfile
import os
import re
import tempfile

current_directory = os.getcwd()
zip_file_path = os.path.join(current_directory, "vector_database_wikipedia_articles_embedded.zip")
output_directory = os.path.join(current_directory, "../../data")

with zipfile.ZipFile(zip_file_path, "r") as zip_ref:
    zip_ref.extractall(output_directory)


# check the csv file exist
file_name = "vector_database_wikipedia_articles_embedded.csv"
data_directory = os.path.join(current_directory, "../../data")
file_path = os.path.join(data_directory, file_name)


if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Create Index

Tair stores data in indexes where each object is described by one key. Each key contains a vector and multiple attribute_keys.

We will start with creating two indexes, one for **title_vector** and one for **content_vector**, and then we will fill it with our precomputed embeddings.


```python
# set index parameters
index = "openai_test"
embedding_dim = 1536
distance_type = "L2"
index_type = "HNSW"
data_type = "FLOAT32"

# Create two indexes, one for title_vector and one for content_vector, skip if already exists
index_names = [index + "_title_vector", index+"_content_vector"]
for index_name in index_names:
    index_connection = client.tvs_get_index(index_name)
    if index_connection is not None:
        print("Index already exists")
    else:
        client.tvs_create_index(name=index_name, dim=embedding_dim, distance_type=distance_type,
                                index_type=index_type, data_type=data_type)
```

    Index already exists
    Index already exists


## Load data

In this section we are going to load the data prepared previous to this session, so you don't have to recompute the embeddings of Wikipedia articles with your own credits.


key: _node_content
string_value:
{"id_": "019d3610-4282-437e-9dfa-8d098c41da62", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a7056a36-aed5-4408-a22e-86fe02c4b6a3", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "f875bcbf802eca81c1e8cb2e013381777cfec07334082b9e95eb7590f9891ce8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5fd35366-5cb1-43bf-89ba-543d6fd424bc", "node_type": "1", "metadata": {}, "hash": "64efa858381f9586943366ef6a5e928a5863d0d445c19416a169a67bdfbc43fc", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 8022, "end_char_idx": 11104, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# check the csv file exist
file_name = "vector_database_wikipedia_articles_embedded.csv"
data_directory = os.path.join(current_directory, "../../data")
file_path = os.path.join(data_directory, file_name)


if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Create Index

Tair stores data in indexes where each object is described by one key. Each key contains a vector and multiple attribute_keys.

We will start with creating two indexes, one for **title_vector** and one for **content_vector**, and then we will fill it with our precomputed embeddings.


```python
# set index parameters
index = "openai_test"
embedding_dim = 1536
distance_type = "L2"
index_type = "HNSW"
data_type = "FLOAT32"

# Create two indexes, one for title_vector and one for content_vector, skip if already exists
index_names = [index + "_title_vector", index+"_content_vector"]
for index_name in index_names:
    index_connection = client.tvs_get_index(index_name)
    if index_connection is not None:
        print("Index already exists")
    else:
        client.tvs_create_index(name=index_name, dim=embedding_dim, distance_type=distance_type,
                                index_type=index_type, data_type=data_type)
```

    Index already exists
    Index already exists


## Load data

In this section we are going to load the data prepared previous to this session, so you don't have to recompute the embeddings of Wikipedia articles with your own credits.


```python
import pandas as pd
from ast import literal_eval
# Path to your local CSV file
csv_file_path = '../../data/vector_database_wikipedia_articles_embedded.csv'
article_df = pd.read_csv(csv_file_path)

# Read vectors from strings back into a list
article_df['title_vector'] = article_df.title_vector.apply(literal_eval).values
article_df['content_vector'] = article_df.content_vector.apply(literal_eval).values

# add/update data to indexes
for i in range(len(article_df)):
    # add data to index with title_vector
    client.tvs_hset(index=index_names[0], key=article_df.id[i].item(), vector=article_df.title_vector[i], is_binary=False,
                    **{"url": article_df.url[i], "title": article_df.title[i], "text": article_df.text[i]})
    # add data to index with content_vector
    client.tvs_hset(index=index_names[1], key=article_df.id[i].item(), vector=article_df.content_vector[i], is_binary=False,
                    **{"url": article_df.url[i], "title": article_df.title[i], "text": article_df.text[i]})
```


```python
# Check the data count to make sure all the points have been stored
for index_name in index_names:
    stats = client.tvs_get_index(index_name)
    count = int(stats["current_record_count"]) - int(stats["delete_record_count"])
    print(f"Count in {index_name}:{count}")

```

    Count in openai_test_title_vector:25000
    Count in openai_test_content_vector:25000


key: _node_content
string_value:
{"id_": "5fd35366-5cb1-43bf-89ba-543d6fd424bc", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "019d3610-4282-437e-9dfa-8d098c41da62", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "eb80b70974b3990d50ffe7c4e43bb8ed30d1c78e5884f7601782ff9697acff95", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "7468d248-8cb3-4ce3-a2f4-10426b310fd1", "node_type": "1", "metadata": {}, "hash": "c947b9821467ab678ba52a52e85f01f94fc1cdc2f0ff54052031aa2f232fa4e6", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9505, "end_char_idx": 12545, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Load data

In this section we are going to load the data prepared previous to this session, so you don't have to recompute the embeddings of Wikipedia articles with your own credits.


```python
import pandas as pd
from ast import literal_eval
# Path to your local CSV file
csv_file_path = '../../data/vector_database_wikipedia_articles_embedded.csv'
article_df = pd.read_csv(csv_file_path)

# Read vectors from strings back into a list
article_df['title_vector'] = article_df.title_vector.apply(literal_eval).values
article_df['content_vector'] = article_df.content_vector.apply(literal_eval).values

# add/update data to indexes
for i in range(len(article_df)):
    # add data to index with title_vector
    client.tvs_hset(index=index_names[0], key=article_df.id[i].item(), vector=article_df.title_vector[i], is_binary=False,
                    **{"url": article_df.url[i], "title": article_df.title[i], "text": article_df.text[i]})
    # add data to index with content_vector
    client.tvs_hset(index=index_names[1], key=article_df.id[i].item(), vector=article_df.content_vector[i], is_binary=False,
                    **{"url": article_df.url[i], "title": article_df.title[i], "text": article_df.text[i]})
```


```python
# Check the data count to make sure all the points have been stored
for index_name in index_names:
    stats = client.tvs_get_index(index_name)
    count = int(stats["current_record_count"]) - int(stats["delete_record_count"])
    print(f"Count in {index_name}:{count}")

```

    Count in openai_test_title_vector:25000
    Count in openai_test_content_vector:25000


## Search data

Once the data is put into Tair we will start querying the collection for the closest vectors. We may provide an additional parameter `vector_name` to switch from title to content based search. Since the precomputed embeddings were created with `text-embedding-3-small` OpenAI model, we also have to use it during search.



```python
def query_tair(client, query, vector_name="title_vector", top_k=5):

    # Creates embedding vector from user query
    embedded_query = openai.Embedding.create(
        input= query,
        model="text-embedding-3-small",
    )["data"][0]['embedding']
    embedded_query = np.array(embedded_query)

    # search for the top k approximate nearest neighbors of vector in an index
    query_result = client.tvs_knnsearch(index=index+"_"+vector_name, k=top_k, vector=embedded_query)

    return query_result
```


```python
import openai
import numpy as np

query_result = query_tair(client=client, query="modern art in Europe", vector_name="title_vector")
for i in range(len(query_result)):
    title = client.tvs_hmget(index+"_"+"content_vector", query_result[i][0].decode('utf-8'), "title")
    print(f"{i + 1}. {title[0].decode('utf-8')} (Distance: {round(query_result[i][1],3)})")
```

    1. Museum of Modern Art (Distance: 0.125)
    2. Western Europe (Distance: 0.133)
    3. Renaissance art (Distance: 0.136)
    4. Pop art (Distance: 0.14)
    5. Northern Europe (Distance: 0.145)


key: _node_content
string_value:
{"id_": "7468d248-8cb3-4ce3-a2f4-10426b310fd1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "e01d7985-9f32-4e15-b4bf-7008a78fb7c2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a23c511eba2cbf8fc78758d2fd631c1b08036313f70102f709ad31ebf07b4a0f", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5fd35366-5cb1-43bf-89ba-543d6fd424bc", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/Getting_started_with_Tair_and_OpenAI.ipynb", "title": "Using Tair as a vector database for OpenAI embeddings", "tokens": 3360, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "ed7a4e87de7657cc38a073b20ea0040143516b4896e312a8f56056baadcb7408", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11107, "end_char_idx": 13207, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Search data

Once the data is put into Tair we will start querying the collection for the closest vectors. We may provide an additional parameter `vector_name` to switch from title to content based search. Since the precomputed embeddings were created with `text-embedding-3-small` OpenAI model, we also have to use it during search.



```python
def query_tair(client, query, vector_name="title_vector", top_k=5):

    # Creates embedding vector from user query
    embedded_query = openai.Embedding.create(
        input= query,
        model="text-embedding-3-small",
    )["data"][0]['embedding']
    embedded_query = np.array(embedded_query)

    # search for the top k approximate nearest neighbors of vector in an index
    query_result = client.tvs_knnsearch(index=index+"_"+vector_name, k=top_k, vector=embedded_query)

    return query_result
```


```python
import openai
import numpy as np

query_result = query_tair(client=client, query="modern art in Europe", vector_name="title_vector")
for i in range(len(query_result)):
    title = client.tvs_hmget(index+"_"+"content_vector", query_result[i][0].decode('utf-8'), "title")
    print(f"{i + 1}. {title[0].decode('utf-8')} (Distance: {round(query_result[i][1],3)})")
```

    1. Museum of Modern Art (Distance: 0.125)
    2. Western Europe (Distance: 0.133)
    3. Renaissance art (Distance: 0.136)
    4. Pop art (Distance: 0.14)
    5. Northern Europe (Distance: 0.145)



```python
# This time we'll query using content vector
query_result = query_tair(client=client, query="Famous battles in Scottish history", vector_name="content_vector")
for i in range(len(query_result)):
    title = client.tvs_hmget(index+"_"+"content_vector", query_result[i][0].decode('utf-8'), "title")
    print(f"{i + 1}. {title[0].decode('utf-8')} (Distance: {round(query_result[i][1],3)})")
```

    1. Battle of Bannockburn (Distance: 0.131)
    2. Wars of Scottish Independence (Distance: 0.139)
    3. 1651 (Distance: 0.147)
    4. First War of Scottish Independence (Distance: 0.15)
    5. Robert I of Scotland (Distance: 0.154)



```python

```


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb


key: _node_content
string_value:
{"id_": "dc9ead63-694c-4b8e-ae49-a48b1e838969", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9f6fed76-dc62-4bc2-acc0-108c5b4dc3e1", "node_type": "1", "metadata": {}, "hash": "2b48771f4ca9bbf4b81538d05c3fe158dfc7192264d096ce829878087c71b699", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2854, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Question Answering with Langchain, Tair and OpenAI
This notebook presents how to implement a Question Answering system with Langchain, Tair as a knowledge based and OpenAI embeddings. If you are not familiar with Tair, it’s better to check out the [Getting_started_with_Tair_and_OpenAI.ipynb](Getting_started_with_Tair_and_OpenAI.ipynb) notebook.

This notebook presents an end-to-end process of:
- Calculating the embeddings with OpenAI API.
- Storing the embeddings in an Tair instance to build a knowledge base.
- Converting raw text query to an embedding with OpenAI API.
- Using Tair to perform the nearest neighbour search in the created collection to find some context.
- Asking LLM to find the answer in a given context.

All the steps will be simplified to calling some corresponding Langchain methods.

## Prerequisites
For the purposes of this exercise we need to prepare a couple of things:
[Tair cloud instance](https://www.alibabacloud.com/help/en/tair/latest/what-is-tair).
[Langchain](https://github.com/hwchase17/langchain) as a framework.
An OpenAI API key.

### Install requirements
This notebook requires the following Python packages: `openai`, `tiktoken`, `langchain` and `tair`.
- `openai` provides convenient access to the OpenAI API.
- `tiktoken` is a fast BPE tokeniser for use with OpenAI's models.
- `langchain` helps us to build applications with LLM more easily.
- `tair` library is used to interact with the tair vector database.


```python
! pip install openai tiktoken langchain tair 
```

    Looking in indexes: http://sg.mirrors.cloud.aliyuncs.com/pypi/simple/
    Requirement already satisfied: openai in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.28.0)
    Requirement already satisfied: tiktoken in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.4.0)
    Requirement already satisfied: langchain in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.0.281)
    Requirement already satisfied: tair in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (1.3.6)
    Requirement already satisfied: requests>=2.20 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (2.31.0)
    Requirement already satisfied: tqdm in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (4.66.1)
    Requirement already satisfied: aiohttp in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (3.8.5)
    Requirement already satisfied: regex>=2022.1.18 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from tiktoken) (2023.8.8)
    Requirement already satisfied: PyYAML>=5.3 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (6.0.1)
    Requirement already satisfied: SQLAlchemy<3,>=1.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (2.0.


key: _node_content
string_value:
{"id_": "9f6fed76-dc62-4bc2-acc0-108c5b4dc3e1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "dc9ead63-694c-4b8e-ae49-a48b1e838969", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "9a493468946fb2427ba16c6d10a5fb10d284094f461ad0f3542ce1564d367252", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c4a266f0-f381-4337-a50c-83babccfd6f4", "node_type": "1", "metadata": {}, "hash": "0c100157a2668b244a7ad3c06c33e5582393a48ae014de3a7110c86e8bed76e5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1560, "end_char_idx": 3919, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
mirrors.cloud.aliyuncs.com/pypi/simple/
    Requirement already satisfied: openai in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.28.0)
    Requirement already satisfied: tiktoken in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.4.0)
    Requirement already satisfied: langchain in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (0.0.281)
    Requirement already satisfied: tair in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (1.3.6)
    Requirement already satisfied: requests>=2.20 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (2.31.0)
    Requirement already satisfied: tqdm in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (4.66.1)
    Requirement already satisfied: aiohttp in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from openai) (3.8.5)
    Requirement already satisfied: regex>=2022.1.18 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from tiktoken) (2023.8.8)
    Requirement already satisfied: PyYAML>=5.3 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (6.0.1)
    Requirement already satisfied: SQLAlchemy<3,>=1.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (2.0.20)
    Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (4.0.3)
    Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (0.5.14)
    Requirement already satisfied: langsmith<0.1.0,>=0.0.21 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (0.0.33)
    Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (2.8.5)
    Requirement already satisfied: numpy<2,>=1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (1.25.2)
    Requirement already satisfied: pydantic<3,>=1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (1.10.12)
    Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (8.2.3)
    Requirement already satisfied: redis>=4.4.


key: _node_content
string_value:
{"id_": "c4a266f0-f381-4337-a50c-83babccfd6f4", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9f6fed76-dc62-4bc2-acc0-108c5b4dc3e1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c3498c332c2d6e17b6f74b8c8e4964bda42638be662779abdbe5123d990e875e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e5498ae0-7722-43ec-b0a6-bbec2dc48865", "node_type": "1", "metadata": {}, "hash": "774aae5ff50ceefb2589622b5089738434d3a91a8fa13b8db75a4078edda29d5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2713, "end_char_idx": 4992, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1)
    Requirement already satisfied: SQLAlchemy<3,>=1.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (2.0.20)
    Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (4.0.3)
    Requirement already satisfied: dataclasses-json<0.6.0,>=0.5.7 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (0.5.14)
    Requirement already satisfied: langsmith<0.1.0,>=0.0.21 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (0.0.33)
    Requirement already satisfied: numexpr<3.0.0,>=2.8.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (2.8.5)
    Requirement already satisfied: numpy<2,>=1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (1.25.2)
    Requirement already satisfied: pydantic<3,>=1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (1.10.12)
    Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from langchain) (8.2.3)
    Requirement already satisfied: redis>=4.4.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from tair) (5.0.0)
    Requirement already satisfied: attrs>=17.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (22.1.0)
    Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (3.2.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: yarl<2.0,>=1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)
    Requirement already satisfied: frozenlist>=1.1.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)
    Requirement already satisfied: aiosignal>=1.1.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)
    Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /root/anaconda3/envs/notebook/lib/python3.


key: _node_content
string_value:
{"id_": "e5498ae0-7722-43ec-b0a6-bbec2dc48865", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c4a266f0-f381-4337-a50c-83babccfd6f4", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "e079f0df51d0a955464b569100faad87d810519b59822db9b263bcb74381f71f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1cf94260-bd7f-49b7-9ed2-08565416fc7c", "node_type": "1", "metadata": {}, "hash": "0e4607097f3a20f467b68c7a98736a145a3301bcb5bb0e2b565804eb69792075", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3831, "end_char_idx": 6075, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
10/site-packages (from langchain) (8.2.3)
    Requirement already satisfied: redis>=4.4.4 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from tair) (5.0.0)
    Requirement already satisfied: attrs>=17.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (22.1.0)
    Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (3.2.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: yarl<2.0,>=1.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.9.2)
    Requirement already satisfied: frozenlist>=1.1.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.4.0)
    Requirement already satisfied: aiosignal>=1.1.2 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from aiohttp->openai) (1.3.1)
    Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (3.20.1)
    Requirement already satisfied: typing-inspect<1,>=0.4.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (0.9.0)
    Requirement already satisfied: typing-extensions>=4.2.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pydantic<3,>=1->langchain) (4.7.1)
    Requirement already satisfied: idna<4,>=2.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2023.7.22)
    Requirement already satisfied: greenlet!=0.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2)
    Requirement already satisfied: packaging>=17.


key: _node_content
string_value:
{"id_": "1cf94260-bd7f-49b7-9ed2-08565416fc7c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e5498ae0-7722-43ec-b0a6-bbec2dc48865", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a364439423fa8964f1d1c3a977466e87db50d40d20f2a911a087a286b2ff5bc0", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a003156e-fe06-437c-8f54-b75ad851aff1", "node_type": "1", "metadata": {}, "hash": "944de6f1ce084d81b5521dcfa7f696664110bea09b0681684caa854da94f369f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4934, "end_char_idx": 7162, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
0.0,>=3.18.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (3.20.1)
    Requirement already satisfied: typing-inspect<1,>=0.4.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from dataclasses-json<0.6.0,>=0.5.7->langchain) (0.9.0)
    Requirement already satisfied: typing-extensions>=4.2.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from pydantic<3,>=1->langchain) (4.7.1)
    Requirement already satisfied: idna<4,>=2.5 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: urllib3<3,>=1.21.1 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from requests>=2.20->openai) (2023.7.22)
    Requirement already satisfied: greenlet!=0.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2)
    Requirement already satisfied: packaging>=17.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (23.1)
    Requirement already satisfied: mypy-extensions>=0.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (1.0.0)
    [33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m[33m
    [0m

### Prepare your OpenAI API key
The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://platform.openai.com/account/api-keys ).

Once you get your key, please add it by getpass.


```python
import getpass

openai_api_key = getpass.getpass("Input your OpenAI API key:")
```

    Input your OpenAI API key:········


### Prepare your Tair URL
To build the Tair connection, you need to have `TAIR_URL`.


key: _node_content
string_value:
{"id_": "a003156e-fe06-437c-8f54-b75ad851aff1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1cf94260-bd7f-49b7-9ed2-08565416fc7c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "18ae778934f503af7731f5d86f18881858f8dfa0a51cc0189714a6ce5b486673", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b868a4a8-4d1e-40bb-ad69-f446deddfef7", "node_type": "1", "metadata": {}, "hash": "2adb94af7f679ce385e584fb7d448c2d39a347305df021e514e1ead7d6b64e61", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5861, "end_char_idx": 8443, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
7.22)
    Requirement already satisfied: greenlet!=0.4.17 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain) (2.0.2)
    Requirement already satisfied: packaging>=17.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (23.1)
    Requirement already satisfied: mypy-extensions>=0.3.0 in /root/anaconda3/envs/notebook/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.6.0,>=0.5.7->langchain) (1.0.0)
    [33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m[33m
    [0m

### Prepare your OpenAI API key
The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://platform.openai.com/account/api-keys ).

Once you get your key, please add it by getpass.


```python
import getpass

openai_api_key = getpass.getpass("Input your OpenAI API key:")
```

    Input your OpenAI API key:········


### Prepare your Tair URL
To build the Tair connection, you need to have `TAIR_URL`.


```python
# The format of url: redis://[[username]:[password]]@localhost:6379/0
TAIR_URL = getpass.getpass("Input your tair url:")
```

    Input your tair url:········


## Load data
In this section we are going to load the data containing some natural questions and answers to them. All the data will be used to create a Langchain application with Tair being the knowledge base.


```python
import wget

# All the examples come from https://ai.google.com/research/NaturalQuestions
# This is a sample of the training set that we download and extract for some
# further processing.
wget.download("https://storage.googleapis.com/dataset-natural-questions/questions.json")
wget.download("https://storage.googleapis.com/dataset-natural-questions/answers.json")
```

    100% [..............................................................................] 95372 / 95372




    'answers (2).json'




```python
import json

with open("questions.json", "r") as fp:
    questions = json.load(fp)

with open("answers.json", "r") as fp:
    answers = json.load(fp)
```


```python
print(questions[0])
```

    when is the last episode of season 8 of the walking dead



```python
print(answers[0])
```

    No . overall No. in season Title Directed by Written by Original air date U.S.


key: _node_content
string_value:
{"id_": "b868a4a8-4d1e-40bb-ad69-f446deddfef7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a003156e-fe06-437c-8f54-b75ad851aff1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "72c63fa729422121241d9ab4306bd1d5e8ea6dce3eed0111afb9529aeb1b9959", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "18ae8268-217c-4f6f-bae5-332e8f0249f0", "node_type": "1", "metadata": {}, "hash": "e1aef99fb175200e696cead06fa28641f9c77cec4832440b5ba5b62f246933b6", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6892, "end_char_idx": 10002, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Once you get your key, please add it by getpass.


```python
import getpass

openai_api_key = getpass.getpass("Input your OpenAI API key:")
```

    Input your OpenAI API key:········


### Prepare your Tair URL
To build the Tair connection, you need to have `TAIR_URL`.


```python
# The format of url: redis://[[username]:[password]]@localhost:6379/0
TAIR_URL = getpass.getpass("Input your tair url:")
```

    Input your tair url:········


## Load data
In this section we are going to load the data containing some natural questions and answers to them. All the data will be used to create a Langchain application with Tair being the knowledge base.


```python
import wget

# All the examples come from https://ai.google.com/research/NaturalQuestions
# This is a sample of the training set that we download and extract for some
# further processing.
wget.download("https://storage.googleapis.com/dataset-natural-questions/questions.json")
wget.download("https://storage.googleapis.com/dataset-natural-questions/answers.json")
```

    100% [..............................................................................] 95372 / 95372




    'answers (2).json'




```python
import json

with open("questions.json", "r") as fp:
    questions = json.load(fp)

with open("answers.json", "r") as fp:
    answers = json.load(fp)
```


```python
print(questions[0])
```

    when is the last episode of season 8 of the walking dead



```python
print(answers[0])
```

    No . overall No. in season Title Directed by Written by Original air date U.S. viewers ( millions ) 100 `` Mercy '' Greg Nicotero Scott M. Gimple October 22 , 2017 ( 2017 - 10 - 22 ) 11.44 Rick , Maggie , and Ezekiel rally their communities together to take down Negan . Gregory attempts to have the Hilltop residents side with Negan , but they all firmly stand behind Maggie . The group attacks the Sanctuary , taking down its fences and flooding the compound with walkers . With the Sanctuary defaced , everyone leaves except Gabriel , who reluctantly stays to save Gregory , but is left behind when Gregory abandons him . Surrounded by walkers , Gabriel hides in a trailer , where he is trapped inside with Negan . 101 `` The Damned '' Rosemary Rodriguez Matthew Negrete & Channing Powell October 29 , 2017 ( 2017 - 10 - 29 ) 8.92 Rick 's forces split into separate parties to attack several of the Saviors ' outposts , during which many members of the group are killed ; Eric is critically injured and rushed away by Aaron . Jesus stops Tara and Morgan from executing a group of surrendered Saviors . While clearing an outpost with Daryl , Rick is confronted and held at gunpoint by Morales , a survivor he met in the initial Atlanta camp , who is now with the Saviors . 102 `` Monsters '' Greg Nicotero Matthew Negrete & Channing Powell November 5 , 2017 ( 2017 - 11 - 05 ) 8.52 Daryl finds Morales threatening Rick and kills him ; the duo then pursue a group of Saviors who are transporting weapons to another outpost . Gregory returns to Hilltop , and after a heated argument , Maggie ultimately allows him back in the community .


key: _node_content
string_value:
{"id_": "18ae8268-217c-4f6f-bae5-332e8f0249f0", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b868a4a8-4d1e-40bb-ad69-f446deddfef7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "edb0717354b4e26ca3d9b3e02bcafbfde53dacdf013ba439857a177b9e64a2c3", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "af6b2248-2d4e-4b61-9e2f-ca6ba259817d", "node_type": "1", "metadata": {}, "hash": "d94c066c92c96e9babc4756e13ab1323a9069011fd0fbf90640191ab388b8e66", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 8370, "end_char_idx": 11605, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
overall No. in season Title Directed by Written by Original air date U.S. viewers ( millions ) 100 `` Mercy '' Greg Nicotero Scott M. Gimple October 22 , 2017 ( 2017 - 10 - 22 ) 11.44 Rick , Maggie , and Ezekiel rally their communities together to take down Negan . Gregory attempts to have the Hilltop residents side with Negan , but they all firmly stand behind Maggie . The group attacks the Sanctuary , taking down its fences and flooding the compound with walkers . With the Sanctuary defaced , everyone leaves except Gabriel , who reluctantly stays to save Gregory , but is left behind when Gregory abandons him . Surrounded by walkers , Gabriel hides in a trailer , where he is trapped inside with Negan . 101 `` The Damned '' Rosemary Rodriguez Matthew Negrete & Channing Powell October 29 , 2017 ( 2017 - 10 - 29 ) 8.92 Rick 's forces split into separate parties to attack several of the Saviors ' outposts , during which many members of the group are killed ; Eric is critically injured and rushed away by Aaron . Jesus stops Tara and Morgan from executing a group of surrendered Saviors . While clearing an outpost with Daryl , Rick is confronted and held at gunpoint by Morales , a survivor he met in the initial Atlanta camp , who is now with the Saviors . 102 `` Monsters '' Greg Nicotero Matthew Negrete & Channing Powell November 5 , 2017 ( 2017 - 11 - 05 ) 8.52 Daryl finds Morales threatening Rick and kills him ; the duo then pursue a group of Saviors who are transporting weapons to another outpost . Gregory returns to Hilltop , and after a heated argument , Maggie ultimately allows him back in the community . Eric dies from his injuries , leaving Aaron distraught . Despite Tara and Morgan 's objections , Jesus leads the group of surrendered Saviors to Hilltop . Ezekiel 's group attacks another Savior compound , during which several Kingdommers are shot while protecting Ezekiel . 103 `` Some Guy '' Dan Liu David Leslie Johnson November 12 , 2017 ( 2017 - 11 - 12 ) 8.69 Ezekiel 's group is overwhelmed by the Saviors , who kill all of them except for Ezekiel himself and Jerry . Carol clears the inside of the compound , killing all but two Saviors , who almost escape but are eventually caught by Rick and Daryl . En route to the Kingdom , Ezekiel , Jerry , and Carol are surrounded by walkers , but Shiva sacrifices herself to save them . The trio returns to the Kingdom , where Ezekiel 's confidence in himself as a leader has diminished . 104 5 `` The Big Scary U '' Michael E. Satrazemis Story by : Scott M. Gimple & David Leslie Johnson & Angela Kang Teleplay by : David Leslie Johnson & Angela Kang November 19 , 2017 ( 2017 - 11 - 19 ) 7.85 After confessing their sins to each other , Gabriel and Negan manage to escape from the trailer . Simon and the other lieutenants grow suspicious of each other , knowing that Rick 's forces must have inside information . The workers in the Sanctuary become increasingly frustrated with their living conditions , and a riot nearly ensues , until Negan returns and restores order . Gabriel is locked in a cell , where Eugene discovers him sick and suffering . Meanwhile , Rick and Daryl argue over how to take out the Saviors , leading Daryl to abandon Rick .


key: _node_content
string_value:
{"id_": "af6b2248-2d4e-4b61-9e2f-ca6ba259817d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "18ae8268-217c-4f6f-bae5-332e8f0249f0", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c6e60fbda7982ff38afbf9873fdd6f0ab411184dead5393adfcfd9550e35b065", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d2e17d61-b684-488e-958a-f75b14a90159", "node_type": "1", "metadata": {}, "hash": "ebc0d829f274fb00d69d09328e9a63ab898617cb659c61d4d41cb4b5bd3312d4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9891, "end_char_idx": 12920, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Gregory returns to Hilltop , and after a heated argument , Maggie ultimately allows him back in the community . Eric dies from his injuries , leaving Aaron distraught . Despite Tara and Morgan 's objections , Jesus leads the group of surrendered Saviors to Hilltop . Ezekiel 's group attacks another Savior compound , during which several Kingdommers are shot while protecting Ezekiel . 103 `` Some Guy '' Dan Liu David Leslie Johnson November 12 , 2017 ( 2017 - 11 - 12 ) 8.69 Ezekiel 's group is overwhelmed by the Saviors , who kill all of them except for Ezekiel himself and Jerry . Carol clears the inside of the compound , killing all but two Saviors , who almost escape but are eventually caught by Rick and Daryl . En route to the Kingdom , Ezekiel , Jerry , and Carol are surrounded by walkers , but Shiva sacrifices herself to save them . The trio returns to the Kingdom , where Ezekiel 's confidence in himself as a leader has diminished . 104 5 `` The Big Scary U '' Michael E. Satrazemis Story by : Scott M. Gimple & David Leslie Johnson & Angela Kang Teleplay by : David Leslie Johnson & Angela Kang November 19 , 2017 ( 2017 - 11 - 19 ) 7.85 After confessing their sins to each other , Gabriel and Negan manage to escape from the trailer . Simon and the other lieutenants grow suspicious of each other , knowing that Rick 's forces must have inside information . The workers in the Sanctuary become increasingly frustrated with their living conditions , and a riot nearly ensues , until Negan returns and restores order . Gabriel is locked in a cell , where Eugene discovers him sick and suffering . Meanwhile , Rick and Daryl argue over how to take out the Saviors , leading Daryl to abandon Rick . 105 6 `` The King , the Widow , and Rick '' John Polson Angela Kang & Corey Reed November 26 , 2017 ( 2017 - 11 - 26 ) 8.28 Rick visits Jadis in hopes of convincing her to turn against Negan ; Jadis refuses , and locks Rick in a shipping container . Carl encounters Siddiq in the woods and recruits him to Alexandria . Daryl and Tara plot to deviate from Rick 's plans by destroying the Sanctuary . Ezekiel isolates himself at the Kingdom , where Carol tries to encourage him to be the leader his people need . Maggie has the group of captured Saviors placed in a holding area and forces Gregory to join them as punishment for betraying Hilltop . 106 7 `` Time for After '' Larry Teng Matthew Negrete & Corey Reed December 3 , 2017 ( 2017 - 12 - 03 ) 7.47 After learning of Dwight 's association with Rick 's group , Eugene affirms his loyalty to Negan and outlines a plan to get rid of the walkers surrounding the Sanctuary . With help from Morgan and Tara , Daryl drives a truck through the Sanctuary 's walls , flooding its interior with walkers , killing many Saviors . Rick finally convinces Jadis and the Scavengers to align with him , and they plan to force the Saviors to surrender . However , when they arrive at the Sanctuary , Rick is horrified to see the breached walls and no sign of the walker herd .


key: _node_content
string_value:
{"id_": "d2e17d61-b684-488e-958a-f75b14a90159", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "af6b2248-2d4e-4b61-9e2f-ca6ba259817d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "bff7afeeb7bf461a0d3dc9aa2298adb2ad6c770a0b7fa4db74397cefb566acba", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a3beca38-151a-424b-b3ab-17b7c361a7df", "node_type": "1", "metadata": {}, "hash": "424b6095d7fc27109986391a79e9512227ac6068048015d827aa8fde766526e8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11269, "end_char_idx": 14223, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The workers in the Sanctuary become increasingly frustrated with their living conditions , and a riot nearly ensues , until Negan returns and restores order . Gabriel is locked in a cell , where Eugene discovers him sick and suffering . Meanwhile , Rick and Daryl argue over how to take out the Saviors , leading Daryl to abandon Rick . 105 6 `` The King , the Widow , and Rick '' John Polson Angela Kang & Corey Reed November 26 , 2017 ( 2017 - 11 - 26 ) 8.28 Rick visits Jadis in hopes of convincing her to turn against Negan ; Jadis refuses , and locks Rick in a shipping container . Carl encounters Siddiq in the woods and recruits him to Alexandria . Daryl and Tara plot to deviate from Rick 's plans by destroying the Sanctuary . Ezekiel isolates himself at the Kingdom , where Carol tries to encourage him to be the leader his people need . Maggie has the group of captured Saviors placed in a holding area and forces Gregory to join them as punishment for betraying Hilltop . 106 7 `` Time for After '' Larry Teng Matthew Negrete & Corey Reed December 3 , 2017 ( 2017 - 12 - 03 ) 7.47 After learning of Dwight 's association with Rick 's group , Eugene affirms his loyalty to Negan and outlines a plan to get rid of the walkers surrounding the Sanctuary . With help from Morgan and Tara , Daryl drives a truck through the Sanctuary 's walls , flooding its interior with walkers , killing many Saviors . Rick finally convinces Jadis and the Scavengers to align with him , and they plan to force the Saviors to surrender . However , when they arrive at the Sanctuary , Rick is horrified to see the breached walls and no sign of the walker herd . 107 8 `` How It 's Gotta Be '' Michael E. Satrazemis David Leslie Johnson & Angela Kang December 10 , 2017 ( 2017 - 12 - 10 ) 7.89 Eugene 's plan allows the Saviors to escape , and separately , the Saviors waylay the Alexandria , Hilltop , and Kingdom forces . The Scavengers abandon Rick , after which he returns to Alexandria . Ezekiel ensures that the Kingdom residents are able to escape before locking himself in the community with the Saviors . Eugene aids Gabriel and Doctor Carson in escaping the Sanctuary in order to ease his conscience . Negan attacks Alexandria , but Carl devises a plan to allow the Alexandria residents to escape into the sewers . Carl reveals he was bitten by a walker while escorting Siddiq to Alexandria . 108 9 `` Honor '' Greg Nicotero Matthew Negrete & Channing Powell February 25 , 2018 ( 2018 - 02 - 25 ) 8.28 After the Saviors leave Alexandria , the survivors make for the Hilltop while Rick and Michonne stay behind to say their final goodbyes to a dying Carl , who pleads with Rick to build a better future alongside the Saviors before killing himself . In the Kingdom , Morgan and Carol launch a rescue mission for Ezekiel . Although they are successful and retake the Kingdom , the Saviors ' lieutenant Gavin is killed by Benjamin 's vengeful brother Henry .


key: _node_content
string_value:
{"id_": "a3beca38-151a-424b-b3ab-17b7c361a7df", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d2e17d61-b684-488e-958a-f75b14a90159", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "6eebd152b2639e65f5db62ac06896d87c2679eb8c71aaf58dd2e06e35d9d0af7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "10e8e501-9c65-4cf6-9eb1-6cb5e9c19d7f", "node_type": "1", "metadata": {}, "hash": "894ad37ee349a4ecf6ef930307658cc4bfe7b78a7daaa96867242b64c4755b8f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 12533, "end_char_idx": 15464, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
With help from Morgan and Tara , Daryl drives a truck through the Sanctuary 's walls , flooding its interior with walkers , killing many Saviors . Rick finally convinces Jadis and the Scavengers to align with him , and they plan to force the Saviors to surrender . However , when they arrive at the Sanctuary , Rick is horrified to see the breached walls and no sign of the walker herd . 107 8 `` How It 's Gotta Be '' Michael E. Satrazemis David Leslie Johnson & Angela Kang December 10 , 2017 ( 2017 - 12 - 10 ) 7.89 Eugene 's plan allows the Saviors to escape , and separately , the Saviors waylay the Alexandria , Hilltop , and Kingdom forces . The Scavengers abandon Rick , after which he returns to Alexandria . Ezekiel ensures that the Kingdom residents are able to escape before locking himself in the community with the Saviors . Eugene aids Gabriel and Doctor Carson in escaping the Sanctuary in order to ease his conscience . Negan attacks Alexandria , but Carl devises a plan to allow the Alexandria residents to escape into the sewers . Carl reveals he was bitten by a walker while escorting Siddiq to Alexandria . 108 9 `` Honor '' Greg Nicotero Matthew Negrete & Channing Powell February 25 , 2018 ( 2018 - 02 - 25 ) 8.28 After the Saviors leave Alexandria , the survivors make for the Hilltop while Rick and Michonne stay behind to say their final goodbyes to a dying Carl , who pleads with Rick to build a better future alongside the Saviors before killing himself . In the Kingdom , Morgan and Carol launch a rescue mission for Ezekiel . Although they are successful and retake the Kingdom , the Saviors ' lieutenant Gavin is killed by Benjamin 's vengeful brother Henry . 109 10 `` The Lost and the Plunderers '' TBA TBA March 4 , 2018 ( 2018 - 03 - 04 ) TBD 110 11 `` Dead or Alive Or '' TBA TBA March 11 , 2018 ( 2018 - 03 - 11 ) TBD 111 12 `` The Key '' TBA TBA March 18 , 2018 ( 2018 - 03 - 18 ) TBD


## Chain definition

Langchain is already integrated with Tair and performs all the indexing for given list of documents. In our case we are going to store the set of answers we have.


```python
from langchain.vectorstores import Tair
from langchain.embeddings import OpenAIEmbeddings
from langchain import VectorDBQA, OpenAI

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)
doc_store = Tair.from_texts(
    texts=answers, embedding=embeddings, tair_url=TAIR_URL,
)
```

At this stage all the possible answers are already stored in Tair, so we can define the whole QA chain.


```python
llm = OpenAI(openai_api_key=openai_api_key)
qa = VectorDBQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    vectorstore=doc_store,
    return_source_documents=False,
)
```

    /root/anaconda3/envs/notebook/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py:251: UserWarning: `VectorDBQA` is deprecated - please use `from langchain.chains import RetrievalQA`
      warnings.warn(


key: _node_content
string_value:
{"id_": "10e8e501-9c65-4cf6-9eb1-6cb5e9c19d7f", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a3beca38-151a-424b-b3ab-17b7c361a7df", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "874ec3c20b5864e2912b382875e0d4cb9e19059db5cbc7f317849171b809ffa8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "73f5bc46-1e4e-40ad-9e49-c9f1ce71cac5", "node_type": "1", "metadata": {}, "hash": "0890d8fdd6e1241e2b670c4a7df45b8631a43ae62be4828755bcf4fd0fc8cfe3", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 14089, "end_char_idx": 15909, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Although they are successful and retake the Kingdom , the Saviors ' lieutenant Gavin is killed by Benjamin 's vengeful brother Henry . 109 10 `` The Lost and the Plunderers '' TBA TBA March 4 , 2018 ( 2018 - 03 - 04 ) TBD 110 11 `` Dead or Alive Or '' TBA TBA March 11 , 2018 ( 2018 - 03 - 11 ) TBD 111 12 `` The Key '' TBA TBA March 18 , 2018 ( 2018 - 03 - 18 ) TBD


## Chain definition

Langchain is already integrated with Tair and performs all the indexing for given list of documents. In our case we are going to store the set of answers we have.


```python
from langchain.vectorstores import Tair
from langchain.embeddings import OpenAIEmbeddings
from langchain import VectorDBQA, OpenAI

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)
doc_store = Tair.from_texts(
    texts=answers, embedding=embeddings, tair_url=TAIR_URL,
)
```

At this stage all the possible answers are already stored in Tair, so we can define the whole QA chain.


```python
llm = OpenAI(openai_api_key=openai_api_key)
qa = VectorDBQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    vectorstore=doc_store,
    return_source_documents=False,
)
```

    /root/anaconda3/envs/notebook/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py:251: UserWarning: `VectorDBQA` is deprecated - please use `from langchain.chains import RetrievalQA`
      warnings.warn(


## Search data

Once the data is put into Tair we can start asking some questions. A question will be automatically vectorized by OpenAI model, and the created vector will be used to find some possibly matching answers in Tair. Once retrieved, the most similar answers will be incorporated into the prompt sent to OpenAI Large Language Model.



```python
import random

random.seed(52)
selected_questions = random.choices(questions, k=5)
```


key: _node_content
string_value:
{"id_": "73f5bc46-1e4e-40ad-9e49-c9f1ce71cac5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "10e8e501-9c65-4cf6-9eb1-6cb5e9c19d7f", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "9be7517029803a3df79cefd8566b98ba930457a5a89cd50b4816e0510b1ef4aa", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "61e2ead4-9fdd-4265-b1ce-9f1b38908482", "node_type": "1", "metadata": {}, "hash": "6a2ed9dc173bf305747376b3a351038ee1543cfedd734814de4450bac83a3787", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 14458, "end_char_idx": 17120, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Chain definition

Langchain is already integrated with Tair and performs all the indexing for given list of documents. In our case we are going to store the set of answers we have.


```python
from langchain.vectorstores import Tair
from langchain.embeddings import OpenAIEmbeddings
from langchain import VectorDBQA, OpenAI

embeddings = OpenAIEmbeddings(openai_api_key=openai_api_key)
doc_store = Tair.from_texts(
    texts=answers, embedding=embeddings, tair_url=TAIR_URL,
)
```

At this stage all the possible answers are already stored in Tair, so we can define the whole QA chain.


```python
llm = OpenAI(openai_api_key=openai_api_key)
qa = VectorDBQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    vectorstore=doc_store,
    return_source_documents=False,
)
```

    /root/anaconda3/envs/notebook/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py:251: UserWarning: `VectorDBQA` is deprecated - please use `from langchain.chains import RetrievalQA`
      warnings.warn(


## Search data

Once the data is put into Tair we can start asking some questions. A question will be automatically vectorized by OpenAI model, and the created vector will be used to find some possibly matching answers in Tair. Once retrieved, the most similar answers will be incorporated into the prompt sent to OpenAI Large Language Model.



```python
import random

random.seed(52)
selected_questions = random.choices(questions, k=5)
```


```python
import time
for question in selected_questions:
    print(">", question)
    print(qa.run(question), end="\n\n")
    # wait 20seconds because of the rate limit
    time.sleep(20)
```

    > where do frankenstein and the monster first meet
     Frankenstein and the monster first meet in the mountains.
    
    > who are the actors in fast and furious
     The actors in Fast & Furious are Vin Diesel ( Dominic Toretto ), Paul Walker ( Brian O'Conner ), Michelle Rodriguez ( Letty Ortiz ), Jordana Brewster ( Mia Toretto ), Tyrese Gibson ( Roman Pearce ), Ludacris ( Tej Parker ), Lucas Black ( Sean Boswell ), Sung Kang ( Han Lue ), Gal Gadot ( Gisele Yashar ), and Dwayne Johnson ( Luke Hobbs ).
    
    > properties of red black tree in data structure
     The properties of a red-black tree in data structure are that each node is either red or black, the root is black, if a node is red then both its children must be black, and every path from a given node to any of its descendant NIL nodes contains the same number of black nodes.
    
    > who designed the national coat of arms of south africa
     Iaan Bekker
    
    > caravaggio's death of the virgin pamela askew
     I don't know.


key: _node_content
string_value:
{"id_": "61e2ead4-9fdd-4265-b1ce-9f1b38908482", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "73f5bc46-1e4e-40ad-9e49-c9f1ce71cac5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "5cedb6bceadb78fa0b5a322b6d405bad3d84114b8dec3af76cd1dd2b4db167ab", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "cdfdc0c0-352f-432e-befe-09923150ec54", "node_type": "1", "metadata": {}, "hash": "a4097d7fe51257a78f2fa074102ab57a3c50ae89aac7c47cf9103da6f7558847", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 15467, "end_char_idx": 18737, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
## Search data

Once the data is put into Tair we can start asking some questions. A question will be automatically vectorized by OpenAI model, and the created vector will be used to find some possibly matching answers in Tair. Once retrieved, the most similar answers will be incorporated into the prompt sent to OpenAI Large Language Model.



```python
import random

random.seed(52)
selected_questions = random.choices(questions, k=5)
```


```python
import time
for question in selected_questions:
    print(">", question)
    print(qa.run(question), end="\n\n")
    # wait 20seconds because of the rate limit
    time.sleep(20)
```

    > where do frankenstein and the monster first meet
     Frankenstein and the monster first meet in the mountains.
    
    > who are the actors in fast and furious
     The actors in Fast & Furious are Vin Diesel ( Dominic Toretto ), Paul Walker ( Brian O'Conner ), Michelle Rodriguez ( Letty Ortiz ), Jordana Brewster ( Mia Toretto ), Tyrese Gibson ( Roman Pearce ), Ludacris ( Tej Parker ), Lucas Black ( Sean Boswell ), Sung Kang ( Han Lue ), Gal Gadot ( Gisele Yashar ), and Dwayne Johnson ( Luke Hobbs ).
    
    > properties of red black tree in data structure
     The properties of a red-black tree in data structure are that each node is either red or black, the root is black, if a node is red then both its children must be black, and every path from a given node to any of its descendant NIL nodes contains the same number of black nodes.
    
    > who designed the national coat of arms of south africa
     Iaan Bekker
    
    > caravaggio's death of the virgin pamela askew
     I don't know.
    


### Custom prompt templates

The `stuff` chain type in Langchain uses a specific prompt with question and context documents incorporated. This is what the default prompt looks like:

```text
Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:
```

We can, however, provide our prompt template and change the behaviour of the OpenAI LLM, while still using the `stuff` chain type. It is important to keep `{context}` and `{question}` as placeholders.

#### Experimenting with custom prompts

We can try using a different prompt template, so the model:
1. Responds with a single-sentence answer if it knows it.
2. Suggests a random song title if it doesn't know the answer to our question.


```python
from langchain.prompts import PromptTemplate
custom_prompt = """
Use the following pieces of context to answer the question at the end. Please provide
a short single-sentence summary answer only. If you don't know the answer or if it's
not present in given context, don't try to make up an answer, but suggest me a random
unrelated song title I could listen to.
Context: {context}
Question: {question}
Helpful Answer:
"""

custom_prompt_template = PromptTemplate(
    template=custom_prompt, input_variables=["context", "question"]
)
```


```python
custom_qa = VectorDBQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    vectorstore=doc_store,
    return_source_documents=False,
    chain_type_kwargs={"prompt": custom_prompt_template},
)
```


key: _node_content
string_value:
{"id_": "cdfdc0c0-352f-432e-befe-09923150ec54", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "435ab317-9592-4483-bab8-dd267250f583", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "55d43c30c659e1a4e81b87662f031e0c5dfd2aa0f4d791ff084dda636c803fad", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "61e2ead4-9fdd-4265-b1ce-9f1b38908482", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/tair/QA_with_Langchain_Tair_and_OpenAI.ipynb", "title": "Question Answering with Langchain, Tair and OpenAI", "tokens": 5196, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "309a222d30dd9a8ce7b43586bff8ce888dbe0c60970e3a8ed27bb3443abc2ccb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 17128, "end_char_idx": 19882, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
### Custom prompt templates

The `stuff` chain type in Langchain uses a specific prompt with question and context documents incorporated. This is what the default prompt looks like:

```text
Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:
```

We can, however, provide our prompt template and change the behaviour of the OpenAI LLM, while still using the `stuff` chain type. It is important to keep `{context}` and `{question}` as placeholders.

#### Experimenting with custom prompts

We can try using a different prompt template, so the model:
1. Responds with a single-sentence answer if it knows it.
2. Suggests a random song title if it doesn't know the answer to our question.


```python
from langchain.prompts import PromptTemplate
custom_prompt = """
Use the following pieces of context to answer the question at the end. Please provide
a short single-sentence summary answer only. If you don't know the answer or if it's
not present in given context, don't try to make up an answer, but suggest me a random
unrelated song title I could listen to.
Context: {context}
Question: {question}
Helpful Answer:
"""

custom_prompt_template = PromptTemplate(
    template=custom_prompt, input_variables=["context", "question"]
)
```


```python
custom_qa = VectorDBQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    vectorstore=doc_store,
    return_source_documents=False,
    chain_type_kwargs={"prompt": custom_prompt_template},
)
```


```python
random.seed(41)
for question in random.choices(questions, k=5):
    print(">", question)
    print(custom_qa.run(question), end="\n\n")
    # wait 20seconds because of the rate limit
    time.sleep(20)
```

    > what was uncle jesse's original last name on full house
    Uncle Jesse's original last name on Full House was Cochran.
    
    > when did the volcano erupt in indonesia 2018
    The given context does not mention any volcanic eruption in Indonesia in 2018. Suggested song title: "The Heat Is On" by Glenn Frey.
    
    > what does a dualist way of thinking mean
    Dualism means the belief that there is a distinction between the mind and the body, and that the mind is a non-extended, non-physical substance.
    
    > the first civil service commission in india was set up on the basis of recommendation of
    The first Civil Service Commission in India was not set up on the basis of the recommendation of the Election Commission of India's Model Code of Conduct.
    
    > how old do you have to be to get a tattoo in utah
    You must be at least 18 years old to get a tattoo in Utah.
    



```python

```


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb


key: _node_content
string_value:
{"id_": "ca9ab4ba-df54-4733-a10b-1476a0a9d7e3", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9c57d12c-75df-4ad2-9361-8d6499a9f6e7", "node_type": "1", "metadata": {}, "hash": "85bbd20ac6d4674e6e5c8fa079a7d47157aed9a3937729352a6e129bcd5de8a9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2638, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Retrieval augmented generation using Elasticsearch and OpenAI

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb)


This notebook demonstrates how to: 
- Index the OpenAI Wikipedia vector dataset into Elasticsearch 
- Embed a question with the OpenAI [`embeddings`](https://platform.openai.com/docs/api-reference/embeddings) endpoint
- Perform semantic search on the Elasticsearch index using the encoded question
- Send the top search results to the OpenAI [Chat Completions](https://platform.openai.com/docs/guides/gpt/chat-completions-api) API endpoint for retrieval augmented generation (RAG)

ℹ️ If you've already worked through our semantic search notebook, you can skip ahead to the final step!

## Install packages and import modules 


```python
# install packages

!python3 -m pip install -qU openai pandas wget elasticsearch

# import modules

from getpass import getpass
from elasticsearch import Elasticsearch, helpers
import wget
import zipfile
import pandas as pd
import json
import openai
```

## Connect to Elasticsearch

ℹ️ We're using an Elastic Cloud deployment of Elasticsearch for this notebook.
If you don't already have an Elastic deployment, you can sign up for a free [Elastic Cloud trial](https://cloud.elastic.co/registration?utm_source=github&utm_content=openai-cookbook).

To connect to Elasticsearch, you need to create a client instance with the Cloud ID and password for your deployment.

Find the Cloud ID for your deployment by going to https://cloud.elastic.co/deployments and selecting your deployment.


```python
CLOUD_ID = getpass("Elastic deployment Cloud ID")
CLOUD_PASSWORD = getpass("Elastic deployment Password")
client = Elasticsearch(
  cloud_id = CLOUD_ID,
  basic_auth=("elastic", CLOUD_PASSWORD) # Alternatively use `api_key` instead of `basic_auth`
)

# Test connection to Elasticsearch
print(client.info())
```

    {'name': 'instance-0000000001', 'cluster_name': '29ef9817e13142f5ba0ea7b29c2a86e2', 'cluster_uuid': 'absjWgQvRw63IlwWKisN8w', 'version': {'number': '8.9.1', 'build_flavor': 'default', 'build_type': 'docker', 'build_hash': 'a813d015ef1826148d9d389bd1c0d781c6e349f0', 'build_date': '2023-08-10T05:02:32.517455352Z', 'build_snapshot': False, 'lucene_version': '9.7.0', 'minimum_wire_compatibility_version': '7.17.0', 'minimum_index_compatibility_version': '7.0.0'}, 'tagline': 'You Know, for Search'}


## Download the dataset 

In this step we download the OpenAI Wikipedia embeddings dataset, and extract the zip file.


key: _node_content
string_value:
{"id_": "9c57d12c-75df-4ad2-9361-8d6499a9f6e7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ca9ab4ba-df54-4733-a10b-1476a0a9d7e3", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "4b600156b63cc6d53657d0ca451f79d7524cdb1c50f9dae2ef603e763d12ef87", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f953dab2-0947-4892-8064-d5787516d172", "node_type": "1", "metadata": {}, "hash": "31c3dcf3bbea8bddaaddbd3810fffb5f6c399d0603a0fa490f0dcf81a977d41d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1695, "end_char_idx": 4535, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
CLOUD_ID = getpass("Elastic deployment Cloud ID")
CLOUD_PASSWORD = getpass("Elastic deployment Password")
client = Elasticsearch(
  cloud_id = CLOUD_ID,
  basic_auth=("elastic", CLOUD_PASSWORD) # Alternatively use `api_key` instead of `basic_auth`
)

# Test connection to Elasticsearch
print(client.info())
```

    {'name': 'instance-0000000001', 'cluster_name': '29ef9817e13142f5ba0ea7b29c2a86e2', 'cluster_uuid': 'absjWgQvRw63IlwWKisN8w', 'version': {'number': '8.9.1', 'build_flavor': 'default', 'build_type': 'docker', 'build_hash': 'a813d015ef1826148d9d389bd1c0d781c6e349f0', 'build_date': '2023-08-10T05:02:32.517455352Z', 'build_snapshot': False, 'lucene_version': '9.7.0', 'minimum_wire_compatibility_version': '7.17.0', 'minimum_index_compatibility_version': '7.0.0'}, 'tagline': 'You Know, for Search'}


## Download the dataset 

In this step we download the OpenAI Wikipedia embeddings dataset, and extract the zip file.


```python
embeddings_url = 'https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip'
wget.download(embeddings_url)

with zipfile.ZipFile("vector_database_wikipedia_articles_embedded.zip",
"r") as zip_ref:
    zip_ref.extractall("data")
```

##  Read CSV file into a Pandas DataFrame.

Next we use the Pandas library to read the unzipped CSV file into a DataFrame. This step makes it easier to index the data into Elasticsearch in bulk.


```python

wikipedia_dataframe = pd.read_csv("data/vector_database_wikipedia_articles_embedded.csv")
```

## Create index with mapping

Now we need to create an Elasticsearch index with the necessary mappings. This will enable us to index the data into Elasticsearch.

We use the `dense_vector` field type for the `title_vector` and  `content_vector` fields. This is a special field type that allows us to store dense vectors in Elasticsearch.

Later, we'll need to target the `dense_vector` field for kNN search.



```python
index_mapping= {
    "properties": {
      "title_vector": {
          "type": "dense_vector",
          "dims": 1536,
          "index": "true",
          "similarity": "cosine"
      },
      "content_vector": {
          "type": "dense_vector",
          "dims": 1536,
          "index": "true",
          "similarity": "cosine"
      },
      "text": {"type": "text"},
      "title": {"type": "text"},
      "url": { "type": "keyword"},
      "vector_id": {"type": "long"}
      
    }
}

client.indices.create(index="wikipedia_vector_index", mappings=index_mapping)
```

## Index data into Elasticsearch 

The following function generates the required bulk actions that can be passed to Elasticsearch's Bulk API, so we can index multiple documents efficiently in a single request.

For each row in the DataFrame, the function yields a dictionary representing a single document to be indexed.


key: _node_content
string_value:
{"id_": "f953dab2-0947-4892-8064-d5787516d172", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9c57d12c-75df-4ad2-9361-8d6499a9f6e7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "4e5e65201bec934cde37782e1c60813b2796b1a6b83f5b54921781c00dd5e964", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "303b2957-c1e7-4054-bbc3-98c695e0541b", "node_type": "1", "metadata": {}, "hash": "1136b25c0f517150b6d76193f73b46366aa209a4b6ba0d3d563e65713f8e4a63", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3112, "end_char_idx": 5714, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python

wikipedia_dataframe = pd.read_csv("data/vector_database_wikipedia_articles_embedded.csv")
```

## Create index with mapping

Now we need to create an Elasticsearch index with the necessary mappings. This will enable us to index the data into Elasticsearch.

We use the `dense_vector` field type for the `title_vector` and  `content_vector` fields. This is a special field type that allows us to store dense vectors in Elasticsearch.

Later, we'll need to target the `dense_vector` field for kNN search.



```python
index_mapping= {
    "properties": {
      "title_vector": {
          "type": "dense_vector",
          "dims": 1536,
          "index": "true",
          "similarity": "cosine"
      },
      "content_vector": {
          "type": "dense_vector",
          "dims": 1536,
          "index": "true",
          "similarity": "cosine"
      },
      "text": {"type": "text"},
      "title": {"type": "text"},
      "url": { "type": "keyword"},
      "vector_id": {"type": "long"}
      
    }
}

client.indices.create(index="wikipedia_vector_index", mappings=index_mapping)
```

## Index data into Elasticsearch 

The following function generates the required bulk actions that can be passed to Elasticsearch's Bulk API, so we can index multiple documents efficiently in a single request.

For each row in the DataFrame, the function yields a dictionary representing a single document to be indexed. 


```python
def dataframe_to_bulk_actions(df):
    for index, row in df.iterrows():
        yield {
            "_index": 'wikipedia_vector_index',
            "_id": row['id'],
            "_source": {
                'url' : row["url"],
                'title' : row["title"],
                'text' : row["text"],
                'title_vector' : json.loads(row["title_vector"]),
                'content_vector' : json.loads(row["content_vector"]),
                'vector_id' : row["vector_id"]
            }
        }
```

As the dataframe is large, we will index data in batches of `100`. We index the data into Elasticsearch using the Python client's [helpers](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/client-helpers.html#bulk-helpers) for the bulk API.


```python
start = 0
end = len(wikipedia_dataframe)
batch_size = 100
for batch_start in range(start, end, batch_size):
    batch_end = min(batch_start + batch_size, end)
    batch_dataframe = wikipedia_dataframe.iloc[batch_start:batch_end]
    actions = dataframe_to_bulk_actions(batch_dataframe)
    helpers.bulk(client, actions)
```

Let's test the index with a simple match query.


key: _node_content
string_value:
{"id_": "303b2957-c1e7-4054-bbc3-98c695e0541b", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f953dab2-0947-4892-8064-d5787516d172", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "8ac1cb516da3d1289bac9b2c9aae6cb6e9205b766747c9edd02478a7db6324a4", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b201a154-cb74-4176-9c63-3a37b65f9176", "node_type": "1", "metadata": {}, "hash": "0e706173ca3ca409ab690a01c492ebc1af9c506ab83eaeb32af24135becb4f17", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4539, "end_char_idx": 7528, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
def dataframe_to_bulk_actions(df):
    for index, row in df.iterrows():
        yield {
            "_index": 'wikipedia_vector_index',
            "_id": row['id'],
            "_source": {
                'url' : row["url"],
                'title' : row["title"],
                'text' : row["text"],
                'title_vector' : json.loads(row["title_vector"]),
                'content_vector' : json.loads(row["content_vector"]),
                'vector_id' : row["vector_id"]
            }
        }
```

As the dataframe is large, we will index data in batches of `100`. We index the data into Elasticsearch using the Python client's [helpers](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/client-helpers.html#bulk-helpers) for the bulk API.


```python
start = 0
end = len(wikipedia_dataframe)
batch_size = 100
for batch_start in range(start, end, batch_size):
    batch_end = min(batch_start + batch_size, end)
    batch_dataframe = wikipedia_dataframe.iloc[batch_start:batch_end]
    actions = dataframe_to_bulk_actions(batch_dataframe)
    helpers.bulk(client, actions)
```

Let's test the index with a simple match query.


```python
print(client.search(index="wikipedia_vector_index", body={
    "_source": {
        "excludes": ["title_vector", "content_vector"]
    },
    "query": {
        "match": {
            "text": {
                "query": "Hummingbird"
            }
        }
    }
}))
```

    {'took': 10, 'timed_out': False, '_shards': {'total': 1, 'successful': 1, 'skipped': 0, 'failed': 0}, 'hits': {'total': {'value': 4, 'relation': 'eq'}, 'max_score': 14.917897, 'hits': [{'_index': 'wikipedia_vector_index', '_id': '34227', '_score': 14.917897, '_source': {'url': 'https://simple.wikipedia.org/wiki/Hummingbird', 'title': 'Hummingbird', 'text': "Hummingbirds are small birds of the family Trochilidae.\n\nThey are among the smallest of birds: most species measure 7.5–13\xa0cm (3–5\xa0in). The smallest living bird species is the 2–5\xa0cm Bee Hummingbird. They can hover in mid-air by rapidly flapping their wings 12–80 times per second (depending on the species). They are also the only group of birds able to fly backwards. Their rapid wing beats do actually hum. They can fly at speeds over 15\xa0m/s (54\xa0km/h, 34\xa0mi/h).\n\nEating habits and pollination \nHummingbirds help flowers to pollinate, though most insects are best known for doing so. The hummingbird enjoys nectar, like the butterfly and other flower-loving insects, such as bees.\n\nHummingbirds do not have a good sense of smell; instead, they are attracted to color, especially the color red. Unlike the butterfly, the hummingbird hovers over the flower as it drinks nectar from it, like a moth. When it does so, it flaps its wings very quickly to stay in one place, which makes it look like a blur and also beats so fast it makes a humming sound. A hummingbird sometimes puts its whole head into the flower to drink the nectar properly.


key: _node_content
string_value:
{"id_": "b201a154-cb74-4176-9c63-3a37b65f9176", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "303b2957-c1e7-4054-bbc3-98c695e0541b", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f6661bd1ea1145340672c08369e6f124af36ec2ef9a68ac6c5068d49a0bb26e6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "94993dcc-0e81-4cba-ac09-3744f30a2f4a", "node_type": "1", "metadata": {}, "hash": "955a9ce0c51deed8620c945917c6226193f9afa3418da050b4646f7d7fa5527b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6507, "end_char_idx": 9560, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The smallest living bird species is the 2–5\xa0cm Bee Hummingbird. They can hover in mid-air by rapidly flapping their wings 12–80 times per second (depending on the species). They are also the only group of birds able to fly backwards. Their rapid wing beats do actually hum. They can fly at speeds over 15\xa0m/s (54\xa0km/h, 34\xa0mi/h).\n\nEating habits and pollination \nHummingbirds help flowers to pollinate, though most insects are best known for doing so. The hummingbird enjoys nectar, like the butterfly and other flower-loving insects, such as bees.\n\nHummingbirds do not have a good sense of smell; instead, they are attracted to color, especially the color red. Unlike the butterfly, the hummingbird hovers over the flower as it drinks nectar from it, like a moth. When it does so, it flaps its wings very quickly to stay in one place, which makes it look like a blur and also beats so fast it makes a humming sound. A hummingbird sometimes puts its whole head into the flower to drink the nectar properly. When it takes its head back out, its head is covered with yellow pollen, so that when it moves to another flower, it can pollinate. Or sometimes it may pollinate with its beak.\n\nLike bees, hummingbirds can assess the amount of sugar in the nectar they eat. They reject flowers whose nectar has less than 10% sugar. Nectar is a poor source of nutrients, so hummingbirds meet their needs for protein, amino acids, vitamins, minerals, etc. by preying on insects and spiders.\n\nFeeding apparatus \nMost hummingbirds have bills that are long and straight or nearly so, but in some species the bill shape is adapted for specialized feeding. Thornbills have short, sharp bills adapted for feeding from flowers with short corollas and piercing the bases of longer ones. The Sicklebills' extremely decurved bills are adapted to extracting nectar from the curved corollas of flowers in the family Gesneriaceae. The bill of the Fiery-tailed Awlbill has an upturned tip, as in the Avocets. The male Tooth-billed Hummingbird has barracuda-like spikes at the tip of its long, straight bill.\n\nThe two halves of a hummingbird's bill have a pronounced overlap, with the lower half (mandible) fitting tightly inside the upper half (maxilla). When hummingbirds feed on nectar, the bill is usually only opened slightly, allowing the tongue to dart out into the nectar.\n\nLike the similar nectar-feeding sunbirds and unlike other birds, hummingbirds drink by using grooved or trough-like tongues which they can stick out a long way.\nHummingbirds do not spend all day flying, as the energy cost would be prohibitive; the majority of their activity consists simply of sitting or perching. Hummingbirds feed in many small meals, consuming many small invertebrates and up to twelve times their own body weight in nectar each day. They spend an average of 10–15% of their time feeding and 75–80% sitting and digesting.\n\nCo-evolution with flowers\n\nSince hummingbirds are specialized nectar-eaters, they are tied to the bird-flowers they feed upon.


key: _node_content
string_value:
{"id_": "94993dcc-0e81-4cba-ac09-3744f30a2f4a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b201a154-cb74-4176-9c63-3a37b65f9176", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "023d626890d6f1aa24235b163b1a9c9cdba50fc9d27288d39c57ee5b9305ef0d", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "39bbbc23-eae0-46ed-9a14-2b3cf93fef3c", "node_type": "1", "metadata": {}, "hash": "7d34a5cf04c2e708f27695542d65716e663adc8ea99304519daed92f7acf85a8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7968, "end_char_idx": 10909, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
by preying on insects and spiders.\n\nFeeding apparatus \nMost hummingbirds have bills that are long and straight or nearly so, but in some species the bill shape is adapted for specialized feeding. Thornbills have short, sharp bills adapted for feeding from flowers with short corollas and piercing the bases of longer ones. The Sicklebills' extremely decurved bills are adapted to extracting nectar from the curved corollas of flowers in the family Gesneriaceae. The bill of the Fiery-tailed Awlbill has an upturned tip, as in the Avocets. The male Tooth-billed Hummingbird has barracuda-like spikes at the tip of its long, straight bill.\n\nThe two halves of a hummingbird's bill have a pronounced overlap, with the lower half (mandible) fitting tightly inside the upper half (maxilla). When hummingbirds feed on nectar, the bill is usually only opened slightly, allowing the tongue to dart out into the nectar.\n\nLike the similar nectar-feeding sunbirds and unlike other birds, hummingbirds drink by using grooved or trough-like tongues which they can stick out a long way.\nHummingbirds do not spend all day flying, as the energy cost would be prohibitive; the majority of their activity consists simply of sitting or perching. Hummingbirds feed in many small meals, consuming many small invertebrates and up to twelve times their own body weight in nectar each day. They spend an average of 10–15% of their time feeding and 75–80% sitting and digesting.\n\nCo-evolution with flowers\n\nSince hummingbirds are specialized nectar-eaters, they are tied to the bird-flowers they feed upon. Some species, especially those with unusual bill shapes such as the Sword-billed Hummingbird and the sicklebills, are co-evolved with a small number of flower species.\n\nMany plants pollinated by hummingbirds produce flowers in shades of red, orange, and bright pink, though the birds will take nectar from flowers of many colors. Hummingbirds can see wavelengths into the near-ultraviolet. However, their flowers do not reflect these wavelengths as many insect-pollinated flowers do. The narrow color spectrum may make hummingbird-pollinated flowers inconspicuous to insects, thereby reducing nectar robbing by insects. Hummingbird-pollinated flowers also produce relatively weak nectar (averaging 25% sugars w/w) containing high concentrations of sucrose, whereas insect-pollinated flowers typically produce more concentrated nectars dominated by fructose and glucose.\n\nTaxonomy \nHummingbirds have traditionally been a part of the bird order Apodiformes. This order includes the hummingbirds, the swifts and the tree swifts. The Sibley-Ahlquist taxonomy of birds, based on DNA studies done in the 1970s and 1980s, changed the classification of hummingbirds. Instead of being in the same order as the swifts, the hummingbirds were made an order, the Trochiliformes. Their previous order, Apodiformes was changed to the superorder Apodimorphae.


key: _node_content
string_value:
{"id_": "39bbbc23-eae0-46ed-9a14-2b3cf93fef3c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "94993dcc-0e81-4cba-ac09-3744f30a2f4a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "263abfafe45006f13b0ebf851a08975cceb5b5385ede42dd45d0e528890286ff", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c6212178-ac47-49bf-80ef-8f6d50ed72d0", "node_type": "1", "metadata": {}, "hash": "6e48e53a858e28fb475bfa0f70ad1a5468b51d8b8001d092960fcf8f6f47db98", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9202, "end_char_idx": 11894, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Hummingbirds feed in many small meals, consuming many small invertebrates and up to twelve times their own body weight in nectar each day. They spend an average of 10–15% of their time feeding and 75–80% sitting and digesting.\n\nCo-evolution with flowers\n\nSince hummingbirds are specialized nectar-eaters, they are tied to the bird-flowers they feed upon. Some species, especially those with unusual bill shapes such as the Sword-billed Hummingbird and the sicklebills, are co-evolved with a small number of flower species.\n\nMany plants pollinated by hummingbirds produce flowers in shades of red, orange, and bright pink, though the birds will take nectar from flowers of many colors. Hummingbirds can see wavelengths into the near-ultraviolet. However, their flowers do not reflect these wavelengths as many insect-pollinated flowers do. The narrow color spectrum may make hummingbird-pollinated flowers inconspicuous to insects, thereby reducing nectar robbing by insects. Hummingbird-pollinated flowers also produce relatively weak nectar (averaging 25% sugars w/w) containing high concentrations of sucrose, whereas insect-pollinated flowers typically produce more concentrated nectars dominated by fructose and glucose.\n\nTaxonomy \nHummingbirds have traditionally been a part of the bird order Apodiformes. This order includes the hummingbirds, the swifts and the tree swifts. The Sibley-Ahlquist taxonomy of birds, based on DNA studies done in the 1970s and 1980s, changed the classification of hummingbirds. Instead of being in the same order as the swifts, the hummingbirds were made an order, the Trochiliformes. Their previous order, Apodiformes was changed to the superorder Apodimorphae. This superorder contains the three families of birds which were in it when it was an order.\n\nReferences", 'vector_id': 10024}}, {'_index': 'wikipedia_vector_index', '_id': '84773', '_score': 10.951234, '_source': {'url': 'https://simple.wikipedia.org/wiki/Inagua', 'title': 'Inagua', 'text': "Inagua is the southernmost district of the Bahamas.  It is the islands of Great Inagua and Little Inagua.\n\nGreat Inagua is the third largest island in the Bahamas at 596 square miles (1544\xa0km²) and lies about 55 miles (90\xa0km) from the eastern tip of Cuba. The island is about 55 × 19 miles (90 × 30\xa0km) in extent, the highest point being 108\xa0ft (33 m) on East Hill. It encloses several lakes, most notably the 12-mile long Lake Windsor (also called Lake Rosa) which occupies nearly ¼ of the interior. The population of Great Inagua is 969 (2000 census).\n\nThe island's capital and only harbour is Matthew Town.\n\nThere is a large bird sanctuary in the centre of the island.


key: _node_content
string_value:
{"id_": "c6212178-ac47-49bf-80ef-8f6d50ed72d0", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "39bbbc23-eae0-46ed-9a14-2b3cf93fef3c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "ce2140ab13c6d3cc3abcbf60771148a35618aa5335d006688b6c75397ba5a13a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d0526638-133b-4428-99f6-6ad5e9c8cf0a", "node_type": "1", "metadata": {}, "hash": "3e975620fb2050ded3e23972ae210155e3f74e86ac7cb405cc063bd9477c2fb8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 10522, "end_char_idx": 13359, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
This order includes the hummingbirds, the swifts and the tree swifts. The Sibley-Ahlquist taxonomy of birds, based on DNA studies done in the 1970s and 1980s, changed the classification of hummingbirds. Instead of being in the same order as the swifts, the hummingbirds were made an order, the Trochiliformes. Their previous order, Apodiformes was changed to the superorder Apodimorphae. This superorder contains the three families of birds which were in it when it was an order.\n\nReferences", 'vector_id': 10024}}, {'_index': 'wikipedia_vector_index', '_id': '84773', '_score': 10.951234, '_source': {'url': 'https://simple.wikipedia.org/wiki/Inagua', 'title': 'Inagua', 'text': "Inagua is the southernmost district of the Bahamas.  It is the islands of Great Inagua and Little Inagua.\n\nGreat Inagua is the third largest island in the Bahamas at 596 square miles (1544\xa0km²) and lies about 55 miles (90\xa0km) from the eastern tip of Cuba. The island is about 55 × 19 miles (90 × 30\xa0km) in extent, the highest point being 108\xa0ft (33 m) on East Hill. It encloses several lakes, most notably the 12-mile long Lake Windsor (also called Lake Rosa) which occupies nearly ¼ of the interior. The population of Great Inagua is 969 (2000 census).\n\nThe island's capital and only harbour is Matthew Town.\n\nThere is a large bird sanctuary in the centre of the island.  There are more than 80,000 West Indian Flamingoes and many other exotic birds such as the native Bahama Parrot, the Bahama woodstar hummingbird, Bahama pintails, Brown pelicans, Tri-colored herons, Snowy egrets, Reddish egrets, Stripe-headed tanangers, Cormorants, Roseate spoonbills, American kestrels, and Burrowing owls.\n\nDistricts of the Bahamas\nIslands of the Bahamas\n1999 establishments in the Bahamas", 'vector_id': 22383}}, {'_index': 'wikipedia_vector_index', '_id': '3707', '_score': 1.1967773, '_source': {'url': 'https://simple.wikipedia.org/wiki/Bird', 'title': 'Bird', 'text': 'Birds (Aves) are a group of animals with backbones which evolved from dinosaurs. Technically speaking, they are dinosaurs. \n\nBirds are endothermic. The heat loss from their bodies is slowed down by their feathers. \nModern birds are toothless: they have beaked jaws. They lay hard-shelled eggs. They have a high metabolic rate, a four-chambered heart and a strong yet lightweight skeleton.\n\nBirds live all over the world. They range in size from the 5 cm (2 in) bee hummingbird to the 2.70 m (9 ft) ostrich. They are the tetrapods with the most living species: about ten thousand. More than half of these are passerines, sometimes known as perching birds.\n\nBirds are the closest living relatives of the Crocodilia. This is because they are the two main survivors of a once huge group called the Archosaurs. \n\nModern birds are not descended from Archaeopteryx.


key: _node_content
string_value:
{"id_": "d0526638-133b-4428-99f6-6ad5e9c8cf0a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c6212178-ac47-49bf-80ef-8f6d50ed72d0", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08f88d6b1ce7e2418cb2082c1d0db20d819f8d072c11546e1529868a84152723", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d27c518f-8fe0-4b7c-9340-99a65237c149", "node_type": "1", "metadata": {}, "hash": "ab69c4204450f48c1bb5a2fccc13a8b0bddc0afdc487996143dce1f2682bf60b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11896, "end_char_idx": 14994, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
There are more than 80,000 West Indian Flamingoes and many other exotic birds such as the native Bahama Parrot, the Bahama woodstar hummingbird, Bahama pintails, Brown pelicans, Tri-colored herons, Snowy egrets, Reddish egrets, Stripe-headed tanangers, Cormorants, Roseate spoonbills, American kestrels, and Burrowing owls.\n\nDistricts of the Bahamas\nIslands of the Bahamas\n1999 establishments in the Bahamas", 'vector_id': 22383}}, {'_index': 'wikipedia_vector_index', '_id': '3707', '_score': 1.1967773, '_source': {'url': 'https://simple.wikipedia.org/wiki/Bird', 'title': 'Bird', 'text': 'Birds (Aves) are a group of animals with backbones which evolved from dinosaurs. Technically speaking, they are dinosaurs. \n\nBirds are endothermic. The heat loss from their bodies is slowed down by their feathers. \nModern birds are toothless: they have beaked jaws. They lay hard-shelled eggs. They have a high metabolic rate, a four-chambered heart and a strong yet lightweight skeleton.\n\nBirds live all over the world. They range in size from the 5 cm (2 in) bee hummingbird to the 2.70 m (9 ft) ostrich. They are the tetrapods with the most living species: about ten thousand. More than half of these are passerines, sometimes known as perching birds.\n\nBirds are the closest living relatives of the Crocodilia. This is because they are the two main survivors of a once huge group called the Archosaurs. \n\nModern birds are not descended from Archaeopteryx. According to DNA evidence, modern birds (Neornithes) evolved in the long Upper Cretaceous period. More recent estimates showed that  modern birds originated early in the Upper Cretaceous.\n\nPrimitive bird-like dinosaurs are in the broader group Avialae. They have been found back to the mid-Jurassic period, around 170 million years ago. Many of these early "stem-birds", such as Anchiornis, were not yet capable of fully powered flight. Many had primitive characteristics like teeth in their jaws and long bony tails.p274\n\nThe Cretaceous–Palaeogene extinction event 66 million years ago killed off all the non-avian dinosaur lines. Birds, especially those in the southern continents, survived this event and then migrated to other parts of the world. Diversification occurred around the Cretaceous–Palaeogene extinction event. \n\nBirds have wings which are more or less developed depending on the species. The only known groups without wings are the extinct moa and elephant birds. Wings, which evolved from forelimbs, gave birds the ability to fly. Later, many groups evolved with reduced wings, such as ratites, penguins and many island species of birds. The digestive and respiratory systems of birds are also adapted for flight. Some bird species in aquatic environments, particularly seabirds and some waterbirds, have evolved as good swimmers.\n\nIn general, birds are effective, and inherit their behaviour almost entirely. The key elements of their life are inherited. It was a great discovery that birds never learn to fly. \nSo it is quite wrong to say, when a chick waves its wings in the nest "It\'s learning to fly".


key: _node_content
string_value:
{"id_": "d27c518f-8fe0-4b7c-9340-99a65237c149", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d0526638-133b-4428-99f6-6ad5e9c8cf0a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "bda659740fa8c3912ef15621d70264bea067fe39c22cff5d141c6dc611d226fb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d38cef85-465e-458b-a73e-166b6bd6b42c", "node_type": "1", "metadata": {}, "hash": "9aea84059c02862693de4eb49d6b6b888921cc673133d536c5b97228c22ec7f8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 13305, "end_char_idx": 16701, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
\n\nModern birds are not descended from Archaeopteryx. According to DNA evidence, modern birds (Neornithes) evolved in the long Upper Cretaceous period. More recent estimates showed that  modern birds originated early in the Upper Cretaceous.\n\nPrimitive bird-like dinosaurs are in the broader group Avialae. They have been found back to the mid-Jurassic period, around 170 million years ago. Many of these early "stem-birds", such as Anchiornis, were not yet capable of fully powered flight. Many had primitive characteristics like teeth in their jaws and long bony tails.p274\n\nThe Cretaceous–Palaeogene extinction event 66 million years ago killed off all the non-avian dinosaur lines. Birds, especially those in the southern continents, survived this event and then migrated to other parts of the world. Diversification occurred around the Cretaceous–Palaeogene extinction event. \n\nBirds have wings which are more or less developed depending on the species. The only known groups without wings are the extinct moa and elephant birds. Wings, which evolved from forelimbs, gave birds the ability to fly. Later, many groups evolved with reduced wings, such as ratites, penguins and many island species of birds. The digestive and respiratory systems of birds are also adapted for flight. Some bird species in aquatic environments, particularly seabirds and some waterbirds, have evolved as good swimmers.\n\nIn general, birds are effective, and inherit their behaviour almost entirely. The key elements of their life are inherited. It was a great discovery that birds never learn to fly. \nSo it is quite wrong to say, when a chick waves its wings in the nest "It\'s learning to fly". What the chick is doing is exercising its muscles. They develop the ability to fly automatically (assuming they are species that do fly). And if they are species which migrate, that behaviour is also inherited. Many species migrate over great distances each year. Other main features of their life may be inherited, though they can and do learn. Birds have good memories which they use, for example, when they search for food.\n\nSeveral bird species make and use tools. Some social species pass on some knowledge across generations, a form of culture. Birds are social. They communicate with visual signals, calls and bird songs. Most of their social behaviours are inherited, such as cooperative breeding and hunting, flocking and mobbing of predators.\n\nMost bird species are socially monogamous, usually for one breeding season at a time, sometimes for years, but rarely for life. Other species are polygynous (one male with many females) or, rarely, polyandrous (one female with many males). Birds produce offspring by laying eggs which are fertilised by sexual reproduction. They are often laid in a nest and incubated by the parents. Most birds have an extended period of parental care after hatching. Some birds, such as hens, lay eggs even when not fertilised, though unfertilised eggs do not produce offspring.\n\nMany species of birds are eaten by humans. Domesticated and undomesticated birds are sources of eggs, meat, and feathers.  In English, domesticated birds are often called poultry, undomesticated birds are called game. Songbirds, parrots and other species are popular as pets. Guano, which is bird manure, is harvested for use as a fertiliser. Birds figure throughout human culture.


key: _node_content
string_value:
{"id_": "d38cef85-465e-458b-a73e-166b6bd6b42c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d27c518f-8fe0-4b7c-9340-99a65237c149", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e438b9ed3ce12a1ed6353b5133c0a22faacebf1cc9e4d651806b38f0a1b44f9d", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "4828af32-18ac-4bfc-bf2f-47b61a0db714", "node_type": "1", "metadata": {}, "hash": "4ff555ebb801278dd8d3231db2b6933ee708a27c6a90d9a3ba404c5ceb460fff", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 14995, "end_char_idx": 18286, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
What the chick is doing is exercising its muscles. They develop the ability to fly automatically (assuming they are species that do fly). And if they are species which migrate, that behaviour is also inherited. Many species migrate over great distances each year. Other main features of their life may be inherited, though they can and do learn. Birds have good memories which they use, for example, when they search for food.\n\nSeveral bird species make and use tools. Some social species pass on some knowledge across generations, a form of culture. Birds are social. They communicate with visual signals, calls and bird songs. Most of their social behaviours are inherited, such as cooperative breeding and hunting, flocking and mobbing of predators.\n\nMost bird species are socially monogamous, usually for one breeding season at a time, sometimes for years, but rarely for life. Other species are polygynous (one male with many females) or, rarely, polyandrous (one female with many males). Birds produce offspring by laying eggs which are fertilised by sexual reproduction. They are often laid in a nest and incubated by the parents. Most birds have an extended period of parental care after hatching. Some birds, such as hens, lay eggs even when not fertilised, though unfertilised eggs do not produce offspring.\n\nMany species of birds are eaten by humans. Domesticated and undomesticated birds are sources of eggs, meat, and feathers.  In English, domesticated birds are often called poultry, undomesticated birds are called game. Songbirds, parrots and other species are popular as pets. Guano, which is bird manure, is harvested for use as a fertiliser. Birds figure throughout human culture. About 120–130 species have become extinct due to human activity since the 17th century and hundreds more before then. Human activity threatens about 1,200 bird species with extinction, though efforts are underway to protect them. Recreational bird-watching is an important part of the ecotourism industry.\n\nBird colours \n\nBirds come in a huge range of colours. These colours can be useful to a bird in two ways. Camouflage colours help to hide the bird, and bright colours identify the bird to others of the same species. Often the male is brightly coloured while the female is camouflaged. The logic is as follows: the female carries the "precious package" of developing eggs. The male has to defend a territory, and the function of his colour and song is to let others know that "this place is occupied".\n\nBird camouflage \n\nMany birds are brown, green or grey. These colours make a bird harder to be seen: they camouflage the bird. Brown is the most common colour. Brown birds include: sparrows, emus, thrushes, larks, eagles and falcons and the female birds of many species such as: wrens, ducks, blackbirds and peafowls. When a brown bird is in long grass or among tree trunks or rocks, it is camouflaged. Birds that live in long grass often have brown feathers streaked with black which looks like shadows. A bittern is almost invisible in long reeds because its camouflage is helped by its posture (beak and head pointed upwards). Other birds, including starlings and mynas, are quite dark in colour, but are flecked with little spots that look like raindrops on leaves.


key: _node_content
string_value:
{"id_": "4828af32-18ac-4bfc-bf2f-47b61a0db714", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d38cef85-465e-458b-a73e-166b6bd6b42c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d90b1f78521ffbf6050aa66435ca891b1a924f14d3a49048f82c9a4e233217c5", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "cd84cf44-b37d-43f4-9271-11d558d020b8", "node_type": "1", "metadata": {}, "hash": "337e29a804c5cc9d98cec310af26dabaafc096b2953dd915214f4baa56a95661", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 16663, "end_char_idx": 19827, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Birds figure throughout human culture. About 120–130 species have become extinct due to human activity since the 17th century and hundreds more before then. Human activity threatens about 1,200 bird species with extinction, though efforts are underway to protect them. Recreational bird-watching is an important part of the ecotourism industry.\n\nBird colours \n\nBirds come in a huge range of colours. These colours can be useful to a bird in two ways. Camouflage colours help to hide the bird, and bright colours identify the bird to others of the same species. Often the male is brightly coloured while the female is camouflaged. The logic is as follows: the female carries the "precious package" of developing eggs. The male has to defend a territory, and the function of his colour and song is to let others know that "this place is occupied".\n\nBird camouflage \n\nMany birds are brown, green or grey. These colours make a bird harder to be seen: they camouflage the bird. Brown is the most common colour. Brown birds include: sparrows, emus, thrushes, larks, eagles and falcons and the female birds of many species such as: wrens, ducks, blackbirds and peafowls. When a brown bird is in long grass or among tree trunks or rocks, it is camouflaged. Birds that live in long grass often have brown feathers streaked with black which looks like shadows. A bittern is almost invisible in long reeds because its camouflage is helped by its posture (beak and head pointed upwards). Other birds, including starlings and mynas, are quite dark in colour, but are flecked with little spots that look like raindrops on leaves. Bird may also camouflage their nests.\n\nMany birds from hot countries are green or have some green feathers, particularly parrots. Birds that live in green trees often have green backs, even if they have bright-coloured breasts. From the back, the birds are camouflaged. This is very useful when sitting on a nest. The bird\'s bright-coloured breast is hidden. Budgerigars are bred in different colours such as blue, white and mauve, but in the wild, they are nearly all green and yellow. Even though they fly very well, they normally spend a lot of time on the ground, eating grass seeds. Their yellow and black striped back helps to hide them in the shadows made by long dry grass, while their green breasts are a similar colour to the leaves of gum trees.\n\nGrey birds include most pigeons and doves, cranes, storks and herons. Grey birds are often rock-living birds like pigeons or birds that sit on dead tree trunks looking like a broken branch. Water birds like herons often have a pale grey colour which makes it harder for a fish to notice that the bird is standing, looking down for something to catch. Water birds, no matter what colour they are on top, are often white underneath, so that when a fish looks up, the bird looks like part of the sky.\n\nBlack birds include crows, ravens and male blackbirds. Some birds that are dark colours spend quite a lot of time on the ground, hopping around in the shadows under bushes. Among these birds are the male blackbird and the satin bowerbird which is not black but very dark blue.


key: _node_content
string_value:
{"id_": "cd84cf44-b37d-43f4-9271-11d558d020b8", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "4828af32-18ac-4bfc-bf2f-47b61a0db714", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9bebfa000cc5bcdcb47de06bfe150293221a528464c2a1678cdf54296a957a43", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d5e64078-f175-49bb-8067-0b2edd6b5b5c", "node_type": "1", "metadata": {}, "hash": "5dfa12be929b945639a374052707ba49e87460935305eee7fa8f9c08665d642b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 18147, "end_char_idx": 21387, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Other birds, including starlings and mynas, are quite dark in colour, but are flecked with little spots that look like raindrops on leaves. Bird may also camouflage their nests.\n\nMany birds from hot countries are green or have some green feathers, particularly parrots. Birds that live in green trees often have green backs, even if they have bright-coloured breasts. From the back, the birds are camouflaged. This is very useful when sitting on a nest. The bird\'s bright-coloured breast is hidden. Budgerigars are bred in different colours such as blue, white and mauve, but in the wild, they are nearly all green and yellow. Even though they fly very well, they normally spend a lot of time on the ground, eating grass seeds. Their yellow and black striped back helps to hide them in the shadows made by long dry grass, while their green breasts are a similar colour to the leaves of gum trees.\n\nGrey birds include most pigeons and doves, cranes, storks and herons. Grey birds are often rock-living birds like pigeons or birds that sit on dead tree trunks looking like a broken branch. Water birds like herons often have a pale grey colour which makes it harder for a fish to notice that the bird is standing, looking down for something to catch. Water birds, no matter what colour they are on top, are often white underneath, so that when a fish looks up, the bird looks like part of the sky.\n\nBlack birds include crows, ravens and male blackbirds. Some birds that are dark colours spend quite a lot of time on the ground, hopping around in the shadows under bushes. Among these birds are the male blackbird and the satin bowerbird which is not black but very dark blue. Crows and ravens often perch high on bare trees in the winter, where their black shape against the sky looks like the dark bare branches.\n\nNoticeable colours \n\nMany birds are not camouflaged, but stand out with vivid colours. They are usually male birds whose females are dull and camouflaged. The function of the colours is two-fold. First, the colours help them get mates, and second, the colours identify them to other males of the same species. Many birds are territorial, especially in the nesting season. They give out territory sounds and are easily seen. This lets other males know they will defend their territory. It sends out a "look elsewhere" signal to their competitors.\n\nSome birds are famous for their colour and are named for it, such as the bluebird, the azure kingfisher, the golden pheasant, the scarlet macaw, the violet wren and the robin.\n\nMany other birds are very brightly coloured, in countless combinations. Some of the most colourful birds are quite common, like pheasants, peacocks, domestic fowl and parrots. Colourful small birds include blue tits, the goldfinches, hummingbirds, fairy wrens and bee eaters (which are also called rainbow birds). Some birds, like those of the bird of paradise in Papua New Guinea have such beautiful feathers that they have been hunted for them.\n\nThe peafowl is the best example of a display of colour to attract a mate. Also the male domestic fowl and junglefowl have long shiny feathers above his tail and also long neck feathers that may be a different colour to his wings and body.


key: _node_content
string_value:
{"id_": "d5e64078-f175-49bb-8067-0b2edd6b5b5c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "cd84cf44-b37d-43f4-9271-11d558d020b8", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c73ecfc83ec42d0c16e89d12c42e7347e6bd32c8f1f5c818d18793b39104254a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "17d2579b-abc5-47de-93de-96bf1fbd55b2", "node_type": "1", "metadata": {}, "hash": "341d7bdfcb58f7550480c1a6e979b41759a9e5687700fff611133c27cad35836", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 19724, "end_char_idx": 22985, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Among these birds are the male blackbird and the satin bowerbird which is not black but very dark blue. Crows and ravens often perch high on bare trees in the winter, where their black shape against the sky looks like the dark bare branches.\n\nNoticeable colours \n\nMany birds are not camouflaged, but stand out with vivid colours. They are usually male birds whose females are dull and camouflaged. The function of the colours is two-fold. First, the colours help them get mates, and second, the colours identify them to other males of the same species. Many birds are territorial, especially in the nesting season. They give out territory sounds and are easily seen. This lets other males know they will defend their territory. It sends out a "look elsewhere" signal to their competitors.\n\nSome birds are famous for their colour and are named for it, such as the bluebird, the azure kingfisher, the golden pheasant, the scarlet macaw, the violet wren and the robin.\n\nMany other birds are very brightly coloured, in countless combinations. Some of the most colourful birds are quite common, like pheasants, peacocks, domestic fowl and parrots. Colourful small birds include blue tits, the goldfinches, hummingbirds, fairy wrens and bee eaters (which are also called rainbow birds). Some birds, like those of the bird of paradise in Papua New Guinea have such beautiful feathers that they have been hunted for them.\n\nThe peafowl is the best example of a display of colour to attract a mate. Also the male domestic fowl and junglefowl have long shiny feathers above his tail and also long neck feathers that may be a different colour to his wings and body. There are only a very few types of birds (like the eclectus parrot) where the female is more colourful than the male.\n\n\'\'Pied birds\'\' are black and white. Black and white birds include magpies, pied geese, pelicans and Australian magpies (which are not really magpies at all). Pied birds often have brightly coloured beaks and legs of yellow or red. The silver pheasant, with its long white tail striped with fine bars of black, has a brightly coloured face.\n\nFlight \nMost birds can fly, and if they do, then the ability is inherited, not learnt. They fly by pushing through the air with their wings. The curved surfaces of the wings cause air currents (wind) which lift the bird. Flapping keeps the air current moving to create lift and also moves the bird forward.\n\nSome birds can glide on air currents without flapping. Many birds use this method when they are about to land. Some birds can also hover in the air. This method is used by birds of prey such as falcons that are looking for something to eat. Seagulls are also good at hovering, particularly if there is a strong breeze. The most expert hovering birds are tiny hummingbirds which can beat their wings both backwards and forwards and can stay quite still in the air while they dip their long beaks into flowers to feed on the sweet nectar.\n\nTypes of flight \nDifferent types of birds have different needs. Their wings have evolved to suit their lifestyle. Large birds of prey, such as eagles, spend a lot of time soaring on the wind. They have wings that are large and broad. The main flight feathers are long and wide.


key: _node_content
string_value:
{"id_": "17d2579b-abc5-47de-93de-96bf1fbd55b2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d5e64078-f175-49bb-8067-0b2edd6b5b5c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "70fe6ba05c2ab1c6f77379ff6c5efb968ec45f63a5e59532c8e926c6fabc65de", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "139a8869-0dee-4b05-853b-b855f686df69", "node_type": "1", "metadata": {}, "hash": "dfc87a44f1f5be6bd9e8d9d7beafde4cfad28080f5746a71a3577180d2924be4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 21388, "end_char_idx": 24503, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
There are only a very few types of birds (like the eclectus parrot) where the female is more colourful than the male.\n\n\'\'Pied birds\'\' are black and white. Black and white birds include magpies, pied geese, pelicans and Australian magpies (which are not really magpies at all). Pied birds often have brightly coloured beaks and legs of yellow or red. The silver pheasant, with its long white tail striped with fine bars of black, has a brightly coloured face.\n\nFlight \nMost birds can fly, and if they do, then the ability is inherited, not learnt. They fly by pushing through the air with their wings. The curved surfaces of the wings cause air currents (wind) which lift the bird. Flapping keeps the air current moving to create lift and also moves the bird forward.\n\nSome birds can glide on air currents without flapping. Many birds use this method when they are about to land. Some birds can also hover in the air. This method is used by birds of prey such as falcons that are looking for something to eat. Seagulls are also good at hovering, particularly if there is a strong breeze. The most expert hovering birds are tiny hummingbirds which can beat their wings both backwards and forwards and can stay quite still in the air while they dip their long beaks into flowers to feed on the sweet nectar.\n\nTypes of flight \nDifferent types of birds have different needs. Their wings have evolved to suit their lifestyle. Large birds of prey, such as eagles, spend a lot of time soaring on the wind. They have wings that are large and broad. The main flight feathers are long and wide. They help the eagle to stay on rising air currents without using much energy, while the eagle looks at the ground below, to find the next meal. When the eagle sees some small creature move, it can close its wings and fall from the sky like a missile, opening its great wings again to slow down as it comes to land. The world\'s largest eagle, the Philippine eagle has a wingspan of about 2 m (6.7\xa0ft) wide.\n\nBirds that live in grassland areas or open forests and feed on fruit, insects and reptiles often spend a lot of time flying short journeys looking for food and water. They have wings that are shaped in a similar way to eagles, but rounder and not as good for soaring. These include many Australian birds like cockatoos.\n\nBirds such as geese that migrate from one country to another fly very long distances. Their wings are big and strong, because the birds are large. They stock up on food for the long flight. Migrating water birds usually form family groups of 1230 birds. They fly very high, making use of long streams of air that blow from north to south in different seasons. They are well organised, often flying in a V pattern. The geese at the back do not have to flap so hard; they are pulled on by the wind of the ones at the front. Every so often, they change the leader so that the front bird, who does most work and sets the pace, can have a rest. Geese and swans are the highest-flying birds, reaching 8,000 metres or more when on migration. Geese often honk loudly while they are flying.


key: _node_content
string_value:
{"id_": "139a8869-0dee-4b05-853b-b855f686df69", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "17d2579b-abc5-47de-93de-96bf1fbd55b2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "2a6ebeacd25e42162c58400be274e7f2f16ca65b7c75a6e8cbda1a13af41aeab", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "83d38e1a-c054-44a5-bc76-98e3a84ea541", "node_type": "1", "metadata": {}, "hash": "2ef27cef2b7e94f25639be5d390f3855e597d32aed4da08299317d18f328f500", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 22900, "end_char_idx": 26177, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They have wings that are large and broad. The main flight feathers are long and wide. They help the eagle to stay on rising air currents without using much energy, while the eagle looks at the ground below, to find the next meal. When the eagle sees some small creature move, it can close its wings and fall from the sky like a missile, opening its great wings again to slow down as it comes to land. The world\'s largest eagle, the Philippine eagle has a wingspan of about 2 m (6.7\xa0ft) wide.\n\nBirds that live in grassland areas or open forests and feed on fruit, insects and reptiles often spend a lot of time flying short journeys looking for food and water. They have wings that are shaped in a similar way to eagles, but rounder and not as good for soaring. These include many Australian birds like cockatoos.\n\nBirds such as geese that migrate from one country to another fly very long distances. Their wings are big and strong, because the birds are large. They stock up on food for the long flight. Migrating water birds usually form family groups of 1230 birds. They fly very high, making use of long streams of air that blow from north to south in different seasons. They are well organised, often flying in a V pattern. The geese at the back do not have to flap so hard; they are pulled on by the wind of the ones at the front. Every so often, they change the leader so that the front bird, who does most work and sets the pace, can have a rest. Geese and swans are the highest-flying birds, reaching 8,000 metres or more when on migration. Geese often honk loudly while they are flying. It is thought that they do this to support the leader and help the young ones.\n\nBirds that fly very quickly, such as swifts and swallows, have long narrow pointed wings. These birds need great speed because they eat insects, catching most of them while they are flying. These birds also migrate. They often collect in huge flocks of thousands of birds that move together like a whirling cloud.\n\nBirds that live in bushes and branches have triangular wings that help the bird change direction. Many forest birds are expert at getting up speed by flapping and then gliding steadily among the trees, tilting to avoid things as they go. Members of the kingfisher family are expert at this type of flying.\n\nBirds such as owls that hunt at night have wings with soft rounded feathers so that they do not flap loudly. Birds that are awake at night are called nocturnal birds. Birds that are awake during the day are diurnal.\n\nWandering albatross might spend several years without coming to land. They can sleep while gliding. Arctic terns nest every one to three years.\n\nFlocks \nFlocks of birds can be very highly organised in a way that takes care of all the flock members. Studies of small flocking birds like tree sparrows show that they clearly communicate with each other, as sometimes thousands of birds may fly in close formation and spiral patterns without colliding (or flying into each other).\n\nTwo common behaviours in flocking birds are guarding and reconnaissance. When a flock of birds is feeding it is common for one bird to perch on a high place to keep guard over the flock. In the same way, when a flock is asleep, often, one bird will remain awake.


key: _node_content
string_value:
{"id_": "83d38e1a-c054-44a5-bc76-98e3a84ea541", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "139a8869-0dee-4b05-853b-b855f686df69", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "89da3e31c7fde2947fe608bfefe1e3729e65e28ea833f0e5c0f7e852032ea814", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "768b6896-31c6-4ebf-8151-3a42ace3f9c5", "node_type": "1", "metadata": {}, "hash": "b1b3bea047463d7ada3029bcfe50115b7bdafadc0e188410f219629e87b386e2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 24457, "end_char_idx": 27803, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Geese often honk loudly while they are flying. It is thought that they do this to support the leader and help the young ones.\n\nBirds that fly very quickly, such as swifts and swallows, have long narrow pointed wings. These birds need great speed because they eat insects, catching most of them while they are flying. These birds also migrate. They often collect in huge flocks of thousands of birds that move together like a whirling cloud.\n\nBirds that live in bushes and branches have triangular wings that help the bird change direction. Many forest birds are expert at getting up speed by flapping and then gliding steadily among the trees, tilting to avoid things as they go. Members of the kingfisher family are expert at this type of flying.\n\nBirds such as owls that hunt at night have wings with soft rounded feathers so that they do not flap loudly. Birds that are awake at night are called nocturnal birds. Birds that are awake during the day are diurnal.\n\nWandering albatross might spend several years without coming to land. They can sleep while gliding. Arctic terns nest every one to three years.\n\nFlocks \nFlocks of birds can be very highly organised in a way that takes care of all the flock members. Studies of small flocking birds like tree sparrows show that they clearly communicate with each other, as sometimes thousands of birds may fly in close formation and spiral patterns without colliding (or flying into each other).\n\nTwo common behaviours in flocking birds are guarding and reconnaissance. When a flock of birds is feeding it is common for one bird to perch on a high place to keep guard over the flock. In the same way, when a flock is asleep, often, one bird will remain awake. It is also common for large flocks to send one or two birds ahead of them when they are flying to a new area. The look-out birds can spy the lie of the land to find food, water and good places to perch. Mixed feeding flocks occur, and can help to spot predators.\n\nFlightless birds \nSome birds do not fly. Flightlessness in birds has evolved many times.\nThese include running birds like ostriches and emus and ocean-living birds, the large penguin family. Birds on islands have usually lost the power of flight. This is to their advantage because birds with the power of flight can be blown off their island during a storm. The same ability which got them to the island may later take them away in a storm.\n\nOstriches and emus do not need to fly because although they feed and nest on the ground, their great size and their speed is their protection. Some other ground-feeding birds have not been so lucky. Some birds such as the dodo and the kiwi were ground-feeding birds that lived in safety on islands where there was nothing dangerous to eat them. They lost the power of flight. Kiwis are endangered because European settlement to New Zealand brought animals like cats, dogs and rats which kill kiwis and eat their eggs. However, kiwis and also the rare New Zealand ground parrot have survived. In the case of dodos, they were fat and disgusting in taste. All the same, they were killed and eaten by sailors until there was none left. Other flightless birds which have disappeared are the great auk and the moa.\n\nPenguins are a very successful group of birds. They are a clade. They spend half their time on land.


key: _node_content
string_value:
{"id_": "768b6896-31c6-4ebf-8151-3a42ace3f9c5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "83d38e1a-c054-44a5-bc76-98e3a84ea541", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "2157c6a809f9e8c5e0c92f662a3aa3867127c865f5d777c3c4b35606117c5d19", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f2baeebc-e2bf-404f-bc81-6b35de755dd1", "node_type": "1", "metadata": {}, "hash": "8529cf73feb049214eb38862c5ae7da80db3e8197613da167de7ac6d4837bfc9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 26102, "end_char_idx": 29324, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
In the same way, when a flock is asleep, often, one bird will remain awake. It is also common for large flocks to send one or two birds ahead of them when they are flying to a new area. The look-out birds can spy the lie of the land to find food, water and good places to perch. Mixed feeding flocks occur, and can help to spot predators.\n\nFlightless birds \nSome birds do not fly. Flightlessness in birds has evolved many times.\nThese include running birds like ostriches and emus and ocean-living birds, the large penguin family. Birds on islands have usually lost the power of flight. This is to their advantage because birds with the power of flight can be blown off their island during a storm. The same ability which got them to the island may later take them away in a storm.\n\nOstriches and emus do not need to fly because although they feed and nest on the ground, their great size and their speed is their protection. Some other ground-feeding birds have not been so lucky. Some birds such as the dodo and the kiwi were ground-feeding birds that lived in safety on islands where there was nothing dangerous to eat them. They lost the power of flight. Kiwis are endangered because European settlement to New Zealand brought animals like cats, dogs and rats which kill kiwis and eat their eggs. However, kiwis and also the rare New Zealand ground parrot have survived. In the case of dodos, they were fat and disgusting in taste. All the same, they were killed and eaten by sailors until there was none left. Other flightless birds which have disappeared are the great auk and the moa.\n\nPenguins are a very successful group of birds. They are a clade. They spend half their time on land. Their wings are adapted to life in the sea and have become flippers which let them in swim fast. They catch fish at sea, where they are in danger from seals.\n\nDigestion \nModern birds do not have teeth, and many swallow their prey whole. Nevertheless, they must break up food before it is digested. First of all, along their throat (oesophagus) they have a crop. This stores food items before digestion. That way a bird can eat several items, and then fly off to a quiet spot to digest them. \n\nTheir stomach comes next, with two very different parts. One part is like a straight hollow rod (the proventriculus) which secretes mild hydrochloric acid and an enzyme to break down protein. The other part of the stomach is the gizzard. This is muscular, and grinds up the contents. In herbivorous birds the gizzard contains some gastroliths (small stones or pieces of grit). Bones of fish will mostly be dissolved by the stomach acid. The partly digested and ground-up food now goes to the intestine, where digestion is completed, and most contents are absorbed. Anything indigestible, for example remains of feathers, is regurgitated via the mouth, not the cloaca.\n\nThe system is effective, and carnivorous birds can swallow quite large prey. A blue heron can swallow a fish as large as a carp successfully. Raptors eat by holding the prey down with a foot, and tearing it apart with their beak.\n\nReproduction\n\nMating \nAlthough birds are warm-blooded creatures like mammals, they do not give birth to live young.


key: _node_content
string_value:
{"id_": "f2baeebc-e2bf-404f-bc81-6b35de755dd1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "768b6896-31c6-4ebf-8151-3a42ace3f9c5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e31373ca2c2829d58cf1d81bf346dd6b09533a42928c6638d5eea04cafda5425", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c8e353d2-d91b-42fb-b75b-e8bf34bf49b3", "node_type": "1", "metadata": {}, "hash": "685bb762849be688bae280050ee651e16c2e247c33bd44b6f745238dbd1657c9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 27750, "end_char_idx": 30973, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They are a clade. They spend half their time on land. Their wings are adapted to life in the sea and have become flippers which let them in swim fast. They catch fish at sea, where they are in danger from seals.\n\nDigestion \nModern birds do not have teeth, and many swallow their prey whole. Nevertheless, they must break up food before it is digested. First of all, along their throat (oesophagus) they have a crop. This stores food items before digestion. That way a bird can eat several items, and then fly off to a quiet spot to digest them. \n\nTheir stomach comes next, with two very different parts. One part is like a straight hollow rod (the proventriculus) which secretes mild hydrochloric acid and an enzyme to break down protein. The other part of the stomach is the gizzard. This is muscular, and grinds up the contents. In herbivorous birds the gizzard contains some gastroliths (small stones or pieces of grit). Bones of fish will mostly be dissolved by the stomach acid. The partly digested and ground-up food now goes to the intestine, where digestion is completed, and most contents are absorbed. Anything indigestible, for example remains of feathers, is regurgitated via the mouth, not the cloaca.\n\nThe system is effective, and carnivorous birds can swallow quite large prey. A blue heron can swallow a fish as large as a carp successfully. Raptors eat by holding the prey down with a foot, and tearing it apart with their beak.\n\nReproduction\n\nMating \nAlthough birds are warm-blooded creatures like mammals, they do not give birth to live young. They lay eggs as reptiles do, but the shell of a bird\'s egg is hard. The baby bird grows inside the egg, and after a few weeks hatches (breaks out of the egg).\n\nBirds in cold climates usually have a breeding season once a year in the spring. Migratory birds can have two springs and two mating seasons in a year. \n\nNinety-five per cent of bird species are socially monogamous. These birds pair for at least the length of the breeding season. In some cases this arrangement lasts until the death of one of the pair. Monogamy clearly helps if females need males\' help to raise a brood successfully. It has other practical advantages: the nest is never left without defence. Birds are generally small, and they have many potential enemies.\n\nSome birds mate for life, like married couples. These birds include pigeons, geese, and cranes. Other birds look for new partners each year. For birds that choose new mates, part of the breeding season is display. The male bird will do all sorts of things to attract females. These include singing, dancing, showing off the feathers and building a beautiful nest. Some male birds have splendid feathers for attracting females. The most famous is the peacock who can spread the feathers above his tail into a huge fan. \n\nOther mating systems do occur in some species. Polygyny, polyandry, polygamy, polygynandry, and promiscuity do happen. Polygamous breeding systems arise when females are able to raise broods without the help of males. Some species may use more than one system depending on the circumstances.\n\nNesting \nOnce the birds have found partners, they find a suitable place to lay eggs.


key: _node_content
string_value:
{"id_": "c8e353d2-d91b-42fb-b75b-e8bf34bf49b3", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f2baeebc-e2bf-404f-bc81-6b35de755dd1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "7a5192fa7971fc601ab1daa4d2efe21c7860cd221ce5e9861db4589a1cf8e7ae", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "720a344d-099a-4e2b-b3ed-acd3e187bcbf", "node_type": "1", "metadata": {}, "hash": "62dda77043a1625b780522ae4ad660f62d4e7f76d8b896c73126ee172c6573a9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 29325, "end_char_idx": 32504, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They lay eggs as reptiles do, but the shell of a bird\'s egg is hard. The baby bird grows inside the egg, and after a few weeks hatches (breaks out of the egg).\n\nBirds in cold climates usually have a breeding season once a year in the spring. Migratory birds can have two springs and two mating seasons in a year. \n\nNinety-five per cent of bird species are socially monogamous. These birds pair for at least the length of the breeding season. In some cases this arrangement lasts until the death of one of the pair. Monogamy clearly helps if females need males\' help to raise a brood successfully. It has other practical advantages: the nest is never left without defence. Birds are generally small, and they have many potential enemies.\n\nSome birds mate for life, like married couples. These birds include pigeons, geese, and cranes. Other birds look for new partners each year. For birds that choose new mates, part of the breeding season is display. The male bird will do all sorts of things to attract females. These include singing, dancing, showing off the feathers and building a beautiful nest. Some male birds have splendid feathers for attracting females. The most famous is the peacock who can spread the feathers above his tail into a huge fan. \n\nOther mating systems do occur in some species. Polygyny, polyandry, polygamy, polygynandry, and promiscuity do happen. Polygamous breeding systems arise when females are able to raise broods without the help of males. Some species may use more than one system depending on the circumstances.\n\nNesting \nOnce the birds have found partners, they find a suitable place to lay eggs. The idea of what is a suitable place differs between species, but most build bird nests. The bird is driven by a hormone (estradiol E2) to prepare a place for the eggs to hatch. Birds\' nests may be up a tree, in a cliff or on the ground according to species. When filled with eggs they are almost always guarded by one of the pair. In fact it is virtually impossible for the eggs to survive if one of the parents dies.\n\nRobins will make a beautiful little round nest of woven grass and carefully line it with feathers, bits of fluff and other soft things. Swallows like to nest near other swallows. They make nests from little blobs of clay, often on a beam near the roof of a building where it is well sheltered. Many birds like a hollow tree to nest in. Eagle\'s nests are often just piles of dead wood on the top of the tallest tree or mountain. Scrub turkeys scratch together a huge pile of leaves that may be 10 metres across. Guillemots lay their eggs on rock shelves with no nest at all. Their eggs are shaped so that they roll around in circles and do not fall off cliffs. A cuckoo does not make its own nest. It lays its egg in the nest of another bird and leaves it for them to care for. The cuckoo eggs are camouflaged to look like the host\'s eggs.\n\nWhen the nest has been prepared, the birds mate so that the eggs are fertilised and the chicks will start growing. Unlike mammals, birds (and reptiles) only have one opening as the exit hole for body fluids, and for reproduction. The opening is called the cloaca.


key: _node_content
string_value:
{"id_": "720a344d-099a-4e2b-b3ed-acd3e187bcbf", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c8e353d2-d91b-42fb-b75b-e8bf34bf49b3", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "cf9fe6503118eb5f40b29ededfb7b820d12755c85ce23eb692bcf16234faaaf3", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5e9f85be-3772-4e47-becf-5be47ea426e1", "node_type": "1", "metadata": {}, "hash": "886b166e3bdc8cde7806d2e71f274e1c943849e68e4b39ca53804ba63836464e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 30974, "end_char_idx": 34118, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The idea of what is a suitable place differs between species, but most build bird nests. The bird is driven by a hormone (estradiol E2) to prepare a place for the eggs to hatch. Birds\' nests may be up a tree, in a cliff or on the ground according to species. When filled with eggs they are almost always guarded by one of the pair. In fact it is virtually impossible for the eggs to survive if one of the parents dies.\n\nRobins will make a beautiful little round nest of woven grass and carefully line it with feathers, bits of fluff and other soft things. Swallows like to nest near other swallows. They make nests from little blobs of clay, often on a beam near the roof of a building where it is well sheltered. Many birds like a hollow tree to nest in. Eagle\'s nests are often just piles of dead wood on the top of the tallest tree or mountain. Scrub turkeys scratch together a huge pile of leaves that may be 10 metres across. Guillemots lay their eggs on rock shelves with no nest at all. Their eggs are shaped so that they roll around in circles and do not fall off cliffs. A cuckoo does not make its own nest. It lays its egg in the nest of another bird and leaves it for them to care for. The cuckoo eggs are camouflaged to look like the host\'s eggs.\n\nWhen the nest has been prepared, the birds mate so that the eggs are fertilised and the chicks will start growing. Unlike mammals, birds (and reptiles) only have one opening as the exit hole for body fluids, and for reproduction. The opening is called the cloaca. A female bird, called a hen, has two ovaries, of which the left one usually produces eggs.\n\nMost male birds have no sex organs that can be seen. But inside the male are two testes which produce sperm which is stored in the cloaca. Birds mate by rubbing their cloacas together, although with some birds, particularly large water birds, the male has a sort of a penis inside the cloaca.\n\nHatching \nOnce the hen has mated, she produces fertile eggs which have chicks growing inside them. She lays the eggs in the nest. There might be just one egg or a number of them, called a clutch. Emus might lay as many as fifteen huge dark green eggs in a clutch. After the eggs are laid, they are incubated, or kept warm so the chicks form inside. Most birds stay together for the whole nesting season, and one advantage is that the work is shared. Many birds take turns sitting on the eggs, so that each adult can feed.\n\nThis is not always the case. With emus, the male does all the sitting and all the baby-minding. With emperor penguins it is also the male that cares for the egg. There is only one egg, which he keeps on his feet and under his feathers, standing in a big group of males without feeding until the chick is hatched. While the eggs are hatching, the females are at sea, feeding, so that they can care for the chicks when they return.\n\nSome birds put the eggs inside or on top of the mound of leaves and twigs. The mound acts like a compost heap. The decomposition of the rotting leaves causes the temperature to rise. This is heat released by the chemical action of bacterial and fungal respiration.


key: _node_content
string_value:
{"id_": "5e9f85be-3772-4e47-becf-5be47ea426e1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "720a344d-099a-4e2b-b3ed-acd3e187bcbf", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d9ec86340a52f253b6ab2d53a6c5b821ee41bfa5255682d7fd5882e92c2720fc", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "469f9c63-04b3-4de2-9b4d-3dc01c4e0d0e", "node_type": "1", "metadata": {}, "hash": "efa3949b1cc86c94d190b2dc3095fac371f8fe4357c252bbc23ef5c684f4e5b4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 32471, "end_char_idx": 35659, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The opening is called the cloaca. A female bird, called a hen, has two ovaries, of which the left one usually produces eggs.\n\nMost male birds have no sex organs that can be seen. But inside the male are two testes which produce sperm which is stored in the cloaca. Birds mate by rubbing their cloacas together, although with some birds, particularly large water birds, the male has a sort of a penis inside the cloaca.\n\nHatching \nOnce the hen has mated, she produces fertile eggs which have chicks growing inside them. She lays the eggs in the nest. There might be just one egg or a number of them, called a clutch. Emus might lay as many as fifteen huge dark green eggs in a clutch. After the eggs are laid, they are incubated, or kept warm so the chicks form inside. Most birds stay together for the whole nesting season, and one advantage is that the work is shared. Many birds take turns sitting on the eggs, so that each adult can feed.\n\nThis is not always the case. With emus, the male does all the sitting and all the baby-minding. With emperor penguins it is also the male that cares for the egg. There is only one egg, which he keeps on his feet and under his feathers, standing in a big group of males without feeding until the chick is hatched. While the eggs are hatching, the females are at sea, feeding, so that they can care for the chicks when they return.\n\nSome birds put the eggs inside or on top of the mound of leaves and twigs. The mound acts like a compost heap. The decomposition of the rotting leaves causes the temperature to rise. This is heat released by the chemical action of bacterial and fungal respiration. It is the same reaction as that which keeps mammals and birds at a high temperature. The parents leave the mound. When the chicks hatch, they are able to feed themselves.\n\nMany small birds take 2–4 weeks to hatch eggs. Albatrosses take 80 days. During this time the female loses a lot of her body weight.\n\nThe quickest hatching time is for the cuckoo. Some types of cuckoos take only 10 days. This means that when they hatch in the nest of their \'\'foster parents\'\', the eggs that the parents have laid are not yet ready. Newborn cuckoos are naked, blind and ugly, but they are strong. They get under any eggs that are in the nest and throw them out before they hatch. That means that the cuckoo has the whole care of both parents. Baby cuckoos grow fast and often get bigger than the parents who feed them.\n\nWhen baby birds hatch, in most types of birds, they are fed by both parents, and sometimes by older aunties as well. Their mouths are open all the time and are often very brightly coloured, which acts as a releaser\'\', a trigger which stimulates the parent to feed them. For birds that eat grain and fruit, the parents eat and partly digest the food for the babies. It is then vomited carefully into the baby\'s mouth.\n  \n\n Families \nMany birds, particularly those that mate for life, are very sociable and keep together in a family group which might be anything from 4 or 6 adult birds and their young to a very large flock.\n\nAs chicks grow they change the fluffy down that covers them as babies for real feathers.


key: _node_content
string_value:
{"id_": "469f9c63-04b3-4de2-9b4d-3dc01c4e0d0e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5e9f85be-3772-4e47-becf-5be47ea426e1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "3594f3fecd56a1c84d8ede4e8119c6eb74df098f0b9db31a7b50e8713a99f321", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c8ecba07-d785-4e6a-9e45-2ce60a8799be", "node_type": "1", "metadata": {}, "hash": "98a8d0bd232fc1ecdecf8f9b20f83ae046fa1645f13e1749018ffe0d0a686b39", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 33965, "end_char_idx": 37016, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The decomposition of the rotting leaves causes the temperature to rise. This is heat released by the chemical action of bacterial and fungal respiration. It is the same reaction as that which keeps mammals and birds at a high temperature. The parents leave the mound. When the chicks hatch, they are able to feed themselves.\n\nMany small birds take 2–4 weeks to hatch eggs. Albatrosses take 80 days. During this time the female loses a lot of her body weight.\n\nThe quickest hatching time is for the cuckoo. Some types of cuckoos take only 10 days. This means that when they hatch in the nest of their \'\'foster parents\'\', the eggs that the parents have laid are not yet ready. Newborn cuckoos are naked, blind and ugly, but they are strong. They get under any eggs that are in the nest and throw them out before they hatch. That means that the cuckoo has the whole care of both parents. Baby cuckoos grow fast and often get bigger than the parents who feed them.\n\nWhen baby birds hatch, in most types of birds, they are fed by both parents, and sometimes by older aunties as well. Their mouths are open all the time and are often very brightly coloured, which acts as a releaser\'\', a trigger which stimulates the parent to feed them. For birds that eat grain and fruit, the parents eat and partly digest the food for the babies. It is then vomited carefully into the baby\'s mouth.\n  \n\n Families \nMany birds, particularly those that mate for life, are very sociable and keep together in a family group which might be anything from 4 or 6 adult birds and their young to a very large flock.\n\nAs chicks grow they change the fluffy down that covers them as babies for real feathers. At this stage they are called fledglings. Other family members may help care for fledgling chicks, feeding them and protecting them from attack while parents are feeding. When the fledglings have their new feathers, they come out of the nest to learn to fly. In some types of birds, like pigeons, the parents watch over this and as the young ones get stronger, will give them flying lessons, teaching them how to glide, how to fly in spirals and how to land like an expert.\n\n Communication \nMost birds are social animals, at least part of the time. They communicate to each other using sounds and displays.\n\nAlmost all birds make sounds to communicate. The types of noises that vary greatly. Some birds can sing, and  they are called songbirds or passerines. Examples are robins, larks, canaries, thrushes, nightingales. Corvids are passerines, but they do not sing. Birds that are not songbirds include: pigeons, seagulls, eagles, owls and ducks. Parrots are not songbirds, even though they can be taught to sing human songs.\n\n Songbirds \nAll birds make noises (\'\'bird vocalisation\'\'), but not all sing. Songbirds are passerines, many of which have beautiful melodic songs. Songs have different functions. Danger cries are different from territorial songs and mating calls are a third type. Fledgling may also have different calls from adults.


key: _node_content
string_value:
{"id_": "c8ecba07-d785-4e6a-9e45-2ce60a8799be", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "469f9c63-04b3-4de2-9b4d-3dc01c4e0d0e", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a7520a47a43fd0f9aa250995db40fc096aac916bcd35432e33c258bcb9de77f7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f6fe36aa-515b-4f58-b1a7-dce367b64be5", "node_type": "1", "metadata": {}, "hash": "e483fdd78711de407af626ad88a8d9a9d2d52b3e8fc56a66bcd3f9f2753388e2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 35660, "end_char_idx": 38910, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
At this stage they are called fledglings. Other family members may help care for fledgling chicks, feeding them and protecting them from attack while parents are feeding. When the fledglings have their new feathers, they come out of the nest to learn to fly. In some types of birds, like pigeons, the parents watch over this and as the young ones get stronger, will give them flying lessons, teaching them how to glide, how to fly in spirals and how to land like an expert.\n\n Communication \nMost birds are social animals, at least part of the time. They communicate to each other using sounds and displays.\n\nAlmost all birds make sounds to communicate. The types of noises that vary greatly. Some birds can sing, and  they are called songbirds or passerines. Examples are robins, larks, canaries, thrushes, nightingales. Corvids are passerines, but they do not sing. Birds that are not songbirds include: pigeons, seagulls, eagles, owls and ducks. Parrots are not songbirds, even though they can be taught to sing human songs.\n\n Songbirds \nAll birds make noises (\'\'bird vocalisation\'\'), but not all sing. Songbirds are passerines, many of which have beautiful melodic songs. Songs have different functions. Danger cries are different from territorial songs and mating calls are a third type. Fledgling may also have different calls from adults. Recognition calls for partners are quite common.\n\nAs to where the song comes from, there are three kinds of species:\nThose where the song is mainly inherited, and the bird always sings the same song in the same situations. The capacity is inherited, and only details are learnt from its neighbours.\nThose where the song is partly inherited, but the bird tunes it in by copying others. In this case the slight differences between the calls of different birds may be used by partners for identification.\nThose where the song is entirely learnt, and the bird often copies sounds from its environment. Only the capacity to sing is inherited.\n\nMost singing birds that are kept as pets, like canaries, have several tunes and some variations.\n\nThe same species of bird will sing different songs in different regions. A good example of this is the currawong. This is an Australia bird which is like a black and white crow. In the autumn, families get together in large flocks and do a lot of singing. Currawongs from some areas sing much more complex songs than others. Generally, currawongs from the Blue Mountains are the finest singers. The song of the currawong can be sung as a solo, but is often performed as a choir. One bird will take the lead and sing "Warble-warble-warble-warble!" All the other birds will join in and sing "Wooooooo!". When all the birds know the song, the choir will sing the "Warble" part and the soloist will sing the "Woo!". The song changes from year to year and from place to place.\n\n Lorenz\'s studies \nThe Austrian naturalist Konrad Lorenz studied the way in which birds communicate, or talk to each other. He found that each type of bird had a number of sounds which they made automatically, when ever they felt a certain way. Every sound had an action that went with it. So, if the bird was frightened, it acted frightened and made a frightened sound.


key: _node_content
string_value:
{"id_": "f6fe36aa-515b-4f58-b1a7-dce367b64be5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c8ecba07-d785-4e6a-9e45-2ce60a8799be", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a42ae769e2c8a754ddce7741c8b5bdb6004b8402bc7e8ba98d85109191d9d56b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "4a178521-49e6-408d-aaf7-5ce5c0a9906c", "node_type": "1", "metadata": {}, "hash": "c1c439af84bc74a22e2af60430de694f76da2134686502d7b3c2177ed554936e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 37243, "end_char_idx": 40472, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The capacity is inherited, and only details are learnt from its neighbours.\nThose where the song is partly inherited, but the bird tunes it in by copying others. In this case the slight differences between the calls of different birds may be used by partners for identification.\nThose where the song is entirely learnt, and the bird often copies sounds from its environment. Only the capacity to sing is inherited.\n\nMost singing birds that are kept as pets, like canaries, have several tunes and some variations.\n\nThe same species of bird will sing different songs in different regions. A good example of this is the currawong. This is an Australia bird which is like a black and white crow. In the autumn, families get together in large flocks and do a lot of singing. Currawongs from some areas sing much more complex songs than others. Generally, currawongs from the Blue Mountains are the finest singers. The song of the currawong can be sung as a solo, but is often performed as a choir. One bird will take the lead and sing "Warble-warble-warble-warble!" All the other birds will join in and sing "Wooooooo!". When all the birds know the song, the choir will sing the "Warble" part and the soloist will sing the "Woo!". The song changes from year to year and from place to place.\n\n Lorenz\'s studies \nThe Austrian naturalist Konrad Lorenz studied the way in which birds communicate, or talk to each other. He found that each type of bird had a number of sounds which they made automatically, when ever they felt a certain way. Every sound had an action that went with it. So, if the bird was frightened, it acted frightened and made a frightened sound. This told the other birds around it that something frightening was happening.\n\nIf a flock of birds were flying over a field, they would be calling "Fly! Fly!" But a hungry bird, seeing something good to eat down below might start calling "Food! Food!" If other birds were also hungry, they would make the same call until more birds were calling "Food! Food!" than "Fly! Fly!". At this point, the mind of the flock would be changed. Some of the birds would start to yell "Fly downwards! Fly downwards!" as they sank from the sky, until the whole flock was all noisily calling the same thing.\n\nThese communication sounds are often short hard sounds like: chirps, squeaks, squawks and twitters. Sometimes the calls are longer and more musical. They include the "Rookety-coo" sound of a pigeon and the "Cockadoodledoo!" of a rooster. The bird cannot change these sounds. They always make them in the same way. The bird is locked into making each sound every time a particular idea comes into its head. The connection between how they feel and how they call is innate: they are born with it. Some calls in some species are learnt. Then, it is the tendency to learn which is inherited.\n\n The Jackdaw of Altenberg \nKonrad Lorenz noticed that when birds sing, they often use a lot of their regular calls as part of the song. Lorenz had a flock of jackdaws which were scattered during World War II. One day, an old bird returned. For many months she sat on the chimney singing her song, but in the song she kept making the call which Lorenz knew meant "Come home!


key: _node_content
string_value:
{"id_": "4a178521-49e6-408d-aaf7-5ce5c0a9906c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f6fe36aa-515b-4f58-b1a7-dce367b64be5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b09b7fcc5252c4500ecd1e82ccee3be4f97443b0b49100c189c605be59659758", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5845b793-0dde-427e-a848-de89c5cc31c8", "node_type": "1", "metadata": {}, "hash": "64fa5a9ecee0fd06bfaeb44649803337cc5aed0a9a5ef1efeaba03b21702dbe7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 38911, "end_char_idx": 41861, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
This told the other birds around it that something frightening was happening.\n\nIf a flock of birds were flying over a field, they would be calling "Fly! Fly!" But a hungry bird, seeing something good to eat down below might start calling "Food! Food!" If other birds were also hungry, they would make the same call until more birds were calling "Food! Food!" than "Fly! Fly!". At this point, the mind of the flock would be changed. Some of the birds would start to yell "Fly downwards! Fly downwards!" as they sank from the sky, until the whole flock was all noisily calling the same thing.\n\nThese communication sounds are often short hard sounds like: chirps, squeaks, squawks and twitters. Sometimes the calls are longer and more musical. They include the "Rookety-coo" sound of a pigeon and the "Cockadoodledoo!" of a rooster. The bird cannot change these sounds. They always make them in the same way. The bird is locked into making each sound every time a particular idea comes into its head. The connection between how they feel and how they call is innate: they are born with it. Some calls in some species are learnt. Then, it is the tendency to learn which is inherited.\n\n The Jackdaw of Altenberg \nKonrad Lorenz noticed that when birds sing, they often use a lot of their regular calls as part of the song. Lorenz had a flock of jackdaws which were scattered during World War II. One day, an old bird returned. For many months she sat on the chimney singing her song, but in the song she kept making the call which Lorenz knew meant "Come home! Come home!" One day, to the great surprise of Lorenz, a male bird flew from a passing flock and joined her on the chimney. Lorenz was sure that it was her long-lost "husband" who had found his way home at last.\n\n Evolution and taxonomy \n\nPalaeontologists have found some exceptional places (lagerstätten) where fossils of early birds are found. The preservation is so good that on the best examples impressions of their feathers can be seen, and sometimes even the remains of meals they have eaten. From these remains we know that birds evolved from small carnivorous dinosaurs (theropods) in the Jurassic period. They radiated into a huge variety in the Lower Cretaceous. At the same time, their direct competitors, the pterosaurs, dwindled in numbers and variety, and became extinct at the end of the Mesozoic.\n\nBirds are classified by taxonomists as \'Aves\' (Avialae). Birds are the only living descendants of dinosaurs (strictly speaking, they are dinosaurs). Birds and Crocodilia are the only living members of the once-dominant Archosaur reptiles.\n\n Definition \nThe class Aves is was defined (1990) as all the descendants of the most recent common ancestor of modern birds and Archaeopteryx lithographica. But Archaeopteryx is almost certainly not the ancestor of modern birds. The transition to flight happened a number of times. The researchers offered four definitions.


key: _node_content
string_value:
{"id_": "5845b793-0dde-427e-a848-de89c5cc31c8", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "4a178521-49e6-408d-aaf7-5ce5c0a9906c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "7447bf990b81f0c252538da849ed1372ef57cc3ad87d569a3351ea8595e0c876", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "dacf2b54-d95e-46e3-8bdd-5c19e5b94985", "node_type": "1", "metadata": {}, "hash": "9a7c48fdf27b03ea078866c1db294cce76a84210a4d91d9b1d68fe17bac14ce2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 40235, "end_char_idx": 43253, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Lorenz had a flock of jackdaws which were scattered during World War II. One day, an old bird returned. For many months she sat on the chimney singing her song, but in the song she kept making the call which Lorenz knew meant "Come home! Come home!" One day, to the great surprise of Lorenz, a male bird flew from a passing flock and joined her on the chimney. Lorenz was sure that it was her long-lost "husband" who had found his way home at last.\n\n Evolution and taxonomy \n\nPalaeontologists have found some exceptional places (lagerstätten) where fossils of early birds are found. The preservation is so good that on the best examples impressions of their feathers can be seen, and sometimes even the remains of meals they have eaten. From these remains we know that birds evolved from small carnivorous dinosaurs (theropods) in the Jurassic period. They radiated into a huge variety in the Lower Cretaceous. At the same time, their direct competitors, the pterosaurs, dwindled in numbers and variety, and became extinct at the end of the Mesozoic.\n\nBirds are classified by taxonomists as \'Aves\' (Avialae). Birds are the only living descendants of dinosaurs (strictly speaking, they are dinosaurs). Birds and Crocodilia are the only living members of the once-dominant Archosaur reptiles.\n\n Definition \nThe class Aves is was defined (1990) as all the descendants of the most recent common ancestor of modern birds and Archaeopteryx lithographica. But Archaeopteryx is almost certainly not the ancestor of modern birds. The transition to flight happened a number of times. The researchers offered four definitions. Birds can be: \nAll archosaurs closer to birds than crocodiles (Avemetatarsalia).\nAdvanced archosaurs with feathers (Avofilopluma).\nThose feathered dinosaurs that fly (or Avialae)\nAves can mean the last common ancestor of all living birds and all of its descendants (a "crown group", in this sense synonymous with Neornithes).\n\n The first bird-like creatures Archaeopteryx, from the Upper Jurassic some 150–145 million years ago (mya), was for a long time the earliest known bird which could fly. It is famous, because it was one of the first important fossils found after Charles Darwin published his ideas about evolution in the 19th century. By modern standards, Archaeopteryx could not fly very well. Other early fossil birds are, for example, Confuciusornis, Anchiornis huxlei and other Paraves.\n\nMany fossils of early birds and small dinosaurs have been discovered in the Liaoning Province of Northeast China. These include Anchiornis huxlei, from about 160 mya. The fossils show that most small theropod dinosaurs had feathers. These deposits have preserved them so well that the impressions of their feathers can be clearly seen. This leads us to think that feathers evolved first as heat insulation and only later for flight. The origin of birds lies in these small feathered dinosaurs.\n\nPalaeontologists now agree that birds are included in Maniraptora group of dinosaurs.


key: _node_content
string_value:
{"id_": "dacf2b54-d95e-46e3-8bdd-5c19e5b94985", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5845b793-0dde-427e-a848-de89c5cc31c8", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "779184f0b317e3e1ee8592f7a3670e6bfb6b736f01111a4eea8708782c1fb8eb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d86ba865-d8b6-465e-ba03-a7e10d67c49e", "node_type": "1", "metadata": {}, "hash": "c302f2f8577d194ac5a38f2694004140f21616d02a077ef76259a950b29fa685", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 41695, "end_char_idx": 44907, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
But Archaeopteryx is almost certainly not the ancestor of modern birds. The transition to flight happened a number of times. The researchers offered four definitions. Birds can be: \nAll archosaurs closer to birds than crocodiles (Avemetatarsalia).\nAdvanced archosaurs with feathers (Avofilopluma).\nThose feathered dinosaurs that fly (or Avialae)\nAves can mean the last common ancestor of all living birds and all of its descendants (a "crown group", in this sense synonymous with Neornithes).\n\n The first bird-like creatures Archaeopteryx, from the Upper Jurassic some 150–145 million years ago (mya), was for a long time the earliest known bird which could fly. It is famous, because it was one of the first important fossils found after Charles Darwin published his ideas about evolution in the 19th century. By modern standards, Archaeopteryx could not fly very well. Other early fossil birds are, for example, Confuciusornis, Anchiornis huxlei and other Paraves.\n\nMany fossils of early birds and small dinosaurs have been discovered in the Liaoning Province of Northeast China. These include Anchiornis huxlei, from about 160 mya. The fossils show that most small theropod dinosaurs had feathers. These deposits have preserved them so well that the impressions of their feathers can be clearly seen. This leads us to think that feathers evolved first as heat insulation and only later for flight. The origin of birds lies in these small feathered dinosaurs.\n\nPalaeontologists now agree that birds are included in Maniraptora group of dinosaurs. This explains why we say that birds are living dinosaurs.\n\n Evolution of modern birds \nA leading authority says "Most living birds have fossil representatives in the Cenozoic"... "Key problems remain in understanding bird phylogeny... we seem to understand as little about the relationships among living birds as among Cretaceous birds".\n\n Origin of birds\n Paraves\n\nBirds and people \n\nSome birds are eaten as food. Most usually it is the chicken and its eggs, but people often also eat geese, pheasants, turkeys and ducks. Other birds are sometimes eaten are: emus, ostriches, pigeons, grouse, quails, doves, woodcocks and even songbirds. Some species have died out because they have been hunted for food, for example the dodo and the passenger pigeon.\n\nMany species have learned how to get food from people. The number of birds of these species has grown because of it. Seagulls and crows find food from garbage dumps. The feral pigeon (Columba livia), sparrows (Passer domesticus and starlings (Sturnus vulgaris) live in large numbers in towns and cities all over the world.\n\nSometimes people also use working birds. For example, homing pigeons carry messages. Nowadays people sometimes race them for sport. People also use falcons for hunting, and cormorants for fishing. In the past, people in mines often used a canary to see if there were bad gas methane in the air.\n\nPeople often have colorful birds such as parrots and mynahs as pets. These intelligent birds are popular because they can copy human talking. Because of this, some people trap birds and take them to other countries to sell. This is not usually allowed these days.


key: _node_content
string_value:
{"id_": "d86ba865-d8b6-465e-ba03-a7e10d67c49e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "dacf2b54-d95e-46e3-8bdd-5c19e5b94985", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "04d92482aad0b3018c757ebce6a7e3cf43e8a0d8d85897b1f96aac4460a67b38", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ab08d6de-565d-4344-82e3-b06bde4e9b67", "node_type": "1", "metadata": {}, "hash": "5aeb01015287806a863067b3d728852e714127786cf5f460bf224767403eb05f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 43254, "end_char_idx": 45432, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
This explains why we say that birds are living dinosaurs.\n\n Evolution of modern birds \nA leading authority says "Most living birds have fossil representatives in the Cenozoic"... "Key problems remain in understanding bird phylogeny... we seem to understand as little about the relationships among living birds as among Cretaceous birds".\n\n Origin of birds\n Paraves\n\nBirds and people \n\nSome birds are eaten as food. Most usually it is the chicken and its eggs, but people often also eat geese, pheasants, turkeys and ducks. Other birds are sometimes eaten are: emus, ostriches, pigeons, grouse, quails, doves, woodcocks and even songbirds. Some species have died out because they have been hunted for food, for example the dodo and the passenger pigeon.\n\nMany species have learned how to get food from people. The number of birds of these species has grown because of it. Seagulls and crows find food from garbage dumps. The feral pigeon (Columba livia), sparrows (Passer domesticus and starlings (Sturnus vulgaris) live in large numbers in towns and cities all over the world.\n\nSometimes people also use working birds. For example, homing pigeons carry messages. Nowadays people sometimes race them for sport. People also use falcons for hunting, and cormorants for fishing. In the past, people in mines often used a canary to see if there were bad gas methane in the air.\n\nPeople often have colorful birds such as parrots and mynahs as pets. These intelligent birds are popular because they can copy human talking. Because of this, some people trap birds and take them to other countries to sell. This is not usually allowed these days. Most pet birds are specially bred and are sold in pet shops.\n\nPeople can catch some bird diseases, for example: psittacosis, salmonellosis, campylobacteriosis, Newcastle\'s disease, mycobacteriosis, influenza, giardiasis and cryptosporiadiosis. In 2005, there was an epidemic of bird influenza spreading through some parts of the world, often called avian flu.\n\nSome people have birdboxes in their gardens to give birds a place to nest and bird tables where birds can get food and water in very cold or very dry weather.


key: _node_content
string_value:
{"id_": "ab08d6de-565d-4344-82e3-b06bde4e9b67", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d86ba865-d8b6-465e-ba03-a7e10d67c49e", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "57d6c746d7a6dbb9996afb573d5ab49e373bfa78f015280e594a798151139537", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "92187d6e-64d7-4f3b-8a87-594ef43fe7d1", "node_type": "1", "metadata": {}, "hash": "7eef2531684a8337cf9f73855d47ed69703a45df1f70cd6f8b8972eb249a0f49", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 43787, "end_char_idx": 46572, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Other birds are sometimes eaten are: emus, ostriches, pigeons, grouse, quails, doves, woodcocks and even songbirds. Some species have died out because they have been hunted for food, for example the dodo and the passenger pigeon.\n\nMany species have learned how to get food from people. The number of birds of these species has grown because of it. Seagulls and crows find food from garbage dumps. The feral pigeon (Columba livia), sparrows (Passer domesticus and starlings (Sturnus vulgaris) live in large numbers in towns and cities all over the world.\n\nSometimes people also use working birds. For example, homing pigeons carry messages. Nowadays people sometimes race them for sport. People also use falcons for hunting, and cormorants for fishing. In the past, people in mines often used a canary to see if there were bad gas methane in the air.\n\nPeople often have colorful birds such as parrots and mynahs as pets. These intelligent birds are popular because they can copy human talking. Because of this, some people trap birds and take them to other countries to sell. This is not usually allowed these days. Most pet birds are specially bred and are sold in pet shops.\n\nPeople can catch some bird diseases, for example: psittacosis, salmonellosis, campylobacteriosis, Newcastle\'s disease, mycobacteriosis, influenza, giardiasis and cryptosporiadiosis. In 2005, there was an epidemic of bird influenza spreading through some parts of the world, often called avian flu.\n\nSome people have birdboxes in their gardens to give birds a place to nest and bird tables where birds can get food and water in very cold or very dry weather. This lets people see some small birds close up which are normally hidden away in bushes and trees.\n\nBird orders \nThe following is a listing of all bird orders:\n Infraclass Palaeognathae\n Superorder Struthionimorphae\n Struthioniformes\n Superorder Notopalaeognathae\n Rheiformes\n Tinamiformes\n Casuariiformes\n Apterygiformes\n Infraclass Neognathae\n Superorder Galloanserae\n Galliformes\n Anseriformes\n Superorder Neoaves\n Phoenicopteriformes\n Podicipediformes\n Columbiformes\n Mesitornithiformes\n Pteroclidiformes\n Apodiformes\n Caprimulgiformes\n Cuculiformes\n Otidiformes\n Musophagiformes\n Opisthocomiformes\n Gruiformes\n Charadriiformes\n Gaviiformes\n Procellariiformes\n Sphenisciformes\n Ciconiiformes\n Suliformes\n Pelecaniformes\n Eurypygiformes\n Phaethontiformes\n Cathartiformes\n Accipitriformes\n Strigiformes\n Coliiformes\n Leptosomiformes\n Trogoniformes\n Bucerotiformes\n Coraciiformes\n Piciformes\n Cariamiformes\n Falconiformes\n Psittaciformes\n Passeriformes\n\nBird population decreasing\nA report produced by BirdLife International every five years measures the population of birds worldwide.


key: _node_content
string_value:
{"id_": "92187d6e-64d7-4f3b-8a87-594ef43fe7d1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ab08d6de-565d-4344-82e3-b06bde4e9b67", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "26ef9fbed6100eb607c2b26e18d721c102516faf971a2deb68066d33aa27b891", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "4b5a237d-23cc-4213-98ea-f7fc712b41f1", "node_type": "1", "metadata": {}, "hash": "88ef2f56aef8083645d1f0aed4108001b7a76d7d9b71c5d55d8164f49a1226cb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 45155, "end_char_idx": 48044, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
In 2005, there was an epidemic of bird influenza spreading through some parts of the world, often called avian flu.\n\nSome people have birdboxes in their gardens to give birds a place to nest and bird tables where birds can get food and water in very cold or very dry weather. This lets people see some small birds close up which are normally hidden away in bushes and trees.\n\nBird orders \nThe following is a listing of all bird orders:\n Infraclass Palaeognathae\n Superorder Struthionimorphae\n Struthioniformes\n Superorder Notopalaeognathae\n Rheiformes\n Tinamiformes\n Casuariiformes\n Apterygiformes\n Infraclass Neognathae\n Superorder Galloanserae\n Galliformes\n Anseriformes\n Superorder Neoaves\n Phoenicopteriformes\n Podicipediformes\n Columbiformes\n Mesitornithiformes\n Pteroclidiformes\n Apodiformes\n Caprimulgiformes\n Cuculiformes\n Otidiformes\n Musophagiformes\n Opisthocomiformes\n Gruiformes\n Charadriiformes\n Gaviiformes\n Procellariiformes\n Sphenisciformes\n Ciconiiformes\n Suliformes\n Pelecaniformes\n Eurypygiformes\n Phaethontiformes\n Cathartiformes\n Accipitriformes\n Strigiformes\n Coliiformes\n Leptosomiformes\n Trogoniformes\n Bucerotiformes\n Coraciiformes\n Piciformes\n Cariamiformes\n Falconiformes\n Psittaciformes\n Passeriformes\n\nBird population decreasing\nA report produced by BirdLife International every five years measures the population of birds worldwide. One in every eight types of birds is now "in decline".\n\nReferences\n\nOther websites \n\n Avibase - The World Bird Database \n Bird Hybrids Database - Search by bird name, use Sibley classification\n International Ornithological Committee \n\nBasic English 850 words', 'vector_id': 898}}, {'_index': 'wikipedia_vector_index', '_id': '42874', '_score': 0.89821434, '_source': {'url': 'https://simple.wikipedia.org/wiki/History%20of%20the%20world', 'title': 'History of the world', 'text': 'The history of the world (also called human history) is the study of what the entire human race did in the past. It includes the time from prehistory to the present day.  It is different from natural history.\n\nDevelopment of the human species \n\nModern human beings are called Homo sapiens (\'wise man\'). They have existed for about 250,000 years. Biologists believe that Homo sapiens evolved in Africa.\n\nHomo sapiens, lived at the same time as  other species of human. These included Homo erectus (\'standing man\') and Homo neanderthalensis (\'man from Neanderthal\'). The theory of human evolution says that modern humans, Neanderthals, and Homo erectus slowly developed from other earlier species of human-like creatures.\n\nHomo neanderthalensis are the first humans scientists discovered which were not Homo sapiens. Homo neanderthalensis are usually called Neanderthal Man. They were discovered when the cranium of a skull was found in the Neanderthal Valley in 1856.


key: _node_content
string_value:
{"id_": "4b5a237d-23cc-4213-98ea-f7fc712b41f1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "92187d6e-64d7-4f3b-8a87-594ef43fe7d1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "77d116504ca5264aba1d413163998fb31908de16abece145852da58bac384555", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "497ce8d0-b5f6-49ef-bf6e-f704b8c2bfcb", "node_type": "1", "metadata": {}, "hash": "f7f3db1c491f0fd69fd854ceea4636f5e6a2687aa35fb95815afd54121f291d1", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 46573, "end_char_idx": 49720, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
One in every eight types of birds is now "in decline".\n\nReferences\n\nOther websites \n\n Avibase - The World Bird Database \n Bird Hybrids Database - Search by bird name, use Sibley classification\n International Ornithological Committee \n\nBasic English 850 words', 'vector_id': 898}}, {'_index': 'wikipedia_vector_index', '_id': '42874', '_score': 0.89821434, '_source': {'url': 'https://simple.wikipedia.org/wiki/History%20of%20the%20world', 'title': 'History of the world', 'text': 'The history of the world (also called human history) is the study of what the entire human race did in the past. It includes the time from prehistory to the present day.  It is different from natural history.\n\nDevelopment of the human species \n\nModern human beings are called Homo sapiens (\'wise man\'). They have existed for about 250,000 years. Biologists believe that Homo sapiens evolved in Africa.\n\nHomo sapiens, lived at the same time as  other species of human. These included Homo erectus (\'standing man\') and Homo neanderthalensis (\'man from Neanderthal\'). The theory of human evolution says that modern humans, Neanderthals, and Homo erectus slowly developed from other earlier species of human-like creatures.\n\nHomo neanderthalensis are the first humans scientists discovered which were not Homo sapiens. Homo neanderthalensis are usually called Neanderthal Man. They were discovered when the cranium of a skull was found in the Neanderthal Valley in 1856. It was different from a modern human skull so scientists believed it was from a new species. Entire Neanderthal skeletons have been found in other places since then. When ancient stone tools are found, their style often shows whether they were made by Homo sapiens or Neanderthals (see Palaeolithic). Neanderthals existed before modern humans. They knew how to use tools and fire.\n\nScientists believe that Homo sapiens spread from Africa to all other parts of the world, replacing Homo neanderthalensis in Europe and Homo erectus in Asia. By the end of the Stone Age, it is believed that Homo sapiens were the only type of humans left.\n\nInfluence of climate \n\nClimate is the normal weather in a place. It changes from one part of the world to another. Some areas are hot all year, and some are cold all year. Some areas are dry all year, and others are wet all year. Most areas have climates that are warmer in the summer and cooler in the winter.  Most parts of the world get rain at some times of the year and do not get rain at other times of the year. Some parts of the world have oceanic climates and others have alpine climates.\n\nClimate affects what food people eat. This is because climate affects what foods can grow.  If one food is easier to grow, people usually eat that food more often than other foods. Foods that people eat more of than other foods are called staple foods.  Staple foods are usually grains or vegetables because they are easy to grow.  Wheat, maize, millet, rice, oats, rye, potatoes, yams, breadfruit and beans are examples of different staple foods from around the world.\n\nClimate can affect the way people live in many other ways.


key: _node_content
string_value:
{"id_": "497ce8d0-b5f6-49ef-bf6e-f704b8c2bfcb", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "4b5a237d-23cc-4213-98ea-f7fc712b41f1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "7de17250763afce9dcde5fa8b63ae4a94a69b90e827e154b0d014be3f45981eb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "565af2f2-0e9a-46c7-ad10-210d9e11f19e", "node_type": "1", "metadata": {}, "hash": "7b7d12f2c4520527f126703d88d2b47307d62aac6e34849a0628d93d9363b25d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 48045, "end_char_idx": 51228, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It was different from a modern human skull so scientists believed it was from a new species. Entire Neanderthal skeletons have been found in other places since then. When ancient stone tools are found, their style often shows whether they were made by Homo sapiens or Neanderthals (see Palaeolithic). Neanderthals existed before modern humans. They knew how to use tools and fire.\n\nScientists believe that Homo sapiens spread from Africa to all other parts of the world, replacing Homo neanderthalensis in Europe and Homo erectus in Asia. By the end of the Stone Age, it is believed that Homo sapiens were the only type of humans left.\n\nInfluence of climate \n\nClimate is the normal weather in a place. It changes from one part of the world to another. Some areas are hot all year, and some are cold all year. Some areas are dry all year, and others are wet all year. Most areas have climates that are warmer in the summer and cooler in the winter.  Most parts of the world get rain at some times of the year and do not get rain at other times of the year. Some parts of the world have oceanic climates and others have alpine climates.\n\nClimate affects what food people eat. This is because climate affects what foods can grow.  If one food is easier to grow, people usually eat that food more often than other foods. Foods that people eat more of than other foods are called staple foods.  Staple foods are usually grains or vegetables because they are easy to grow.  Wheat, maize, millet, rice, oats, rye, potatoes, yams, breadfruit and beans are examples of different staple foods from around the world.\n\nClimate can affect the way people live in many other ways. It affects the types of animals that can live in any area, which affect the types of meats that are available to eat.\nClimate also affects the buildings that people make, the clothes that they wear and the way that they travel.\n\nClimate change \n\nThe climate on earth has not stayed the same through human history. There are long periods of time when it is generally warmer, and there are long periods of time when it is generally colder. When it is generally colder, there is more ice on the poles of the planet. A cold period is called an ice age. There have been many ice ages in the history of the earth. Two have affected humans.\n\nFrom 70,000 to around 10,000 years ago there was a big ice age which affected humans and the way that they lived. Between 1600\xa0AD and 1900\xa0AD there was a period called the Little Ice Age when the climate was a little bit colder than usual.\n\nPrehistory \n\nThe word "Prehistory" means "before history".  It is used for the long period of time before humans began to write about their lives.  This time is divided into two main ages: the Paleolithic Age (or Early Stone Age) and the Neolithic Age  (or late Stone Age).  The two ages did not start and end at the same time everywhere.  A place moved from one age to another depending on when people changed their technology.\n\nThe end of prehistory varies from one place to another. It depends on the date when that place began to use writing. In Egypt the first written documents date from around 3200\xa0BC.


key: _node_content
string_value:
{"id_": "565af2f2-0e9a-46c7-ad10-210d9e11f19e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "497ce8d0-b5f6-49ef-bf6e-f704b8c2bfcb", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "44cec163066d5dc640e1017ceacb1d852e0386b14df6c25e42e51ae2fdfdbeb7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "02e24da9-de34-4e46-86c0-dc87991c213c", "node_type": "1", "metadata": {}, "hash": "086997ea4becde911803d8101f6f621241fe4f3c319d814cda3f75b20788252d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 49721, "end_char_idx": 53014, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It affects the types of animals that can live in any area, which affect the types of meats that are available to eat.\nClimate also affects the buildings that people make, the clothes that they wear and the way that they travel.\n\nClimate change \n\nThe climate on earth has not stayed the same through human history. There are long periods of time when it is generally warmer, and there are long periods of time when it is generally colder. When it is generally colder, there is more ice on the poles of the planet. A cold period is called an ice age. There have been many ice ages in the history of the earth. Two have affected humans.\n\nFrom 70,000 to around 10,000 years ago there was a big ice age which affected humans and the way that they lived. Between 1600\xa0AD and 1900\xa0AD there was a period called the Little Ice Age when the climate was a little bit colder than usual.\n\nPrehistory \n\nThe word "Prehistory" means "before history".  It is used for the long period of time before humans began to write about their lives.  This time is divided into two main ages: the Paleolithic Age (or Early Stone Age) and the Neolithic Age  (or late Stone Age).  The two ages did not start and end at the same time everywhere.  A place moved from one age to another depending on when people changed their technology.\n\nThe end of prehistory varies from one place to another. It depends on the date when that place began to use writing. In Egypt the first written documents date from around 3200\xa0BC. In Australia the first written records date from 1788 and in New Guinea from about 1900.\n\nPaleolithic Era \n\nThe Paleolithic Era is by far the longest age of humanity\'s time, about 99% of human history. The Paleolithic Age started about 2.6  million years ago and ended around 10,000\xa0BC. The age began when hominids (early humans) started to use stones as tools for bashing, cutting and scraping. The age ended when humans began to plant crops and have other types of agriculture. In some areas, such as Western Europe, the way that people lived was affected by the Ice age. In these places, people moved towards agriculture quicker than in warmer places where there was always lots of food to gather. Their culture is sometimes called the Mesolithic Era (Middle Stone Age).\n\nDuring the Paleolithic Era humans grouped together in small bands. They lived by gathering plants and hunting wild animals. This way of living is called a "hunter-gatherer society".  People hunted small burrowing animals like rabbits, as well as birds and herds of animals like deer and cattle.  They also gathered plants to eat, including grains. Grain often grows on grasslands where herds of grass-eating animals are found.  People also gathered root vegetables, green vegetables, beans, fruit, seeds, berries, nuts, eggs, insects and small reptiles.\n\nMany Paleolithic bands were nomadic. They moved from place to place as the weather changed. They followed herds of animals that they hunted from their winter feeding places to their summer feeding places. If there was a drought,flood, or some other disaster, the herds and the people might haved moved a long distance, looking for food.  During the "Ice Age" a lot of the water on Earth turned to ice.  This made sea much lower than it is now.


key: _node_content
string_value:
{"id_": "02e24da9-de34-4e46-86c0-dc87991c213c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "565af2f2-0e9a-46c7-ad10-210d9e11f19e", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f9acced62772b2d7d8d3713b8b0e48db75c7237a731e1d45fc3d400c6b18856e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e83f296b-7743-4833-af26-9eb6b3a495dd", "node_type": "1", "metadata": {}, "hash": "449ffdd594592098e7c21067d0336a2c34a54e16017c576142b773a7ef3fad1c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 51436, "end_char_idx": 54744, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The Paleolithic Age started about 2.6  million years ago and ended around 10,000\xa0BC. The age began when hominids (early humans) started to use stones as tools for bashing, cutting and scraping. The age ended when humans began to plant crops and have other types of agriculture. In some areas, such as Western Europe, the way that people lived was affected by the Ice age. In these places, people moved towards agriculture quicker than in warmer places where there was always lots of food to gather. Their culture is sometimes called the Mesolithic Era (Middle Stone Age).\n\nDuring the Paleolithic Era humans grouped together in small bands. They lived by gathering plants and hunting wild animals. This way of living is called a "hunter-gatherer society".  People hunted small burrowing animals like rabbits, as well as birds and herds of animals like deer and cattle.  They also gathered plants to eat, including grains. Grain often grows on grasslands where herds of grass-eating animals are found.  People also gathered root vegetables, green vegetables, beans, fruit, seeds, berries, nuts, eggs, insects and small reptiles.\n\nMany Paleolithic bands were nomadic. They moved from place to place as the weather changed. They followed herds of animals that they hunted from their winter feeding places to their summer feeding places. If there was a drought,flood, or some other disaster, the herds and the people might haved moved a long distance, looking for food.  During the "Ice Age" a lot of the water on Earth turned to ice.  This made sea much lower than it is now.  People were able to walk through Beringia from Siberia to Alaska.  Bands of Homo sapiens ( another word for people) travelled to that area from Asia. At that time there were rich grasslands with many large animals that are now extinct. It is believed that many groups of people travelled there over a long time and later spread to other parts of America, as the weather changed.\n\nPaleolithic people used stone tools. Sometimes a stone tool was just a rock. It might have been useful for smashing a shell or an animal\'s skull, or for grinding grain on another rock. Other tools were made by breaking rocks to make a sharp edge. The next development in stone tool making was to chip all the edges of a rock so that it made a pointed shape, useful for a spearhead, or arrow tip. Some stone tools are carefully "flaked" at the edges to make them sharp, and symmetrically shaped. Paleolithic people also used tools of wood and bone. They probably also used leather and vegetable fibers but these have not lasted from that time. Paleolithic people also knew how to make fire which they used for warmth and cooking.\n\nThe Neolithic\n\nSettling down \n\nIn the Paleolithic Era there were many different human species. According to current research, only the modern human reached the Neolithic Era.\n\nThe Neolithic era was marked by changes in society. During the Neolithic era, people started to settle down. They developed agriculture and domesticated animals, both of which took a very long time. Because of these two things, people did not have to migrate as much any more. Villages could grow to much larger sizes than before. Over time, villages fought and spread their control over larger areas and some became civilisations.


key: _node_content
string_value:
{"id_": "e83f296b-7743-4833-af26-9eb6b3a495dd", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "02e24da9-de34-4e46-86c0-dc87991c213c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b1575c5a71a0a4d90ae90932894c150aa1db5a8ea5581e7b2ccfc76e6c462ec7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9cf70b61-d0c0-4c77-9473-9e40b29d9b82", "node_type": "1", "metadata": {}, "hash": "a41aa6a6b8462e2a3a755a26db33f53c4bb4528ba92aa84351976474e2a2aad9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 52974, "end_char_idx": 56316, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
This made sea much lower than it is now.  People were able to walk through Beringia from Siberia to Alaska.  Bands of Homo sapiens ( another word for people) travelled to that area from Asia. At that time there were rich grasslands with many large animals that are now extinct. It is believed that many groups of people travelled there over a long time and later spread to other parts of America, as the weather changed.\n\nPaleolithic people used stone tools. Sometimes a stone tool was just a rock. It might have been useful for smashing a shell or an animal\'s skull, or for grinding grain on another rock. Other tools were made by breaking rocks to make a sharp edge. The next development in stone tool making was to chip all the edges of a rock so that it made a pointed shape, useful for a spearhead, or arrow tip. Some stone tools are carefully "flaked" at the edges to make them sharp, and symmetrically shaped. Paleolithic people also used tools of wood and bone. They probably also used leather and vegetable fibers but these have not lasted from that time. Paleolithic people also knew how to make fire which they used for warmth and cooking.\n\nThe Neolithic\n\nSettling down \n\nIn the Paleolithic Era there were many different human species. According to current research, only the modern human reached the Neolithic Era.\n\nThe Neolithic era was marked by changes in society. During the Neolithic era, people started to settle down. They developed agriculture and domesticated animals, both of which took a very long time. Because of these two things, people did not have to migrate as much any more. Villages could grow to much larger sizes than before. Over time, villages fought and spread their control over larger areas and some became civilisations. During this time, humankind also developed further intellectually, militarily and spiritually.\n\nWhen humans started to grow crops and domesticate certain animals such as dogs, goats, sheep, and cattle;  their societies changed. Because people now grew crops and raised livestock, they started to stay in the same place and build permanent settlements. In most places, this happened between 10,000 and 12,000 years ago. Their diet also changed. People ate more cereals and vegetables. They started to keep extra foods and seeds for later. In some years there were surpluses (extras) that could be traded for other goods.\n\nThese changes happened independently in many parts of the world. They did not happen in the same order though. For example, the earliest farming societies in the Near East did not use pottery. No one is sure if Britain had agriculture, or if permanent villages existed there at all. Early Japanese societies used pottery before developing agriculture.\n\nVere Gordon Childe gave the name Neolithic Revolution to this process in the 1920s. He thought that it was as important as the Industrial Revolution (which happened in the 18th and 19th century).\n\nAncient history – the early civilizations \n\nAncient history was the time from the development of writing to the fall of the Roman Empire. The fall of the Roman Empire caused chaos in Europe, leading to the Middle Ages (also called the Dark Ages or the Age of Faith).\n\nThe first civilizations were built along major river systems. These civilizations are called river valley civilizations.


key: _node_content
string_value:
{"id_": "9cf70b61-d0c0-4c77-9473-9e40b29d9b82", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e83f296b-7743-4833-af26-9eb6b3a495dd", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "80fc6aa6eb02da3b286f48d12efa7a1b4af0b7bd006468f10425500d6311d841", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "cca5543b-cf10-4821-a8a0-0524b630a587", "node_type": "1", "metadata": {}, "hash": "e5b8b030f3de8ca1a7180c51c525cc314fb8b54b28ea6256e70ea9e57300b709", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 54512, "end_char_idx": 57936, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Because of these two things, people did not have to migrate as much any more. Villages could grow to much larger sizes than before. Over time, villages fought and spread their control over larger areas and some became civilisations. During this time, humankind also developed further intellectually, militarily and spiritually.\n\nWhen humans started to grow crops and domesticate certain animals such as dogs, goats, sheep, and cattle;  their societies changed. Because people now grew crops and raised livestock, they started to stay in the same place and build permanent settlements. In most places, this happened between 10,000 and 12,000 years ago. Their diet also changed. People ate more cereals and vegetables. They started to keep extra foods and seeds for later. In some years there were surpluses (extras) that could be traded for other goods.\n\nThese changes happened independently in many parts of the world. They did not happen in the same order though. For example, the earliest farming societies in the Near East did not use pottery. No one is sure if Britain had agriculture, or if permanent villages existed there at all. Early Japanese societies used pottery before developing agriculture.\n\nVere Gordon Childe gave the name Neolithic Revolution to this process in the 1920s. He thought that it was as important as the Industrial Revolution (which happened in the 18th and 19th century).\n\nAncient history – the early civilizations \n\nAncient history was the time from the development of writing to the fall of the Roman Empire. The fall of the Roman Empire caused chaos in Europe, leading to the Middle Ages (also called the Dark Ages or the Age of Faith).\n\nThe first civilizations were built along major river systems. These civilizations are called river valley civilizations. River valley civilizations were the most powerful civilizations in this time period because water was needed to have an agricultural society.\n\nThese civilizations were similar in that:\n They developed along river systems\n They had polytheistic religions\n They used writing systems\n\nMiddle East and North Africa\n\nSumer \n\nSumer was the world\'s first known ancient civilization. The Sumerians took over the fertile crescent region of Mesopotamia around 3300 BCE. They grew crops on the Tigris and Euphrates rivers. By 3000 BCE, many cities had been built in parts of Sumerian Mesopotamia. They formed independently and each had their own government. They were called city-states and often fought with each other.\n\nA surplus in food led to a Division of labour. This means that some people were able to stop growing crops and do other jobs, since enough crops were already grown. This brought a split in society. Today, such a split is called social pyramid. In a social pyramid, people are grouped into social classes based on their wealth and power. In Sumer, the king, priests, and government officials were at the top of the social pyramid. Below them were the artisans, merchants, farmers, and fishers. At the bottom of the pyramid were slaves. Slaves were often prisoners of war, criminals, or people working to pay off debt.\n\nThe Sumerians created the world\'s first system of writing; it was called cuneiform. The oldest versions of one of the world\'s first literary works, the Epic of Gilgamesh, go back to this time. In Sumer, only the sons of the rich and powerful learned how to read and write.


key: _node_content
string_value:
{"id_": "cca5543b-cf10-4821-a8a0-0524b630a587", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9cf70b61-d0c0-4c77-9473-9e40b29d9b82", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "6a3ac84d772f7408aa3d3fd2752d32f1da105b84b269c9feac454a9e0467055b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "647f229c-e970-4d9c-9b78-77c5b579f0a2", "node_type": "1", "metadata": {}, "hash": "7b7b181423136f07463092e4c11755d5a32feb18e996da67989e763c45f483f4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 56258, "end_char_idx": 59180, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
These civilizations are called river valley civilizations. River valley civilizations were the most powerful civilizations in this time period because water was needed to have an agricultural society.\n\nThese civilizations were similar in that:\n They developed along river systems\n They had polytheistic religions\n They used writing systems\n\nMiddle East and North Africa\n\nSumer \n\nSumer was the world\'s first known ancient civilization. The Sumerians took over the fertile crescent region of Mesopotamia around 3300 BCE. They grew crops on the Tigris and Euphrates rivers. By 3000 BCE, many cities had been built in parts of Sumerian Mesopotamia. They formed independently and each had their own government. They were called city-states and often fought with each other.\n\nA surplus in food led to a Division of labour. This means that some people were able to stop growing crops and do other jobs, since enough crops were already grown. This brought a split in society. Today, such a split is called social pyramid. In a social pyramid, people are grouped into social classes based on their wealth and power. In Sumer, the king, priests, and government officials were at the top of the social pyramid. Below them were the artisans, merchants, farmers, and fishers. At the bottom of the pyramid were slaves. Slaves were often prisoners of war, criminals, or people working to pay off debt.\n\nThe Sumerians created the world\'s first system of writing; it was called cuneiform. The oldest versions of one of the world\'s first literary works, the Epic of Gilgamesh, go back to this time. In Sumer, only the sons of the rich and powerful learned how to read and write. They went to a school called edubba. Only the boys who went to edubba could become scribes.\n\nThe Sumerians also invented sun-dried bricks, the wheel, the ox plow, and were skilled at making pottery. They are also thought to have invented the sailboat.\n\nAfter the Sumerians, the civilizations of Babylonia and then Assyria rose to power in Mesopotamia.\n\nBabylonia had a king named Hammurabi. He is famous for the Codex Hammurabi.\n\nJust to the east was the long-lasting civilization of Elam.\n\nAncient Egypt \n\nAncient Egypt grew along the Nile river. It was created around 3500\xa0BC. It was most powerful in the second millennium BC. When it was its biggest, it went all the way from the Nile delta to a mountain called Jebel Barkal in Sudan. It probably ended at about 30\xa0BC when the country was invaded by the Roman Empire.\n\nThe society of ancient Egypt depended on a balance of natural and human resources, especially the irrigation of the Nile Valley so that Egyptians could grow crops.\n\nThere was a great difference between classes in this society. Most of the people were farmers but they did not own the agricultural products they produced. These were property of the state, temple, or noble family that owned the land.


key: _node_content
string_value:
{"id_": "647f229c-e970-4d9c-9b78-77c5b579f0a2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "cca5543b-cf10-4821-a8a0-0524b630a587", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a9828ff69082a81417e47895608e520b8831b9a3fee7fac0a05dd08be806c8fd", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "18f175a1-1425-42c7-a1c4-a0f536df7b52", "node_type": "1", "metadata": {}, "hash": "203db568ec3cd211e083ea25970d8f4710b2b4b59e764859791fc6a6d6b8df44", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 57577, "end_char_idx": 60587, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Slaves were often prisoners of war, criminals, or people working to pay off debt.\n\nThe Sumerians created the world\'s first system of writing; it was called cuneiform. The oldest versions of one of the world\'s first literary works, the Epic of Gilgamesh, go back to this time. In Sumer, only the sons of the rich and powerful learned how to read and write. They went to a school called edubba. Only the boys who went to edubba could become scribes.\n\nThe Sumerians also invented sun-dried bricks, the wheel, the ox plow, and were skilled at making pottery. They are also thought to have invented the sailboat.\n\nAfter the Sumerians, the civilizations of Babylonia and then Assyria rose to power in Mesopotamia.\n\nBabylonia had a king named Hammurabi. He is famous for the Codex Hammurabi.\n\nJust to the east was the long-lasting civilization of Elam.\n\nAncient Egypt \n\nAncient Egypt grew along the Nile river. It was created around 3500\xa0BC. It was most powerful in the second millennium BC. When it was its biggest, it went all the way from the Nile delta to a mountain called Jebel Barkal in Sudan. It probably ended at about 30\xa0BC when the country was invaded by the Roman Empire.\n\nThe society of ancient Egypt depended on a balance of natural and human resources, especially the irrigation of the Nile Valley so that Egyptians could grow crops.\n\nThere was a great difference between classes in this society. Most of the people were farmers but they did not own the agricultural products they produced. These were property of the state, temple, or noble family that owned the land. There was slavery, but it is not clear how it was practiced.\nThe Religion of Ancient Egypt encouraged people to respect their rulers and their past.\n\nThe Egyptians are known for writing in hieroglyphs, building the famous pyramids, and building other sorts of tombs and big temples and for their military.\n\nThe religion of Judaism formed about 1500 BCE around the Egyptian and Babylonian civilizations.\n\nMid and Eastern Asia\n\nAncient China \n\nChina began as city-states in the Yellow River valley. The Shang Dynasty (商朝) was the first dynasty of Ancient China.Turtle shells with writing on them have been carbon dated to about 1500\xa0BC.\n\nThe Zhou Dynasty came after the Shang Dynasty. Kong Fuzi and Laozi lived at the end of the Zhou Dynasty. They were the greatest Chinese philosophers. They founded new philosophies, or ways of thinking. Confucius founded Confucianism and Laozi founded Daoism.\n\nAfter the Zhou Dynasty came the Warring States Period.\n\nThe Qin (秦) dynasty came after the Warring States Period. The Qin emperor Qin Shi Huang created the first centralized state in China in 221\xa0BC. It was based on his based on his political philosophy of legalism. He made everyone write the same way. He fought against Confucianism. He also started building what would later become the Great Wall.\n\nIn 202\xa0BC the Han Dynasty took over. It was about as strong as the Roman Empire.


key: _node_content
string_value:
{"id_": "18f175a1-1425-42c7-a1c4-a0f536df7b52", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "647f229c-e970-4d9c-9b78-77c5b579f0a2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a38bb3a0cf4eb4017f84d6cc8547750828bf79c87d346bed2369b56f1de43bd9", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "3757e871-6141-48f7-bba4-7a39cfb5d0ef", "node_type": "1", "metadata": {}, "hash": "77bcd75f1dd2d870d1567361cd9e118adcf1604cc13f17a8ccc90df963828945", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 59008, "end_char_idx": 62267, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Most of the people were farmers but they did not own the agricultural products they produced. These were property of the state, temple, or noble family that owned the land. There was slavery, but it is not clear how it was practiced.\nThe Religion of Ancient Egypt encouraged people to respect their rulers and their past.\n\nThe Egyptians are known for writing in hieroglyphs, building the famous pyramids, and building other sorts of tombs and big temples and for their military.\n\nThe religion of Judaism formed about 1500 BCE around the Egyptian and Babylonian civilizations.\n\nMid and Eastern Asia\n\nAncient China \n\nChina began as city-states in the Yellow River valley. The Shang Dynasty (商朝) was the first dynasty of Ancient China.Turtle shells with writing on them have been carbon dated to about 1500\xa0BC.\n\nThe Zhou Dynasty came after the Shang Dynasty. Kong Fuzi and Laozi lived at the end of the Zhou Dynasty. They were the greatest Chinese philosophers. They founded new philosophies, or ways of thinking. Confucius founded Confucianism and Laozi founded Daoism.\n\nAfter the Zhou Dynasty came the Warring States Period.\n\nThe Qin (秦) dynasty came after the Warring States Period. The Qin emperor Qin Shi Huang created the first centralized state in China in 221\xa0BC. It was based on his based on his political philosophy of legalism. He made everyone write the same way. He fought against Confucianism. He also started building what would later become the Great Wall.\n\nIn 202\xa0BC the Han Dynasty took over. It was about as strong as the Roman Empire. Towards the end of the Han Dynasty,  Buddhism became influential in China.\n\nAncient India/Pakistan \n\nThe Indus Valley Civilization lasted from about 2600\xa0BC to 1900\xa0BC. It was the first urban civilization on the subcontinent. It was centered on the Indus River and its tributaries. The civilization is famous for its brick cities that had road-side drainage systems and multi-storied houses.\n\nThe Maurya dynasty started in 321 BCE. This was the first time most of the Indian subcontinent was united under a single government. Ashoka the Great was a famous Mauryan emperor. When he started ruling, he sought to expand his empire, but then followed a policy of ahimsa (non-violence) after converting to Buddhism. He wrote about this in the Edicts of Ashoka. The Edicts of Ashoka are the oldest historical documents from India that still exist. While Ashoka ruled, Buddhist ideals spread across all of East Asia and South-East Asia.\n\nThe Gupta dynasty ruled from around 320 to 550\xa0AD. The Gupta Empire included only Central India, and the area east of current day Bangladesh. This empire never included present-day Pakistan to the west. Gupta society was ordered in accordance with Hindu beliefs. Historians place the Gupta dynasty alongside with the Han Dynasty, Tang Dynasty and Roman Empire as a model of a classical civilization.\n\nThe Americas\n\nAncient Maya \n\nThe Maya civilization is a civilization that started in Central America. They lived mostly on the Yucatán Peninsula in what is now known as Mexico, but also Honduras, Belize and Guatemala. They were the only known civilization of pre-Columbian America to have a fully developed written language.


key: _node_content
string_value:
{"id_": "3757e871-6141-48f7-bba4-7a39cfb5d0ef", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "18f175a1-1425-42c7-a1c4-a0f536df7b52", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d09716aa60d5dfeed7a35dd7236a0bc31629b386ca1ff864dafefd8c6cd619f7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "aeff6b16-e414-47d9-8b3e-0b50b37dc4ee", "node_type": "1", "metadata": {}, "hash": "a5120d8439932e3dc30978e9bb332c60051fe68cf27619314b3bd427f674dba7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 60544, "end_char_idx": 63932, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It was about as strong as the Roman Empire. Towards the end of the Han Dynasty,  Buddhism became influential in China.\n\nAncient India/Pakistan \n\nThe Indus Valley Civilization lasted from about 2600\xa0BC to 1900\xa0BC. It was the first urban civilization on the subcontinent. It was centered on the Indus River and its tributaries. The civilization is famous for its brick cities that had road-side drainage systems and multi-storied houses.\n\nThe Maurya dynasty started in 321 BCE. This was the first time most of the Indian subcontinent was united under a single government. Ashoka the Great was a famous Mauryan emperor. When he started ruling, he sought to expand his empire, but then followed a policy of ahimsa (non-violence) after converting to Buddhism. He wrote about this in the Edicts of Ashoka. The Edicts of Ashoka are the oldest historical documents from India that still exist. While Ashoka ruled, Buddhist ideals spread across all of East Asia and South-East Asia.\n\nThe Gupta dynasty ruled from around 320 to 550\xa0AD. The Gupta Empire included only Central India, and the area east of current day Bangladesh. This empire never included present-day Pakistan to the west. Gupta society was ordered in accordance with Hindu beliefs. Historians place the Gupta dynasty alongside with the Han Dynasty, Tang Dynasty and Roman Empire as a model of a classical civilization.\n\nThe Americas\n\nAncient Maya \n\nThe Maya civilization is a civilization that started in Central America. They lived mostly on the Yucatán Peninsula in what is now known as Mexico, but also Honduras, Belize and Guatemala. They were the only known civilization of pre-Columbian America to have a fully developed written language. They also made great achievements in art and architecture and had a very advanced system of mathematics and astronomy.\n\nThe area where the Maya civilization developed was inhabited from around the 10th millennium BC. The first Maya settlements were built there in about 1800\xa0BC, in the Soconusco region. This is in the modern-day state of Chiapas in Mexico, on the Pacific Ocean. Today, this is called the Early Preclassic period. At the time, humans began to settle down permanently. They started to grow livestock. Pottery and small clay figures were made. They constructed simple burial mounds. Later they developed these mounds into step pyramids. There were other civilizations around, especially in the north, such as the Olmec, the Mixe-Zoque, and Zapotec civilizations. These people mostly lived in the area of the modern-day state Oaxaca. The exact borders of the Maya empire in the north are unclear. There were probably areas where Maya culture overlapped with other cultures. Many of the earliest significant inscriptions and buildings appeared in this overlapping zone. These cultures and the Maya probably influenced one another.\n\nAustralia \nThere has been a long history of contact between Papuan peoples of the Papua New Guinea and the Aboriginal people. Aboriginal people seem to have lived a long time in the same environment as the now extinct Australian megafauna. Stories about that are told in the oral culture of many Aboriginal groups.\n\nAncient Europe\n\nHallstatt culture \n\nThe Hallstatt era is named after the city Hallstatt in Austria, where the first artifacts were found. It lasted from about 1200\xa0BC to about 275\xa0BC.


key: _node_content
string_value:
{"id_": "aeff6b16-e414-47d9-8b3e-0b50b37dc4ee", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "3757e871-6141-48f7-bba4-7a39cfb5d0ef", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e28ff19390546d0ee7a9868f15e6d91a56d2072b6da558306d926adf26ab486c", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a6f64558-5bb9-4768-a3d5-34b720252bcd", "node_type": "1", "metadata": {}, "hash": "1da0a2c276a8cc4c284409d5a4740f8624ec72bc25c78b0a924010f3797842a2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 62161, "end_char_idx": 65574, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They were the only known civilization of pre-Columbian America to have a fully developed written language. They also made great achievements in art and architecture and had a very advanced system of mathematics and astronomy.\n\nThe area where the Maya civilization developed was inhabited from around the 10th millennium BC. The first Maya settlements were built there in about 1800\xa0BC, in the Soconusco region. This is in the modern-day state of Chiapas in Mexico, on the Pacific Ocean. Today, this is called the Early Preclassic period. At the time, humans began to settle down permanently. They started to grow livestock. Pottery and small clay figures were made. They constructed simple burial mounds. Later they developed these mounds into step pyramids. There were other civilizations around, especially in the north, such as the Olmec, the Mixe-Zoque, and Zapotec civilizations. These people mostly lived in the area of the modern-day state Oaxaca. The exact borders of the Maya empire in the north are unclear. There were probably areas where Maya culture overlapped with other cultures. Many of the earliest significant inscriptions and buildings appeared in this overlapping zone. These cultures and the Maya probably influenced one another.\n\nAustralia \nThere has been a long history of contact between Papuan peoples of the Papua New Guinea and the Aboriginal people. Aboriginal people seem to have lived a long time in the same environment as the now extinct Australian megafauna. Stories about that are told in the oral culture of many Aboriginal groups.\n\nAncient Europe\n\nHallstatt culture \n\nThe Hallstatt era is named after the city Hallstatt in Austria, where the first artifacts were found. It lasted from about 1200\xa0BC to about 275\xa0BC. There were different periods, which today are mainly told apart by the kinds of brooches used at the time. These brooches changed rather rapidly, and can therefore give us good guesses at to what time they came from. Hallstatt culture sites have been found in the east of France, in Switzerland, in the south of Germany, in Austria, in Slovenia and Croatia, northwestern Hungary, southwestern Slovakia and southern Moravia. The culture can be divided into an eastern and a western one quite easily; the dividing line runs through the Czech Republic, and Austria, between longitudes 14 and 15 degrees east.\n\nIn this time, the social structure developed into a hierarchy. This can be documented by various things that were added to graves. In the Bronze Age, people used to live in big settlements. As iron became available, trade routes changed. A new richer class evolved. Unlike before, these richer class people liked to live in big houses in the countryside, as a demonstration of their wealth. Funerals also changed, from cremation burials, to burials with stone coffins. The new upper class used their wealth for import goods, mostly from the Mediterranean.\n\nLa Tène culture \n\nThe La Tène culture is a culture that lasted from about 500\xa0BC to about 100\xa0AD. It is named after the city of La Tène (today, Marin-Epagnier, next to Neuchâtel). It was influenced a lot by the Roman and Greek cultures. There are two sources for this:\n Objects found there\n Romans and Greeks came in contact with the culture. They called them Celts, usually. They wrote about them. The most important work about them was written by Julius Caesar.


key: _node_content
string_value:
{"id_": "a6f64558-5bb9-4768-a3d5-34b720252bcd", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "aeff6b16-e414-47d9-8b3e-0b50b37dc4ee", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b95182f6583cafed5f7b7c3dbd2e15f2b04ef7aee2a47ddc15ae727196939b3f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1a0b1d7b-669c-48a3-bc2f-80e972712557", "node_type": "1", "metadata": {}, "hash": "6cf62dcbcd3b3c7b7b4197ae21ca3639ab9ba4a4650d6e69d77972496d7c5cdf", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 63881, "end_char_idx": 66998, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It lasted from about 1200\xa0BC to about 275\xa0BC. There were different periods, which today are mainly told apart by the kinds of brooches used at the time. These brooches changed rather rapidly, and can therefore give us good guesses at to what time they came from. Hallstatt culture sites have been found in the east of France, in Switzerland, in the south of Germany, in Austria, in Slovenia and Croatia, northwestern Hungary, southwestern Slovakia and southern Moravia. The culture can be divided into an eastern and a western one quite easily; the dividing line runs through the Czech Republic, and Austria, between longitudes 14 and 15 degrees east.\n\nIn this time, the social structure developed into a hierarchy. This can be documented by various things that were added to graves. In the Bronze Age, people used to live in big settlements. As iron became available, trade routes changed. A new richer class evolved. Unlike before, these richer class people liked to live in big houses in the countryside, as a demonstration of their wealth. Funerals also changed, from cremation burials, to burials with stone coffins. The new upper class used their wealth for import goods, mostly from the Mediterranean.\n\nLa Tène culture \n\nThe La Tène culture is a culture that lasted from about 500\xa0BC to about 100\xa0AD. It is named after the city of La Tène (today, Marin-Epagnier, next to Neuchâtel). It was influenced a lot by the Roman and Greek cultures. There are two sources for this:\n Objects found there\n Romans and Greeks came in contact with the culture. They called them Celts, usually. They wrote about them. The most important work about them was written by Julius Caesar. It is called On the Gallic War (De bello gallico).\n\nThe Celts basically lived in clans. Each clan was headed by a leader, which came from the  Druids or the Bards. Women were much better off than with the Romans, they were almost equal to men. There was polygamy and polyandry (A man could have several women, a woman could have several men).\n\nIllyria \n\nIllyria  is the part of west-south Balkan Peninsula populated by Illyrians whose descendants are Albanians.\nIllyrians lived in tribunes such as Epirus, Dardania, Taulantia etc.\nThey had their own language, the Illyrian language that was different from the Greek language and Latin.\nAt the year 1000\xa0BC the population of Illyria is estimated to be around 500,000.\n\nAncient Greece \n\nWhat is known today as Ancient Greece is a very important period in history. Most people agree that it came after the Minoan and Mycenaean civilizations. It ended when the Romans invaded Greece, in 146\xa0BC. Greek culture had a very powerful influence on later civilizations, especially the Romans. The Greeks developed what is now called a city-state, or a polis. There were many polises. Some of the more important ones were Athens, Sparta, Corinth and Thebes. The word politics comes from there. It literally means: things that are about the polis. Greek cities did not have much contact with each other, because of the mountains and many islands Greece is made up of.


key: _node_content
string_value:
{"id_": "1a0b1d7b-669c-48a3-bc2f-80e972712557", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a6f64558-5bb9-4768-a3d5-34b720252bcd", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "56222beaf5b98ade164e3165c92dc7e9bf767e7e361780ad58f9056a68b5f93e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "70c5e199-3726-44a2-906a-910995b4fa00", "node_type": "1", "metadata": {}, "hash": "b7d6130d74fe88ce4d80d03d0ca5e439f027ab68217939884692463879f023eb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 65454, "end_char_idx": 68526, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They called them Celts, usually. They wrote about them. The most important work about them was written by Julius Caesar. It is called On the Gallic War (De bello gallico).\n\nThe Celts basically lived in clans. Each clan was headed by a leader, which came from the  Druids or the Bards. Women were much better off than with the Romans, they were almost equal to men. There was polygamy and polyandry (A man could have several women, a woman could have several men).\n\nIllyria \n\nIllyria  is the part of west-south Balkan Peninsula populated by Illyrians whose descendants are Albanians.\nIllyrians lived in tribunes such as Epirus, Dardania, Taulantia etc.\nThey had their own language, the Illyrian language that was different from the Greek language and Latin.\nAt the year 1000\xa0BC the population of Illyria is estimated to be around 500,000.\n\nAncient Greece \n\nWhat is known today as Ancient Greece is a very important period in history. Most people agree that it came after the Minoan and Mycenaean civilizations. It ended when the Romans invaded Greece, in 146\xa0BC. Greek culture had a very powerful influence on later civilizations, especially the Romans. The Greeks developed what is now called a city-state, or a polis. There were many polises. Some of the more important ones were Athens, Sparta, Corinth and Thebes. The word politics comes from there. It literally means: things that are about the polis. Greek cities did not have much contact with each other, because of the mountains and many islands Greece is made up of. When a city no longer had enough food to care for all its citizens, some people were sent out to set up a new city. This was called a colony. Each city was independent, and ruled by someone within that city. Colonies also looked to the city where they originally came from for guidance.\n\nWhen Greece went to war (for example against the Persian Empire), there was an alliance of such city states, against the Persians. There were also many wars between different city states.\n\nThere were many artists and philosophers who lived in that period. Most of them are still important for philosophy today. A well-known artist was Homer. He wrote epics about the war against the Trojans, and the early history of Greece. Other well-known artists were Aristophanes and Sappho. Well-known philosophers include Socrates, Plato, and Aristotle. A well known mathematician of the time was Euclid. Statesmen of the time were Pericles and Alexander the Great.\n\nAncient Rome \n\nAncient Rome was a civilization that started in modern-day Italy, in the 8th Century before Christ. The civilization lasted for 12 centuries. It ended, when Mehmed II conquered Constantinople, on May 29, 1453. According to legend, the Roman civilization was founded by Romulus and Remus, in the year 753\xa0BC. The Roman Empire developed in wars against Carthage and the Seleucid Empire. Julius Caesar conquered Gaul, modern France, and Augustus ended the Roman republic by becoming emperor. At its biggest extent, the empire covered all of the Mediterranean.


key: _node_content
string_value:
{"id_": "70c5e199-3726-44a2-906a-910995b4fa00", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1a0b1d7b-669c-48a3-bc2f-80e972712557", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b564bf6ba3e782662b0a1d24fd135abc2faca087411cca60a9432b2fde34c1bc", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f56c1d1b-2144-4473-b562-82f4616003e5", "node_type": "1", "metadata": {}, "hash": "b7829365d93209870338b1933b69c0969d49a0da5a29781cf213563facd1ac88", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 66790, "end_char_idx": 70079, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The word politics comes from there. It literally means: things that are about the polis. Greek cities did not have much contact with each other, because of the mountains and many islands Greece is made up of. When a city no longer had enough food to care for all its citizens, some people were sent out to set up a new city. This was called a colony. Each city was independent, and ruled by someone within that city. Colonies also looked to the city where they originally came from for guidance.\n\nWhen Greece went to war (for example against the Persian Empire), there was an alliance of such city states, against the Persians. There were also many wars between different city states.\n\nThere were many artists and philosophers who lived in that period. Most of them are still important for philosophy today. A well-known artist was Homer. He wrote epics about the war against the Trojans, and the early history of Greece. Other well-known artists were Aristophanes and Sappho. Well-known philosophers include Socrates, Plato, and Aristotle. A well known mathematician of the time was Euclid. Statesmen of the time were Pericles and Alexander the Great.\n\nAncient Rome \n\nAncient Rome was a civilization that started in modern-day Italy, in the 8th Century before Christ. The civilization lasted for 12 centuries. It ended, when Mehmed II conquered Constantinople, on May 29, 1453. According to legend, the Roman civilization was founded by Romulus and Remus, in the year 753\xa0BC. The Roman Empire developed in wars against Carthage and the Seleucid Empire. Julius Caesar conquered Gaul, modern France, and Augustus ended the Roman republic by becoming emperor. At its biggest extent, the empire covered all of the Mediterranean. Rome became so big, because it led war against other nations and then assimilated their culture.\n\nSplit of the Empire into East and West \nIn 293, Diocletian organized a separate administration of the western and the eastern part of the empire. The capital of the western part was Rome, the capital of the eastern part was Constantinople. Constantine I was the first to stop discrimination against Christians (313). Christianity became state religion under the reign of Theodosius I.\n\nThe western part of the empire had many problems with barbarians. In the 5th century, the Huns migrated westwards. This meant that the Visigoths moved into the empire, to seek protection. Rome was sacked by barbarians multiple times. On September 4, 476, the Germanic chief Odoacer forced the last Roman emperor in the west, Romulus Augustus, to quit. After about 1200 years, the rule of Rome in the West came to an end.\n\nThe eastern part had similar problems. Justinian I managed to conquer parts of North Africa and Italy. Shortly after he died, all that was left were parts of Southern Italy, and Sicily. In the east, the empire was threatened by the Sassanid Empire.\n\nNew departures and continuity \nAfter the fall of Western Rome, the Germanic tribes that took over tried to learn from Roman civilization, but much was forgotten and up to the Renaissance not many achievements happened in Europe. But with the rise of Islam, many changes happened during the Islamic Golden Age. The Greek and Roman traditions were kept and further development took place.


key: _node_content
string_value:
{"id_": "f56c1d1b-2144-4473-b562-82f4616003e5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "70c5e199-3726-44a2-906a-910995b4fa00", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a9b6d916d0cdf434280d1cbebb5146118f28aeb3839e68e082a55015de275043", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "fdfdfb04-e393-4e5b-982b-3b92ffcfa90d", "node_type": "1", "metadata": {}, "hash": "ca3dbbab2f1923313e7bed16a2e4c4db7af675a671fdc2d2d76b9d0e42bfd134", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 68355, "end_char_idx": 71543, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Julius Caesar conquered Gaul, modern France, and Augustus ended the Roman republic by becoming emperor. At its biggest extent, the empire covered all of the Mediterranean. Rome became so big, because it led war against other nations and then assimilated their culture.\n\nSplit of the Empire into East and West \nIn 293, Diocletian organized a separate administration of the western and the eastern part of the empire. The capital of the western part was Rome, the capital of the eastern part was Constantinople. Constantine I was the first to stop discrimination against Christians (313). Christianity became state religion under the reign of Theodosius I.\n\nThe western part of the empire had many problems with barbarians. In the 5th century, the Huns migrated westwards. This meant that the Visigoths moved into the empire, to seek protection. Rome was sacked by barbarians multiple times. On September 4, 476, the Germanic chief Odoacer forced the last Roman emperor in the west, Romulus Augustus, to quit. After about 1200 years, the rule of Rome in the West came to an end.\n\nThe eastern part had similar problems. Justinian I managed to conquer parts of North Africa and Italy. Shortly after he died, all that was left were parts of Southern Italy, and Sicily. In the east, the empire was threatened by the Sassanid Empire.\n\nNew departures and continuity \nAfter the fall of Western Rome, the Germanic tribes that took over tried to learn from Roman civilization, but much was forgotten and up to the Renaissance not many achievements happened in Europe. But with the rise of Islam, many changes happened during the Islamic Golden Age. The Greek and Roman traditions were kept and further development took place. The Chinese civilization had a Golden Age during the Tang period, when their capital was the biggest in the world. During the Renaissance, Europe developed and made great advancements in many areas as well.\n\nAsia\n\nMiddle East – Islamic rise, Byzantine decline \n\nIn Arabia, Muhammad founded Islam in 632.  His followers rapidly conquered territories in Syria and Egypt.  They soon were a danger to the Byzantine Empire.  In the 8th and 9th centuries, the Byzantine Empire stopped Islamic expansion and reconquered some lost territories.  In 1000 A.D. the eastern Empire was at its height: Basileios II reconquered Bulgaria and Armenia.  Culture and trade flourished.  In 1071 the Battle of Manzikert led the empire into a dramatic decline.  For the Byzantine Empire this meant centuries of civil wars and Turkic invasions.  The Muslim caliphate had an Golden Age under the Abbasids.\n\nTheir power forced Emperor Alexius I Comnenus of the Byzantine Empire to send a call for help to the West in 1095.  The West sent the Crusades.  These eventually led to the Sack of Constantinople in the Fourth Crusade in 1204.  Because of this, what was left of the Empire broke into successor states.  The winner of these disputes was that of Nicaea.  After Constantinople was again conquered by imperial forces, the empire was little more than a Greek state on the Aegean coast.  The Eastern Empire came to an end when Mehmed II conquered Constantinople on May 29, 1453.


key: _node_content
string_value:
{"id_": "fdfdfb04-e393-4e5b-982b-3b92ffcfa90d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f56c1d1b-2144-4473-b562-82f4616003e5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c756e9a61453b9ee9f9c3e0928d110b99f892f516e3f0719d4e719222ff2f2cc", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "db981b7c-c231-4ef3-b2c0-aef702b65573", "node_type": "1", "metadata": {}, "hash": "f5c1e51ecabc87568066d18d93b256ce40e8c5f0be4f75af96e987d18f4991e8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 69922, "end_char_idx": 72995, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
But with the rise of Islam, many changes happened during the Islamic Golden Age. The Greek and Roman traditions were kept and further development took place. The Chinese civilization had a Golden Age during the Tang period, when their capital was the biggest in the world. During the Renaissance, Europe developed and made great advancements in many areas as well.\n\nAsia\n\nMiddle East – Islamic rise, Byzantine decline \n\nIn Arabia, Muhammad founded Islam in 632.  His followers rapidly conquered territories in Syria and Egypt.  They soon were a danger to the Byzantine Empire.  In the 8th and 9th centuries, the Byzantine Empire stopped Islamic expansion and reconquered some lost territories.  In 1000 A.D. the eastern Empire was at its height: Basileios II reconquered Bulgaria and Armenia.  Culture and trade flourished.  In 1071 the Battle of Manzikert led the empire into a dramatic decline.  For the Byzantine Empire this meant centuries of civil wars and Turkic invasions.  The Muslim caliphate had an Golden Age under the Abbasids.\n\nTheir power forced Emperor Alexius I Comnenus of the Byzantine Empire to send a call for help to the West in 1095.  The West sent the Crusades.  These eventually led to the Sack of Constantinople in the Fourth Crusade in 1204.  Because of this, what was left of the Empire broke into successor states.  The winner of these disputes was that of Nicaea.  After Constantinople was again conquered by imperial forces, the empire was little more than a Greek state on the Aegean coast.  The Eastern Empire came to an end when Mehmed II conquered Constantinople on May 29, 1453.  The Ottoman Empire took its place and from 1400 to 1600 was the most powerful empire in the Middle East and ruled at the southern and eastern coast of the Mediterranean Sea.\n\nChina \nThe Tang Dynasty (618–907), with its capital at Chang\'an (today Xi\'an), was the biggest city in the world at the time and is considered by historians as a high point in Chinese civilization as well as a golden age of cosmopolitan culture. The Ming Dynasty ruled from 1368 to 1644. The Ming built a vast army and navy.\n\nIndia \nFrom around the 6th–7th century. In South India, Chola kings ruled Tamil Nadu, and Chera kings ruled Kerala. They had trading relationships with the Roman Empire to the west and Southeast Asia to the east. In north India, Rajputs ruled in many kingdoms.\n\nIn 1336, two brothers named Harihara I and Bukka founded the Vijayanagara Empire in an area which is now in the Karnataka state of India. The most famous king of this empire was Krishnadevaraya. In 1565, rulers of this empire were defeated in a battle. But the empire continued for about the next one hundred years.\nNorthern India was ruled by Islamic sultans.\n\nJapan \nThe Heian Period in Japan is famous for its art, poetry and literature. The writing system, Kana, was developed. It was followed by the feudal period (1185–1853) during which samurai and daimyos were the leading figures and the shogun the real monarch whereas the tennō had only a role as religious head.


key: _node_content
string_value:
{"id_": "db981b7c-c231-4ef3-b2c0-aef702b65573", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "fdfdfb04-e393-4e5b-982b-3b92ffcfa90d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b657d3aa0dbb932feaba9bd0c8e227f8d7ee850be61fad30cccf884fde40bf44", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "30251df4-c15f-43a8-8fd5-d220f265111d", "node_type": "1", "metadata": {}, "hash": "94258a44f946d9047f4b57124cc8add1c404ca5b34577ded1da82b1406b7db08", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 71453, "end_char_idx": 74654, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The Eastern Empire came to an end when Mehmed II conquered Constantinople on May 29, 1453.  The Ottoman Empire took its place and from 1400 to 1600 was the most powerful empire in the Middle East and ruled at the southern and eastern coast of the Mediterranean Sea.\n\nChina \nThe Tang Dynasty (618–907), with its capital at Chang\'an (today Xi\'an), was the biggest city in the world at the time and is considered by historians as a high point in Chinese civilization as well as a golden age of cosmopolitan culture. The Ming Dynasty ruled from 1368 to 1644. The Ming built a vast army and navy.\n\nIndia \nFrom around the 6th–7th century. In South India, Chola kings ruled Tamil Nadu, and Chera kings ruled Kerala. They had trading relationships with the Roman Empire to the west and Southeast Asia to the east. In north India, Rajputs ruled in many kingdoms.\n\nIn 1336, two brothers named Harihara I and Bukka founded the Vijayanagara Empire in an area which is now in the Karnataka state of India. The most famous king of this empire was Krishnadevaraya. In 1565, rulers of this empire were defeated in a battle. But the empire continued for about the next one hundred years.\nNorthern India was ruled by Islamic sultans.\n\nJapan \nThe Heian Period in Japan is famous for its art, poetry and literature. The writing system, Kana, was developed. It was followed by the feudal period (1185–1853) during which samurai and daimyos were the leading figures and the shogun the real monarch whereas the tennō had only a role as religious head. Between the years 1272 and 1281 the Mongols tried to invade but were driven out by the Japanese.\nIn 1542, a Portuguese ship reached Japan. Japanese learned about guns and firearms from them.\n\nMongols \nGenghis Khan in 1209 brought together the Mongol tribes and founded the Mongol Empire, one of the largest land empires in history. Later Kublai Khan would go on to expand the empire and found the Mongol-ruled Yuan Dynasty of China. The empire later broke into several empires, all of which were later destroyed.\n\nEuropean Middle Ages \n\nThe Middle Ages was the time from the fall of the Roman empire until the middle of the 15th century. From 500 to about 800 there was some decline compared with the Roman civilization. European villages were often destroyed and looted by barbarians such as the Vikings. During the High Middle Ages magnificent castles and large churches called cathedrals were built and important works of literature were written. In the later Middle Ages, there was a plague called the Black Death. The Black Death killed one-third to one-half of Europe\'s population.\n\nA system called feudalism was a very important part of the Middle Ages. In this system, the king was at the top of the social pyramid. The king gave land to the lord in exchange for loyalty. The lords were the next in the pyramid. The lords gave land (called a fief) to knights in exchange for loyalty and protection. The knights came next in the pyramid. Peasants were not part of the feudal system because they did not give or receive land. They worked on a lord\'s manor in exchange for protection.\n\nThe Crusades were also fought during the Middle Ages.


key: _node_content
string_value:
{"id_": "30251df4-c15f-43a8-8fd5-d220f265111d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "db981b7c-c231-4ef3-b2c0-aef702b65573", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "38dad26008d209e495e6bc3dfa157bc9fa2ebbf07b2f2a513c0e4d970911c1b6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f1332e96-1902-4216-b552-f5c61cda3ef9", "node_type": "1", "metadata": {}, "hash": "81202b6f0a80376d3cc4203f71c38c3817f6de409e7efa72bbe0303522441ef9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 72996, "end_char_idx": 76402, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Between the years 1272 and 1281 the Mongols tried to invade but were driven out by the Japanese.\nIn 1542, a Portuguese ship reached Japan. Japanese learned about guns and firearms from them.\n\nMongols \nGenghis Khan in 1209 brought together the Mongol tribes and founded the Mongol Empire, one of the largest land empires in history. Later Kublai Khan would go on to expand the empire and found the Mongol-ruled Yuan Dynasty of China. The empire later broke into several empires, all of which were later destroyed.\n\nEuropean Middle Ages \n\nThe Middle Ages was the time from the fall of the Roman empire until the middle of the 15th century. From 500 to about 800 there was some decline compared with the Roman civilization. European villages were often destroyed and looted by barbarians such as the Vikings. During the High Middle Ages magnificent castles and large churches called cathedrals were built and important works of literature were written. In the later Middle Ages, there was a plague called the Black Death. The Black Death killed one-third to one-half of Europe\'s population.\n\nA system called feudalism was a very important part of the Middle Ages. In this system, the king was at the top of the social pyramid. The king gave land to the lord in exchange for loyalty. The lords were the next in the pyramid. The lords gave land (called a fief) to knights in exchange for loyalty and protection. The knights came next in the pyramid. Peasants were not part of the feudal system because they did not give or receive land. They worked on a lord\'s manor in exchange for protection.\n\nThe Crusades were also fought during the Middle Ages. There is a theory that says the Crusades helped end the Middle Ages along with the Black Death, increased trade and better farming technology.\n\nRenaissance \n\nThe Renaissance started in Italy. Renaissance is a French word meaning "rebirth". The Renaissance meant that people learned from the ancient Greek and Roman or "classical" cultures that had been forgotten for some time. Artists learned from classical paintings and sculptures. So they reinvented perspective and the art of free standing realistic sculptures that had been characteristic in Greek and Roman art. Some famous Renaissance artists are Leonardo da Vinci, Michelangelo, and Raphael. The Gutenberg printing press, invented by Johannes Gutenberg, was also developed during this time.\n\nThe Renaissance was also a time of great achievements in science (Galileo Galilei, Francis Bacon), philosophy (Thomas More) and literature (Dante Alighieri, William Shakespeare).\n\nAmerica\n\nMaya civilization (classical period) \n\nWhat is known as the classical period lasted from about 250 to about 900. During this time, many monuments were constructed. There are also many big inscriptions from then. In this period, the Maya moved to building large cities. This is known as urbanism. Many important intellectual and artistic developments happened in an area that is known as the southern lowlands.\n\nLike the Ancient Greek, the Maya civilization was made of many independent city-states. Agriculture was important around these city states like Tikal and Copán.\nThe most important monuments are the pyramids they built in their religious centers and the palaces of their rulers. The palace at Cancuén is the largest in the Maya area. There are no pyramids in the area of the palace.


key: _node_content
string_value:
{"id_": "f1332e96-1902-4216-b552-f5c61cda3ef9", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "30251df4-c15f-43a8-8fd5-d220f265111d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "1f8d2dc3daf7be34113b76953be79c894d4d3d0789f07ff151886594d46974d6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2b1afba0-655c-4055-a0e1-e7827de1e16b", "node_type": "1", "metadata": {}, "hash": "77b0ffc91a3532f8c8c026878e93af591ec9e7e441770c11471ffa829061e457", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 74539, "end_char_idx": 77969, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They worked on a lord\'s manor in exchange for protection.\n\nThe Crusades were also fought during the Middle Ages. There is a theory that says the Crusades helped end the Middle Ages along with the Black Death, increased trade and better farming technology.\n\nRenaissance \n\nThe Renaissance started in Italy. Renaissance is a French word meaning "rebirth". The Renaissance meant that people learned from the ancient Greek and Roman or "classical" cultures that had been forgotten for some time. Artists learned from classical paintings and sculptures. So they reinvented perspective and the art of free standing realistic sculptures that had been characteristic in Greek and Roman art. Some famous Renaissance artists are Leonardo da Vinci, Michelangelo, and Raphael. The Gutenberg printing press, invented by Johannes Gutenberg, was also developed during this time.\n\nThe Renaissance was also a time of great achievements in science (Galileo Galilei, Francis Bacon), philosophy (Thomas More) and literature (Dante Alighieri, William Shakespeare).\n\nAmerica\n\nMaya civilization (classical period) \n\nWhat is known as the classical period lasted from about 250 to about 900. During this time, many monuments were constructed. There are also many big inscriptions from then. In this period, the Maya moved to building large cities. This is known as urbanism. Many important intellectual and artistic developments happened in an area that is known as the southern lowlands.\n\nLike the Ancient Greek, the Maya civilization was made of many independent city-states. Agriculture was important around these city states like Tikal and Copán.\nThe most important monuments are the pyramids they built in their religious centers and the palaces of their rulers. The palace at Cancuén is the largest in the Maya area. There are no pyramids in the area of the palace. Other important things the archaeologists found include the carved stone slabs usually called stelae (the Maya called them tetun, or "tree-stones"). These slabs show rulers along with hieroglyphic texts describing their genealogy, military victories, and other accomplishments. In North America, they made Mississipian culture with the largest land field from around 800 CE to 1600.\n\nTrade with other civilizations \nThe Maya also had trade routes that ran over long distances. They traded with many of the other Mesoamerican cultures, such as Teotihuacan, the Zapotec, and other groups in central and gulf-coast Mexico. They also traded with non-Mesoamerican groups, that were farther away. Archaeologists have found gold from Panama in the Sacred Cenote of Chichen Itza.\n\nImportant trade goods were cacao, salt, sea shells, jade and obsidian.\n\nSudden collapse \nIn the 8th and 9th century, the cities in the southern lowlands had problems, and declined. At the same time, the Maya stopped making big monuments and inscriptions. Shortly afterwards, these cities were abandoned. Currently, archaeologists are not sure why this happened. There are different theories. Either ecological factors played a role in this, or the cause of this abandonment was not related to the environment.\n\nPost-classical period and decline \n\nIn the north, development went on, form the 10th to about the 16th century. The influences from the outside left more traces in the Maya culture at that time. Some of the important sites in this era were Chichen Itza, Uxmal, and Coba.


key: _node_content
string_value:
{"id_": "2b1afba0-655c-4055-a0e1-e7827de1e16b", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f1332e96-1902-4216-b552-f5c61cda3ef9", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d3cc5baeed2f60cff9da2e69885e124ef68ec64fccaa947a4d8a8b6486c1737a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a045392a-b689-49cc-9e1e-18783454de64", "node_type": "1", "metadata": {}, "hash": "db035958254acf8d7e9e6422d64a075e7c0ac8769f40b9e12bf5e1ae61da68c4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 76299, "end_char_idx": 79432, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The palace at Cancuén is the largest in the Maya area. There are no pyramids in the area of the palace. Other important things the archaeologists found include the carved stone slabs usually called stelae (the Maya called them tetun, or "tree-stones"). These slabs show rulers along with hieroglyphic texts describing their genealogy, military victories, and other accomplishments. In North America, they made Mississipian culture with the largest land field from around 800 CE to 1600.\n\nTrade with other civilizations \nThe Maya also had trade routes that ran over long distances. They traded with many of the other Mesoamerican cultures, such as Teotihuacan, the Zapotec, and other groups in central and gulf-coast Mexico. They also traded with non-Mesoamerican groups, that were farther away. Archaeologists have found gold from Panama in the Sacred Cenote of Chichen Itza.\n\nImportant trade goods were cacao, salt, sea shells, jade and obsidian.\n\nSudden collapse \nIn the 8th and 9th century, the cities in the southern lowlands had problems, and declined. At the same time, the Maya stopped making big monuments and inscriptions. Shortly afterwards, these cities were abandoned. Currently, archaeologists are not sure why this happened. There are different theories. Either ecological factors played a role in this, or the cause of this abandonment was not related to the environment.\n\nPost-classical period and decline \n\nIn the north, development went on, form the 10th to about the 16th century. The influences from the outside left more traces in the Maya culture at that time. Some of the important sites in this era were Chichen Itza, Uxmal, and Coba. At some point, the ruling dynasties of Chichen and Uxmal declined. Afterwards, Mayapan ruled all of Yucatán until a revolt in 1450. The area then degenerated into competing city-states until the Yucatán was conquered by the Spanish.\n\nBy 1250, there developed other city-states. The Itza maintained their capital at Tayasal. It ruled over an area extending across the Peten Lakes region, including the community of Ekckixil on Lake Quexil. Postclassic Maya states also survived in the southern highlands. One of the Maya kingdoms in this area is responsible for the best-known Maya work of historiography and mythology, the Popol Vuh.\n\nThe Spanish started to conquer Maya lands. This took them much longer than with the Inca or Aztecs, because there was no capital city. This meant that when they had conquered one city, this had little influence on the whole empire. The last Maya states were finally subdued in 1697.\n\nThe Maya people did not disappear though. There are still about 6 million of them. Some are well-integrated, others continue speak one of the Maya languages and uphold their cultural heritage.\n\nThe Aztecs \n\nThe Aztecs built an empire in Central America, mainly in Mexico. The empire lasted from the 14th to the 16th century. They spoke the Nahuatl language. Their capital was Tenochtitlan. It was built on islands in a lake. Tenochtitlan was one of the greatest cities of the world in that time.\n\nThe Aztecs believed in polytheism.


key: _node_content
string_value:
{"id_": "a045392a-b689-49cc-9e1e-18783454de64", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "5bd736d3-7701-4f3a-aa43-d008629204e7", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "12a1654dd81f22f5244b3706fb59fa84d12b3aaee405fa5d9ef919f2c1eabdc2", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2b1afba0-655c-4055-a0e1-e7827de1e16b", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb", "title": "Retrieval augmented generation using Elasticsearch and OpenAI", "tokens": 28563, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5a6c976ecd4e937ea43644d7327900d83da5e8ec812666dc20c1180572fd3028", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "01ef9dde-a57f-4a1c-8731-fdff4a949089", "node_type": "1", "metadata": {}, "hash": "1c3a4c6b8c8c41fd9262e7e64e6d6f0c5c319dddfe63c50401c11f7724601ac3", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 77894, "end_char_idx": 80816, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Some of the important sites in this era were Chichen Itza, Uxmal, and Coba. At some point, the ruling dynasties of Chichen and Uxmal declined. Afterwards, Mayapan ruled all of Yucatán until a revolt in 1450. The area then degenerated into competing city-states until the Yucatán was conquered by the Spanish.\n\nBy 1250, there developed other city-states. The Itza maintained their capital at Tayasal. It ruled over an area extending across the Peten Lakes region, including the community of Ekckixil on Lake Quexil. Postclassic Maya states also survived in the southern highlands. One of the Maya kingdoms in this area is responsible for the best-known Maya work of historiography and mythology, the Popol Vuh.\n\nThe Spanish started to conquer Maya lands. This took them much longer than with the Inca or Aztecs, because there was no capital city. This meant that when they had conquered one city, this had little influence on the whole empire. The last Maya states were finally subdued in 1697.\n\nThe Maya people did not disappear though. There are still about 6 million of them. Some are well-integrated, others continue speak one of the Maya languages and uphold their cultural heritage.\n\nThe Aztecs \n\nThe Aztecs built an empire in Central America, mainly in Mexico. The empire lasted from the 14th to the 16th century. They spoke the Nahuatl language. Their capital was Tenochtitlan. It was built on islands in a lake. Tenochtitlan was one of the greatest cities of the world in that time.\n\nThe Aztecs believed in polytheism. Quetzalcoatl (feathered snake), Huitzilopochtli (hummingbird of the south) and Tezcatlipoca (smoking mirror) were the most important Gods. Sometimes the Aztecs killed humans to please their gods. Between 1519 and 1521 the Spanish leader Hernán Cortés defeated the Aztecs and took their empire. Some Aztecs did not want to fight against the soldiers of Cortés, because they thought they were Gods.\n\nToday many Mexicans have Aztec and other Native American forefathers. People still use Aztec symbols in Mexico. On the Mexican flag there is a picture of an eagle on a cactus with a snake in its mouth. This was an Aztec symbol. Also the name Mexico is an Aztec word.\n\nThe Aztecs ate a lot of plants and vegetables that could be grown easily in the Mexico area. The main food that they ate was corn, which they called maize. Another food that they ate was squash.\n\nAztecs also had a lot of harsh punishments for certain crimes. For the following crimes the punishment was death: adultery, wearing cotton clothes (cotton clothes were only for the nobles), cutting down a living tree, moving a field boundary making your land bigger, making someone else\'s smaller, major theft and treason.\n\nThe Incas \n\nThe Incas were a civilized empire in western South America. The Incas are called a "pre-Columbian" empire. This means that their country was here before Christopher Columbus.


