# embedding_metadata

key: _node_content
string_value:
{"id_": "d742300b-9847-4408-8eef-bfd41ace1c20", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "022bffa3-a899-4579-b0b3-3d90289cd180", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c46ee5d5f71de00222c4a9ba0734debe7fafa79b8f6f33773f78181545f04fe0", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ea2e6ffb-4fef-4277-9cd0-06b96ec0aabd", "node_type": "1", "metadata": {}, "hash": "8d2a38efe835b1742839571f706c974334bfff46efaea28bf2eb14ed7aa0238b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 36409, "end_char_idx": 39708, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Parrots are not songbirds, even though they can be taught to sing human songs.\n\n Songbirds \nAll birds make noises (\'\'bird vocalisation\'\'), but not all sing. Songbirds are passerines, many of which have beautiful melodic songs. Songs have different functions. Danger cries are different from territorial songs and mating calls are a third type. Fledgling may also have different calls from adults. Recognition calls for partners are quite common.\n\nAs to where the song comes from, there are three kinds of species:\nThose where the song is mainly inherited, and the bird always sings the same song in the same situations. The capacity is inherited, and only details are learnt from its neighbours.\nThose where the song is partly inherited, but the bird tunes it in by copying others. In this case the slight differences between the calls of different birds may be used by partners for identification.\nThose where the song is entirely learnt, and the bird often copies sounds from its environment. Only the capacity to sing is inherited.\n\nMost singing birds that are kept as pets, like canaries, have several tunes and some variations.\n\nThe same species of bird will sing different songs in different regions. A good example of this is the currawong. This is an Australia bird which is like a black and white crow. In the autumn, families get together in large flocks and do a lot of singing. Currawongs from some areas sing much more complex songs than others. Generally, currawongs from the Blue Mountains are the finest singers. The song of the currawong can be sung as a solo, but is often performed as a choir. One bird will take the lead and sing "Warble-warble-warble-warble!" All the other birds will join in and sing "Wooooooo!". When all the birds know the song, the choir will sing the "Warble" part and the soloist will sing the "Woo!". The song changes from year to year and from place to place.\n\n Lorenz\'s studies \nThe Austrian naturalist Konrad Lorenz studied the way in which birds communicate, or talk to each other. He found that each type of bird had a number of sounds which they made automatically, when ever they felt a certain way. Every sound had an action that went with it. So, if the bird was frightened, it acted frightened and made a frightened sound. This told the other birds around it that something frightening was happening.\n\nIf a flock of birds were flying over a field, they would be calling "Fly! Fly!" But a hungry bird, seeing something good to eat down below might start calling "Food! Food!" If other birds were also hungry, they would make the same call until more birds were calling "Food! Food!" than "Fly! Fly!". At this point, the mind of the flock would be changed. Some of the birds would start to yell "Fly downwards! Fly downwards!" as they sank from the sky, until the whole flock was all noisily calling the same thing.\n\nThese communication sounds are often short hard sounds like: chirps, squeaks, squawks and twitters. Sometimes the calls are longer and more musical. They include the "Rookety-coo" sound of a pigeon and the "Cockadoodledoo!" of a rooster. The bird cannot change these sounds. They always make them in the same way. The bird is locked into making each sound every time a particular idea comes into its head.


key: _node_content
string_value:
{"id_": "ea2e6ffb-4fef-4277-9cd0-06b96ec0aabd", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d742300b-9847-4408-8eef-bfd41ace1c20", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "da86953a16850211df6ddc71d761d78127265eb76dc0ee1cb1a278d65ea9608e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a538f1d5-5606-424c-9052-a3b57a6f3c57", "node_type": "1", "metadata": {}, "hash": "3470067cfc11da587f25c593a3f4d86f42b8ef65780c1365de2fc052aac9f5b5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 38106, "end_char_idx": 41239, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
All the other birds will join in and sing "Wooooooo!". When all the birds know the song, the choir will sing the "Warble" part and the soloist will sing the "Woo!". The song changes from year to year and from place to place.\n\n Lorenz\'s studies \nThe Austrian naturalist Konrad Lorenz studied the way in which birds communicate, or talk to each other. He found that each type of bird had a number of sounds which they made automatically, when ever they felt a certain way. Every sound had an action that went with it. So, if the bird was frightened, it acted frightened and made a frightened sound. This told the other birds around it that something frightening was happening.\n\nIf a flock of birds were flying over a field, they would be calling "Fly! Fly!" But a hungry bird, seeing something good to eat down below might start calling "Food! Food!" If other birds were also hungry, they would make the same call until more birds were calling "Food! Food!" than "Fly! Fly!". At this point, the mind of the flock would be changed. Some of the birds would start to yell "Fly downwards! Fly downwards!" as they sank from the sky, until the whole flock was all noisily calling the same thing.\n\nThese communication sounds are often short hard sounds like: chirps, squeaks, squawks and twitters. Sometimes the calls are longer and more musical. They include the "Rookety-coo" sound of a pigeon and the "Cockadoodledoo!" of a rooster. The bird cannot change these sounds. They always make them in the same way. The bird is locked into making each sound every time a particular idea comes into its head. The connection between how they feel and how they call is innate: they are born with it. Some calls in some species are learnt. Then, it is the tendency to learn which is inherited.\n\n The Jackdaw of Altenberg \nKonrad Lorenz noticed that when birds sing, they often use a lot of their regular calls as part of the song. Lorenz had a flock of jackdaws which were scattered during World War II. One day, an old bird returned. For many months she sat on the chimney singing her song, but in the song she kept making the call which Lorenz knew meant "Come home! Come home!" One day, to the great surprise of Lorenz, a male bird flew from a passing flock and joined her on the chimney. Lorenz was sure that it was her long-lost "husband" who had found his way home at last.\n\n Evolution and taxonomy \n\nPalaeontologists have found some exceptional places (lagerstätten) where fossils of early birds are found. The preservation is so good that on the best examples impressions of their feathers can be seen, and sometimes even the remains of meals they have eaten. From these remains we know that birds evolved from small carnivorous dinosaurs (theropods) in the Jurassic period. They radiated into a huge variety in the Lower Cretaceous. At the same time, their direct competitors, the pterosaurs, dwindled in numbers and variety, and became extinct at the end of the Mesozoic.\n\nBirds are classified by taxonomists as \'Aves\' (Avialae). Birds are the only living descendants of dinosaurs (strictly speaking, they are dinosaurs).


key: _node_content
string_value:
{"id_": "a538f1d5-5606-424c-9052-a3b57a6f3c57", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ea2e6ffb-4fef-4277-9cd0-06b96ec0aabd", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9a50f02db7ce408a0f6fde0d64d536084cdb85f1bc177b73e4f0232e7c8fe26f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e61a30eb-b074-4661-8fe6-dbec0350c419", "node_type": "1", "metadata": {}, "hash": "8b25cc782a3ec5b8964fff3f48acb32f0c513d3a9552ae1508c149935184a8d3", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 39578, "end_char_idx": 42802, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They always make them in the same way. The bird is locked into making each sound every time a particular idea comes into its head. The connection between how they feel and how they call is innate: they are born with it. Some calls in some species are learnt. Then, it is the tendency to learn which is inherited.\n\n The Jackdaw of Altenberg \nKonrad Lorenz noticed that when birds sing, they often use a lot of their regular calls as part of the song. Lorenz had a flock of jackdaws which were scattered during World War II. One day, an old bird returned. For many months she sat on the chimney singing her song, but in the song she kept making the call which Lorenz knew meant "Come home! Come home!" One day, to the great surprise of Lorenz, a male bird flew from a passing flock and joined her on the chimney. Lorenz was sure that it was her long-lost "husband" who had found his way home at last.\n\n Evolution and taxonomy \n\nPalaeontologists have found some exceptional places (lagerstätten) where fossils of early birds are found. The preservation is so good that on the best examples impressions of their feathers can be seen, and sometimes even the remains of meals they have eaten. From these remains we know that birds evolved from small carnivorous dinosaurs (theropods) in the Jurassic period. They radiated into a huge variety in the Lower Cretaceous. At the same time, their direct competitors, the pterosaurs, dwindled in numbers and variety, and became extinct at the end of the Mesozoic.\n\nBirds are classified by taxonomists as \'Aves\' (Avialae). Birds are the only living descendants of dinosaurs (strictly speaking, they are dinosaurs). Birds and Crocodilia are the only living members of the once-dominant Archosaur reptiles.\n\n Definition \nThe class Aves is was defined (1990) as all the descendants of the most recent common ancestor of modern birds and Archaeopteryx lithographica. But Archaeopteryx is almost certainly not the ancestor of modern birds. The transition to flight happened a number of times. The researchers offered four definitions. Birds can be: \nAll archosaurs closer to birds than crocodiles (Avemetatarsalia).\nAdvanced archosaurs with feathers (Avofilopluma).\nThose feathered dinosaurs that fly (or Avialae)\nAves can mean the last common ancestor of all living birds and all of its descendants (a "crown group", in this sense synonymous with Neornithes).\n\n The first bird-like creatures Archaeopteryx, from the Upper Jurassic some 150–145 million years ago (mya), was for a long time the earliest known bird which could fly. It is famous, because it was one of the first important fossils found after Charles Darwin published his ideas about evolution in the 19th century. By modern standards, Archaeopteryx could not fly very well. Other early fossil birds are, for example, Confuciusornis, Anchiornis huxlei and other Paraves.\n\nMany fossils of early birds and small dinosaurs have been discovered in the Liaoning Province of Northeast China. These include Anchiornis huxlei, from about 160 mya. The fossils show that most small theropod dinosaurs had feathers. These deposits have preserved them so well that the impressions of their feathers can be clearly seen.


key: _node_content
string_value:
{"id_": "e61a30eb-b074-4661-8fe6-dbec0350c419", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a538f1d5-5606-424c-9052-a3b57a6f3c57", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9b84846f3f7ba33384aa2de664ef363447cff8bcd85ceb531a923d1ad4af9dcf", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "25f69d84-b095-4605-b3ea-cbe6e0ed19db", "node_type": "1", "metadata": {}, "hash": "4ae1abb1169760e6f0d8e22213ce422e30d7a242921b459c1997fc562800598e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 41148, "end_char_idx": 44338, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Birds are the only living descendants of dinosaurs (strictly speaking, they are dinosaurs). Birds and Crocodilia are the only living members of the once-dominant Archosaur reptiles.\n\n Definition \nThe class Aves is was defined (1990) as all the descendants of the most recent common ancestor of modern birds and Archaeopteryx lithographica. But Archaeopteryx is almost certainly not the ancestor of modern birds. The transition to flight happened a number of times. The researchers offered four definitions. Birds can be: \nAll archosaurs closer to birds than crocodiles (Avemetatarsalia).\nAdvanced archosaurs with feathers (Avofilopluma).\nThose feathered dinosaurs that fly (or Avialae)\nAves can mean the last common ancestor of all living birds and all of its descendants (a "crown group", in this sense synonymous with Neornithes).\n\n The first bird-like creatures Archaeopteryx, from the Upper Jurassic some 150–145 million years ago (mya), was for a long time the earliest known bird which could fly. It is famous, because it was one of the first important fossils found after Charles Darwin published his ideas about evolution in the 19th century. By modern standards, Archaeopteryx could not fly very well. Other early fossil birds are, for example, Confuciusornis, Anchiornis huxlei and other Paraves.\n\nMany fossils of early birds and small dinosaurs have been discovered in the Liaoning Province of Northeast China. These include Anchiornis huxlei, from about 160 mya. The fossils show that most small theropod dinosaurs had feathers. These deposits have preserved them so well that the impressions of their feathers can be clearly seen. This leads us to think that feathers evolved first as heat insulation and only later for flight. The origin of birds lies in these small feathered dinosaurs.\n\nPalaeontologists now agree that birds are included in Maniraptora group of dinosaurs. This explains why we say that birds are living dinosaurs.\n\n Evolution of modern birds \nA leading authority says "Most living birds have fossil representatives in the Cenozoic"... "Key problems remain in understanding bird phylogeny... we seem to understand as little about the relationships among living birds as among Cretaceous birds".\n\n Origin of birds\n Paraves\n\nBirds and people \n\nSome birds are eaten as food. Most usually it is the chicken and its eggs, but people often also eat geese, pheasants, turkeys and ducks. Other birds are sometimes eaten are: emus, ostriches, pigeons, grouse, quails, doves, woodcocks and even songbirds. Some species have died out because they have been hunted for food, for example the dodo and the passenger pigeon.\n\nMany species have learned how to get food from people. The number of birds of these species has grown because of it. Seagulls and crows find food from garbage dumps. The feral pigeon (Columba livia), sparrows (Passer domesticus and starlings (Sturnus vulgaris) live in large numbers in towns and cities all over the world.\n\nSometimes people also use working birds. For example, homing pigeons carry messages. Nowadays people sometimes race them for sport. People also use falcons for hunting, and cormorants for fishing.


key: _node_content
string_value:
{"id_": "25f69d84-b095-4605-b3ea-cbe6e0ed19db", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e61a30eb-b074-4661-8fe6-dbec0350c419", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "103ef8e1535149b47794b4aa62be6c898f3c19dbac04abd33b32c1ccd45a92e6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5c08e931-46d1-4cb5-9488-b0fc6c9a4083", "node_type": "1", "metadata": {}, "hash": "5aeb01015287806a863067b3d728852e714127786cf5f460bf224767403eb05f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 42634, "end_char_idx": 45228, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The fossils show that most small theropod dinosaurs had feathers. These deposits have preserved them so well that the impressions of their feathers can be clearly seen. This leads us to think that feathers evolved first as heat insulation and only later for flight. The origin of birds lies in these small feathered dinosaurs.\n\nPalaeontologists now agree that birds are included in Maniraptora group of dinosaurs. This explains why we say that birds are living dinosaurs.\n\n Evolution of modern birds \nA leading authority says "Most living birds have fossil representatives in the Cenozoic"... "Key problems remain in understanding bird phylogeny... we seem to understand as little about the relationships among living birds as among Cretaceous birds".\n\n Origin of birds\n Paraves\n\nBirds and people \n\nSome birds are eaten as food. Most usually it is the chicken and its eggs, but people often also eat geese, pheasants, turkeys and ducks. Other birds are sometimes eaten are: emus, ostriches, pigeons, grouse, quails, doves, woodcocks and even songbirds. Some species have died out because they have been hunted for food, for example the dodo and the passenger pigeon.\n\nMany species have learned how to get food from people. The number of birds of these species has grown because of it. Seagulls and crows find food from garbage dumps. The feral pigeon (Columba livia), sparrows (Passer domesticus and starlings (Sturnus vulgaris) live in large numbers in towns and cities all over the world.\n\nSometimes people also use working birds. For example, homing pigeons carry messages. Nowadays people sometimes race them for sport. People also use falcons for hunting, and cormorants for fishing. In the past, people in mines often used a canary to see if there were bad gas methane in the air.\n\nPeople often have colorful birds such as parrots and mynahs as pets. These intelligent birds are popular because they can copy human talking. Because of this, some people trap birds and take them to other countries to sell. This is not usually allowed these days. Most pet birds are specially bred and are sold in pet shops.\n\nPeople can catch some bird diseases, for example: psittacosis, salmonellosis, campylobacteriosis, Newcastle\'s disease, mycobacteriosis, influenza, giardiasis and cryptosporiadiosis. In 2005, there was an epidemic of bird influenza spreading through some parts of the world, often called avian flu.\n\nSome people have birdboxes in their gardens to give birds a place to nest and bird tables where birds can get food and water in very cold or very dry weather.


key: _node_content
string_value:
{"id_": "5c08e931-46d1-4cb5-9488-b0fc6c9a4083", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "25f69d84-b095-4605-b3ea-cbe6e0ed19db", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5eb013f4980d9e6d60bd638c0903cfcd671843db6b5ac7d48de0a7ce6cfaedbb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "aa9ebab8-ba58-43af-9224-e53349f96fdb", "node_type": "1", "metadata": {}, "hash": "7eef2531684a8337cf9f73855d47ed69703a45df1f70cd6f8b8972eb249a0f49", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 43583, "end_char_idx": 46368, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "aa9ebab8-ba58-43af-9224-e53349f96fdb", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5c08e931-46d1-4cb5-9488-b0fc6c9a4083", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "1ecd961a17743d4c5d72e457d6005026df6b52af659feabfa45864d1d5257c8f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "16f8bb3b-f326-4b86-8325-d065c63fa719", "node_type": "1", "metadata": {}, "hash": "88ef2f56aef8083645d1f0aed4108001b7a76d7d9b71c5d55d8164f49a1226cb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 44951, "end_char_idx": 47840, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "16f8bb3b-f326-4b86-8325-d065c63fa719", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "aa9ebab8-ba58-43af-9224-e53349f96fdb", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "39e89bc752c3deb6a3c8e7c0776704ba1b1d81c33428788c274b27df59d45ae4", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1f6fe46c-d438-449d-a473-865f72f419fd", "node_type": "1", "metadata": {}, "hash": "f7f3db1c491f0fd69fd854ceea4636f5e6a2687aa35fb95815afd54121f291d1", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 46369, "end_char_idx": 49516, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "1f6fe46c-d438-449d-a473-865f72f419fd", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "16f8bb3b-f326-4b86-8325-d065c63fa719", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c045bc1935bdcff15150ea0cb6c8670e10fb1e676699472814f83115f641f1b2", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "da4f0ba7-2a41-4303-81ce-36b55a8f15dc", "node_type": "1", "metadata": {}, "hash": "7b7d12f2c4520527f126703d88d2b47307d62aac6e34849a0628d93d9363b25d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 47841, "end_char_idx": 51024, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "da4f0ba7-2a41-4303-81ce-36b55a8f15dc", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1f6fe46c-d438-449d-a473-865f72f419fd", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "8a40575a5a88b3708a731b17a2fe64b0d74c4b8376a39f3d613adbce99759a2e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "19b15847-6915-4dee-a96c-49b77756d5a7", "node_type": "1", "metadata": {}, "hash": "086997ea4becde911803d8101f6f621241fe4f3c319d814cda3f75b20788252d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 49517, "end_char_idx": 52810, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "19b15847-6915-4dee-a96c-49b77756d5a7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "da4f0ba7-2a41-4303-81ce-36b55a8f15dc", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c00dee90123146386254d4c159b6eb46e4236bf8441a75426bc562af66a916d4", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "84f9fb48-4c0b-4910-875a-575380f14976", "node_type": "1", "metadata": {}, "hash": "449ffdd594592098e7c21067d0336a2c34a54e16017c576142b773a7ef3fad1c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 51232, "end_char_idx": 54540, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "84f9fb48-4c0b-4910-875a-575380f14976", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "19b15847-6915-4dee-a96c-49b77756d5a7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "1add1ce84f06902f6238e007b79670a69179c57033b46c06a1fef3a5af2dda73", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "92dc6414-9434-48b0-a5db-44fd3cf2c4f7", "node_type": "1", "metadata": {}, "hash": "4da822e4bfa0795c3a729acf40724430c60bf236f6cf84bf332970b0d555d952", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 52770, "end_char_idx": 56112, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "92dc6414-9434-48b0-a5db-44fd3cf2c4f7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "84f9fb48-4c0b-4910-875a-575380f14976", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "1ef281ba9fe29ab379b5984c74601e54611e0bd0c25a3d877f918475c4de22e5", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "fd637539-9a8b-45e3-aa9c-f42325d46973", "node_type": "1", "metadata": {}, "hash": "e5b8b030f3de8ca1a7180c51c525cc314fb8b54b28ea6256e70ea9e57300b709", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 54308, "end_char_idx": 57769, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Because of these two things, people did not have to migrate as much any more. Villages could grow to much larger sizes than before. Over time, villages fought and spread their control over larger areas and some became civilisations. During this time, humankind also developed further intellectually, militarily and spiritually.\n\nWhen humans started to grow crops and domesticate certain animals such as dogs, goats, sheep, and cattle;  their societies changed. Because people now grew crops and raised livestock, they started to stay in the same place and build permanent settlements. In most places, this happened between 10,000 and 12,000 years ago. Their diet also changed. People ate more cereals and vegetables. They started to keep extra foods and seeds for later. In some years there were surpluses (extras) that could be traded for other goods.\n\nThese changes happened independently in many parts of the world. They did not happen in the same order though. For example, the earliest farming societies in the Near East did not use pottery. No one is sure if Britain had agriculture, or if permanent villages existed there at all. Early Japanese societies used pottery before developing agriculture.\n\nVere Gordon Childe gave the name Neolithic Revolution to this process in the 1920s. He thought that it was as important as the Industrial Revolution (which happened in the 18th and 19th century).\n\nAncient history – the early civilizations \n\nAncient history was the time from the development of writing to the fall of the Roman Empire. The fall of the Roman Empire caused chaos in Europe, leading to the Middle Ages (also called the Dark Ages or the Age of Faith).\n\nThe first civilizations were built along major river systems. These civilizations are called river valley civilizations. River valley civilizations were the most powerful civilizations in this time period because water was needed to have an agricultural society.\n\nThese civilizations were similar in that:\n They developed along river systems\n They had polytheistic religions\n They used writing systems\n\nMiddle East and North Africa\n\nSumer \n\nSumer was the world\'s first known ancient civilization. The Sumerians took over the fertile crescent region of Mesopotamia around 3300 BCE. They grew crops on the Tigris and Euphrates rivers. By 3000 BCE, many cities had been built in parts of Sumerian Mesopotamia. They formed independently and each had their own government. They were called city-states and often fought with each other.\n\nA surplus in food led to a Division of labour. This means that some people were able to stop growing crops and do other jobs, since enough crops were already grown. This brought a split in society. Today, such a split is called social pyramid. In a social pyramid, people are grouped into social classes based on their wealth and power. In Sumer, the king, priests, and government officials were at the top of the social pyramid. Below them were the artisans, merchants, farmers, and fishers. At the bottom of the pyramid were slaves. Slaves were often prisoners of war, criminals, or people working to pay off debt.\n\nThe Sumerians created the world\'s first system of writing; it was called cuneiform. The oldest versions of one of the world\'s first literary works, the Epic of Gilgamesh, go back to this time. In Sumer, only the sons of the rich and powerful learned how to read and write. They went to a school called edubba.


key: _node_content
string_value:
{"id_": "fd637539-9a8b-45e3-aa9c-f42325d46973", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "92dc6414-9434-48b0-a5db-44fd3cf2c4f7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "202f9792ef0bb91c7e76a4a3f5be93de9fda3a3d62aba9b2729346886f8ab29c", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "3927b07f-d141-42d0-8082-d7a9c3a5a445", "node_type": "1", "metadata": {}, "hash": "7b7b181423136f07463092e4c11755d5a32feb18e996da67989e763c45f483f4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 56054, "end_char_idx": 58976, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "3927b07f-d141-42d0-8082-d7a9c3a5a445", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "fd637539-9a8b-45e3-aa9c-f42325d46973", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "34dc765367e5222a384c9a5dd0319465991469e527a322c4fd5876435ffba690", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "55df1698-f4e1-4ee3-b600-3b1f60d0dcaf", "node_type": "1", "metadata": {}, "hash": "203db568ec3cd211e083ea25970d8f4710b2b4b59e764859791fc6a6d6b8df44", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 57373, "end_char_idx": 60383, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "55df1698-f4e1-4ee3-b600-3b1f60d0dcaf", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "3927b07f-d141-42d0-8082-d7a9c3a5a445", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e913040b992c6852aa034178a2121374bb0c04e571faa30b736f7ac39e6f0165", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9435bf6b-39bc-4c30-a03c-985aadd5e4bb", "node_type": "1", "metadata": {}, "hash": "77bcd75f1dd2d870d1567361cd9e118adcf1604cc13f17a8ccc90df963828945", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 58804, "end_char_idx": 62063, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "9435bf6b-39bc-4c30-a03c-985aadd5e4bb", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "55df1698-f4e1-4ee3-b600-3b1f60d0dcaf", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "62de2b229d039ac9f8d76e8f4753f0332f897a1038374389c7c27de0c2a26aca", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b8d0afb0-5c8c-411c-9d17-027908a7d5c1", "node_type": "1", "metadata": {}, "hash": "a5120d8439932e3dc30978e9bb332c60051fe68cf27619314b3bd427f674dba7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 60340, "end_char_idx": 63728, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "b8d0afb0-5c8c-411c-9d17-027908a7d5c1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9435bf6b-39bc-4c30-a03c-985aadd5e4bb", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9e9fc94102f80251052aa35a96f824665c9a342cf48e630334a94c50048fec49", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d760e0f7-9970-47ff-a415-06cfa0893aea", "node_type": "1", "metadata": {}, "hash": "1da0a2c276a8cc4c284409d5a4740f8624ec72bc25c78b0a924010f3797842a2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 61957, "end_char_idx": 65370, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "d760e0f7-9970-47ff-a415-06cfa0893aea", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b8d0afb0-5c8c-411c-9d17-027908a7d5c1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e9afe2c00b2eedcf5c215b8ee613fc457584e69dddc60c29151909faea0897b7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "23120e44-747b-49bc-9fbe-4e64b18210c4", "node_type": "1", "metadata": {}, "hash": "6cf62dcbcd3b3c7b7b4197ae21ca3639ab9ba4a4650d6e69d77972496d7c5cdf", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 63677, "end_char_idx": 66794, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "23120e44-747b-49bc-9fbe-4e64b18210c4", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d760e0f7-9970-47ff-a415-06cfa0893aea", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c10914675456928a4cc3889f598a4bf87a7b19c4bf2ad99cdc3ff277c838179c", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "dceb5972-08aa-4396-b8fb-0966e5269931", "node_type": "1", "metadata": {}, "hash": "b7d6130d74fe88ce4d80d03d0ca5e439f027ab68217939884692463879f023eb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 65250, "end_char_idx": 68322, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "dceb5972-08aa-4396-b8fb-0966e5269931", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "23120e44-747b-49bc-9fbe-4e64b18210c4", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "19ad2236bdfffbf2b7102f312d9ae21c955c4aec2252f6c0452c7b6fcfb30c0d", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "10555f85-3814-4794-b2dc-91d525fcdcdb", "node_type": "1", "metadata": {}, "hash": "b7829365d93209870338b1933b69c0969d49a0da5a29781cf213563facd1ac88", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 66586, "end_char_idx": 69875, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "10555f85-3814-4794-b2dc-91d525fcdcdb", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "dceb5972-08aa-4396-b8fb-0966e5269931", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "37b7f10a83a3d8bbe1401685fd5ac2f603a92ab26406019eb451ac0cd1b65c32", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "69c4a713-341c-4848-9f6f-40dab85e5b82", "node_type": "1", "metadata": {}, "hash": "0b49ff03ca10850e21cb67344574d03727303dad284c459bc73f108d14d0c376", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 68151, "end_char_idx": 71339, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "69c4a713-341c-4848-9f6f-40dab85e5b82", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "10555f85-3814-4794-b2dc-91d525fcdcdb", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "513f1daa453a5cc2ba59bb7e9bac0ac03ef018faf931ca414456d9ce174af5f5", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "23800fcd-5c8d-4d5c-b1d2-bc5175369fa7", "node_type": "1", "metadata": {}, "hash": "a7fb9254fb8ee796fbe392b4d9ee2380a8d35e7db310e1194e4ac6801d51c7f6", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 69718, "end_char_idx": 72931, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
But with the rise of Islam, many changes happened during the Islamic Golden Age. The Greek and Roman traditions were kept and further development took place. The Chinese civilization had a Golden Age during the Tang period, when their capital was the biggest in the world. During the Renaissance, Europe developed and made great advancements in many areas as well.\n\nAsia\n\nMiddle East – Islamic rise, Byzantine decline \n\nIn Arabia, Muhammad founded Islam in 632.  His followers rapidly conquered territories in Syria and Egypt.  They soon were a danger to the Byzantine Empire.  In the 8th and 9th centuries, the Byzantine Empire stopped Islamic expansion and reconquered some lost territories.  In 1000 A.D. the eastern Empire was at its height: Basileios II reconquered Bulgaria and Armenia.  Culture and trade flourished.  In 1071 the Battle of Manzikert led the empire into a dramatic decline.  For the Byzantine Empire this meant centuries of civil wars and Turkic invasions.  The Muslim caliphate had an Golden Age under the Abbasids.\n\nTheir power forced Emperor Alexius I Comnenus of the Byzantine Empire to send a call for help to the West in 1095.  The West sent the Crusades.  These eventually led to the Sack of Constantinople in the Fourth Crusade in 1204.  Because of this, what was left of the Empire broke into successor states.  The winner of these disputes was that of Nicaea.  After Constantinople was again conquered by imperial forces, the empire was little more than a Greek state on the Aegean coast.  The Eastern Empire came to an end when Mehmed II conquered Constantinople on May 29, 1453.  The Ottoman Empire took its place and from 1400 to 1600 was the most powerful empire in the Middle East and ruled at the southern and eastern coast of the Mediterranean Sea.\n\nChina \nThe Tang Dynasty (618–907), with its capital at Chang\'an (today Xi\'an), was the biggest city in the world at the time and is considered by historians as a high point in Chinese civilization as well as a golden age of cosmopolitan culture. The Ming Dynasty ruled from 1368 to 1644. The Ming built a vast army and navy.\n\nIndia \nFrom around the 6th–7th century. In South India, Chola kings ruled Tamil Nadu, and Chera kings ruled Kerala. They had trading relationships with the Roman Empire to the west and Southeast Asia to the east. In north India, Rajputs ruled in many kingdoms.\n\nIn 1336, two brothers named Harihara I and Bukka founded the Vijayanagara Empire in an area which is now in the Karnataka state of India. The most famous king of this empire was Krishnadevaraya. In 1565, rulers of this empire were defeated in a battle. But the empire continued for about the next one hundred years.\nNorthern India was ruled by Islamic sultans.\n\nJapan \nThe Heian Period in Japan is famous for its art, poetry and literature. The writing system, Kana, was developed. It was followed by the feudal period (1185–1853) during which samurai and daimyos were the leading figures and the shogun the real monarch whereas the tennō had only a role as religious head. Between the years 1272 and 1281 the Mongols tried to invade but were driven out by the Japanese.\nIn 1542, a Portuguese ship reached Japan.


key: _node_content
string_value:
{"id_": "23800fcd-5c8d-4d5c-b1d2-bc5175369fa7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "69c4a713-341c-4848-9f6f-40dab85e5b82", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5152f0e13aacb243e88de751d801341bcc7f59e807badb2985802f207e6c0a54", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "aa5bdeb6-8c08-44db-958f-d93e61de85a3", "node_type": "1", "metadata": {}, "hash": "94258a44f946d9047f4b57124cc8add1c404ca5b34577ded1da82b1406b7db08", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 71341, "end_char_idx": 74450, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The Ottoman Empire took its place and from 1400 to 1600 was the most powerful empire in the Middle East and ruled at the southern and eastern coast of the Mediterranean Sea.\n\nChina \nThe Tang Dynasty (618–907), with its capital at Chang\'an (today Xi\'an), was the biggest city in the world at the time and is considered by historians as a high point in Chinese civilization as well as a golden age of cosmopolitan culture. The Ming Dynasty ruled from 1368 to 1644. The Ming built a vast army and navy.\n\nIndia \nFrom around the 6th–7th century. In South India, Chola kings ruled Tamil Nadu, and Chera kings ruled Kerala. They had trading relationships with the Roman Empire to the west and Southeast Asia to the east. In north India, Rajputs ruled in many kingdoms.\n\nIn 1336, two brothers named Harihara I and Bukka founded the Vijayanagara Empire in an area which is now in the Karnataka state of India. The most famous king of this empire was Krishnadevaraya. In 1565, rulers of this empire were defeated in a battle. But the empire continued for about the next one hundred years.\nNorthern India was ruled by Islamic sultans.\n\nJapan \nThe Heian Period in Japan is famous for its art, poetry and literature. The writing system, Kana, was developed. It was followed by the feudal period (1185–1853) during which samurai and daimyos were the leading figures and the shogun the real monarch whereas the tennō had only a role as religious head. Between the years 1272 and 1281 the Mongols tried to invade but were driven out by the Japanese.\nIn 1542, a Portuguese ship reached Japan. Japanese learned about guns and firearms from them.\n\nMongols \nGenghis Khan in 1209 brought together the Mongol tribes and founded the Mongol Empire, one of the largest land empires in history. Later Kublai Khan would go on to expand the empire and found the Mongol-ruled Yuan Dynasty of China. The empire later broke into several empires, all of which were later destroyed.\n\nEuropean Middle Ages \n\nThe Middle Ages was the time from the fall of the Roman empire until the middle of the 15th century. From 500 to about 800 there was some decline compared with the Roman civilization. European villages were often destroyed and looted by barbarians such as the Vikings. During the High Middle Ages magnificent castles and large churches called cathedrals were built and important works of literature were written. In the later Middle Ages, there was a plague called the Black Death. The Black Death killed one-third to one-half of Europe\'s population.\n\nA system called feudalism was a very important part of the Middle Ages. In this system, the king was at the top of the social pyramid. The king gave land to the lord in exchange for loyalty. The lords were the next in the pyramid. The lords gave land (called a fief) to knights in exchange for loyalty and protection. The knights came next in the pyramid. Peasants were not part of the feudal system because they did not give or receive land. They worked on a lord\'s manor in exchange for protection.\n\nThe Crusades were also fought during the Middle Ages.


key: _node_content
string_value:
{"id_": "aa5bdeb6-8c08-44db-958f-d93e61de85a3", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "23800fcd-5c8d-4d5c-b1d2-bc5175369fa7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "ae96b065146dbc15a2c55650e4481702e42d351572af37632874832e3538b69c", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "8508c2f8-d66f-42f9-82db-403a408aa9f3", "node_type": "1", "metadata": {}, "hash": "81202b6f0a80376d3cc4203f71c38c3817f6de409e7efa72bbe0303522441ef9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 72792, "end_char_idx": 76198, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "8508c2f8-d66f-42f9-82db-403a408aa9f3", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "aa5bdeb6-8c08-44db-958f-d93e61de85a3", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "ec920df248dfa8083ab0e67d608b1b48b6defe9a7236491ed6042d402686ea3b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "94f4c305-3a50-4e3e-be6d-48b92f425cae", "node_type": "1", "metadata": {}, "hash": "77b0ffc91a3532f8c8c026878e93af591ec9e7e441770c11471ffa829061e457", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 74335, "end_char_idx": 77765, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "94f4c305-3a50-4e3e-be6d-48b92f425cae", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "8508c2f8-d66f-42f9-82db-403a408aa9f3", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d40d4cb78ea257815564711130de283bc8d95e34853668854d08a51c3c403afa", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "44a2ddb3-b582-41d5-a74a-2dc5f08ad267", "node_type": "1", "metadata": {}, "hash": "db035958254acf8d7e9e6422d64a075e7c0ac8769f40b9e12bf5e1ae61da68c4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 76095, "end_char_idx": 79228, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "44a2ddb3-b582-41d5-a74a-2dc5f08ad267", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "94f4c305-3a50-4e3e-be6d-48b92f425cae", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "4e56377e36f6e3225fe3fce1338b517cf07bdb352fce31ace81e9a9057c645d8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "848ef365-c0ac-452c-9799-bfd0328f200d", "node_type": "1", "metadata": {}, "hash": "5f96904e86ff5e4ac5e803ac3b5c4ea01ec4a639eb7711a28e34d3df2b3ed319", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 77690, "end_char_idx": 80612, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "848ef365-c0ac-452c-9799-bfd0328f200d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "44a2ddb3-b582-41d5-a74a-2dc5f08ad267", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "572211a4a682330dec648cc59b91564ac661ed750825cf760ac56b5c574ced90", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "513e8d77-f5db-4d90-9034-2abf58684b51", "node_type": "1", "metadata": {}, "hash": "8ef629c6e0f66041ac2a0d88402bfa18c366885cf2e71ffa1523c55d5884c274", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 79020, "end_char_idx": 82133, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
They spoke the Nahuatl language. Their capital was Tenochtitlan. It was built on islands in a lake. Tenochtitlan was one of the greatest cities of the world in that time.\n\nThe Aztecs believed in polytheism. Quetzalcoatl (feathered snake), Huitzilopochtli (hummingbird of the south) and Tezcatlipoca (smoking mirror) were the most important Gods. Sometimes the Aztecs killed humans to please their gods. Between 1519 and 1521 the Spanish leader Hernán Cortés defeated the Aztecs and took their empire. Some Aztecs did not want to fight against the soldiers of Cortés, because they thought they were Gods.\n\nToday many Mexicans have Aztec and other Native American forefathers. People still use Aztec symbols in Mexico. On the Mexican flag there is a picture of an eagle on a cactus with a snake in its mouth. This was an Aztec symbol. Also the name Mexico is an Aztec word.\n\nThe Aztecs ate a lot of plants and vegetables that could be grown easily in the Mexico area. The main food that they ate was corn, which they called maize. Another food that they ate was squash.\n\nAztecs also had a lot of harsh punishments for certain crimes. For the following crimes the punishment was death: adultery, wearing cotton clothes (cotton clothes were only for the nobles), cutting down a living tree, moving a field boundary making your land bigger, making someone else\'s smaller, major theft and treason.\n\nThe Incas \n\nThe Incas were a civilized empire in western South America. The Incas are called a "pre-Columbian" empire. This means that their country was here before Christopher Columbus. They ruled parts of South America around what is now Peru for a little over 100 years, until the Spanish invasion in the 16th century.\n\nThe Incan empire or , meaning four regions in Quechua, only lasted for about 100 years as the arrival of the Spaniards in 1532 conquered them. Their main language was Quechua, but as the Incas were basically made up of many different groups there were probably many other different languages.\n\nTheir capital was in the city of Cusco, or Qosqo, in what is now southern Peru.\n\nManco Capac founded the first Inca state around 1200. It covered the area around Cusco. In the 1400s, Pachacuti began to absorb other people in the Andes. The expansion of the Inca Empire had started. The Inca Empire would become the biggest empire in the Americas before Columbus.\n\nIn 1532, the civil war ended. The brothers Huascar and Atahualpa, fought for who would succeed their father. During this time, the Spanish conquerors took possession of the Inca territory. They were led by Francisco Pizarro. In the following years the conquistadors managed to extend their power over the whole Andean region. They suppressed successive Inca rebellions until the establishment of the Viceroyalty of Perú in 1542 and the fall of the resistance of the last Incas of Vilcabamba in 1572. The Inca civilization ends at that time, but many cultural traditions remain in some ethnic groups as Quechuas and Aymara people.\n\nAfrica \n\nAncient Egypt and Carthage are well known civilizations of ancient Africa.


key: _node_content
string_value:
{"id_": "513e8d77-f5db-4d90-9034-2abf58684b51", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "848ef365-c0ac-452c-9799-bfd0328f200d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5d22e981b24b850bac154e5005f2f994ef8e9d7a141e06179597529bd2daa69f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a4622fe5-1010-466e-8d0f-63df1b7dca8c", "node_type": "1", "metadata": {}, "hash": "a3d1d8dd052f1df64ad53a1b467ff2c4b774cce2de7275d38129836e33a10df9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 80498, "end_char_idx": 83827, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "a4622fe5-1010-466e-8d0f-63df1b7dca8c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "513e8d77-f5db-4d90-9034-2abf58684b51", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "26c8dbcc0bc1af4a5d07977cc644ef6505107801cbac9f0eaea0212fc41c9f80", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b031caf5-2872-48b8-b6a5-fef6448c1ba9", "node_type": "1", "metadata": {}, "hash": "d9a31340c046a4cbfe34004064053c30d375d2f0a18577ac720aa5c639c70f60", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 82134, "end_char_idx": 85597, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "b031caf5-2872-48b8-b6a5-fef6448c1ba9", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a4622fe5-1010-466e-8d0f-63df1b7dca8c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b2fbfe53e0af333a454f5ac39184aa5c10c23fc30b109d140433caa0acd8af7f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ae0e7a7d-327c-4a89-83f8-18c2cecf581a", "node_type": "1", "metadata": {}, "hash": "f7b71ee2ad852ed8159de116fb6c042d6003ad415c8da9b2c5760b31680c4564", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 83827, "end_char_idx": 87123, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "ae0e7a7d-327c-4a89-83f8-18c2cecf581a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b031caf5-2872-48b8-b6a5-fef6448c1ba9", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "2a7c453cf312bf14004e3c260e0551078014b64b3ea838429af21eef9981b137", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "6fa95291-3229-4602-b331-7feb2f4e351a", "node_type": "1", "metadata": {}, "hash": "8966fd7a8c1fb0512a6a85e121b4cf1ad0cd3ba27a1dfe68b9185bf3817f47e7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 85369, "end_char_idx": 88573, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "6fa95291-3229-4602-b331-7feb2f4e351a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ae0e7a7d-327c-4a89-83f8-18c2cecf581a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "70d7bde0b555e13836636c319b6af09821e4381a21435d6ac14e6693ca15a71f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "61d6ddf8-adf6-4cb2-9262-a056b16fb9dc", "node_type": "1", "metadata": {}, "hash": "9b354c1cbb852e4b58b9d7ecfb0e2e49f3c9126a6a8213305c5ce35388bfb486", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 87124, "end_char_idx": 90351, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "61d6ddf8-adf6-4cb2-9262-a056b16fb9dc", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "6fa95291-3229-4602-b331-7feb2f4e351a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "6ff100e578986288c799f7b60d1887d26ac199fffcc85a7081bba8e74b85cc28", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "660202ea-9648-4717-a0ba-74c3e152708f", "node_type": "1", "metadata": {}, "hash": "ed7fd05c794f6314d4b7f36265218e04fdf300d4b92f755ce282ab397d3d1d1c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 88703, "end_char_idx": 91905, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "660202ea-9648-4717-a0ba-74c3e152708f", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "61d6ddf8-adf6-4cb2-9262-a056b16fb9dc", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d49ca3198b58ea5bbdd884e2323ae886dbdf7c9c2bb42efcc89dd8c225d52224", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "63aa1f1a-d248-4490-b804-86f672d54bd0", "node_type": "1", "metadata": {}, "hash": "ade7e1cb6877c44208fe30dc61468d69759a2581cccad98f3dc7fb99a92f012e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 90258, "end_char_idx": 93509, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "63aa1f1a-d248-4490-b804-86f672d54bd0", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "660202ea-9648-4717-a0ba-74c3e152708f", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e866c8d6d9252b59eb280d1fe27ff12b9162014d1d855540b9d1198845d4a762", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "3a6dec9a-6c98-488c-ab40-b3ff0b26037b", "node_type": "1", "metadata": {}, "hash": "e2e848c0dda054ae23d7d701ae2c71dec97193d98ffa3f9ad99058984e0be203", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 91906, "end_char_idx": 95512, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "3a6dec9a-6c98-488c-ab40-b3ff0b26037b", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "63aa1f1a-d248-4490-b804-86f672d54bd0", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "6b9ef9cdcd83ec2e769cc82355cfa09c7980ce094ff41abe944ddaef03f77a27", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "046f28ee-41f0-4f03-86c3-32dc231b6890", "node_type": "1", "metadata": {}, "hash": "51a4473c10e1703d9153f40cb37eb6a5f357bb869b19d2270515ba74cb6abc41", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 93510, "end_char_idx": 97037, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
When in 1988/89 Gorbachev made clear that he would not force the countries of the East block to stick to Communism the Berlin Wall was torn down in 1989 and the Soviet Union collapsed (1991). Then the United States was the only superpower left.\n\nAfter Mao Zedong\'s death China\'s communist party proved that economic reform was possible without political freedom and paved the way for enormous economic growth.\n\nAs the 20th century ended, the European Union began to rise and included former satellite states and even parts of the Soviet Union. States in Asia, Africa and South America tried to copy the European Union.\n\nThe twentieth century was a time of great progress in terms of technology. People began to live longer because of better medicine and medical technology. New communications and transportation technologies connected the world. But these advances also helped cause problems with the environment.\n\nThe last half of the century had smaller wars.  Improved information technology and globalization increased trade and cultural exchange.  Space exploration expanded through the solar system.  The structure of DNA was discovered.\n\nThe same period also raised questions about the end of human history because of global dangers: nuclear weapons, greenhouse effect and other problems in the environment.\n\n21st century \n\nAs the 20th century ended, globalization has continued. During this period, communications with mobile phones and the Internet have expanded, which has led to fundamental social changes in corporation, political, and individuals\' personal lives. Due to the population of growth and industrialization, worldwide resource competition is becoming increasingly highly, especially in India, China and Brazil. The increasing demand on the environmental degradation and global warming.\n\nA new Great Recession affected the world in the late 2000s and the early 2010s, and the COVID-19 pandemic spread in 2020, causing further economic and political disruption. Some scientists referred to this as a "Planetary Phase of Civilization".\n\nRelated pages \n History of Africa\n History of America\n History of Asia\n History of Australia\n History of Europe\n History of the Earth\n\nReferences\n\nFurther reading \n \nEnglish translation by Paul G. Bahn from the French edition La Grotte Chauvet\n \n \nTranslation of La Grotte Chauvet, l\'art des origins, Éditions du Seuil, 2001\n\nOther websites \n Universal Concise History of the World, 1832 Full text, free to read, American book on the history of the world with the intriguing perspective of 1832 America.\n WWW-VL: World History at European University Institute\n Five Epochs of Civilization A scheme of organization which divides world history into five epochs marked by changes in communication technology\n World history -Citizendium\n\n+\nFormer good articles', 'vector_id': 11672}}]}}


    /var/folders/vz/v2f6_x6s0kg51j2vbm5rlhww0000gn/T/ipykernel_27262/2105931364.py:1: DeprecationWarning: The 'body' parameter is deprecated and will be removed in a future version. Instead use individual parameters.
      print(client.search(index="wikipedia_vector_index", body={


## Encode a question with OpenAI embedding model

To perform semantic search, we need to encode queries with the same embedding model used to encode the documents at index time.
In this example, we need to use the `text-embedding-3-small` model.

You'll need your OpenAI [API key](https://platform.openai.com/account/api-keys) to generate the embeddings.


key: _node_content
string_value:
{"id_": "046f28ee-41f0-4f03-86c3-32dc231b6890", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "3a6dec9a-6c98-488c-ab40-b3ff0b26037b", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f5b8a2dc34c2be897c9be3ccef800ac15c9b3817b90a6ba8d682315f199e42d1", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "20f70211-e5c3-43bb-986b-c17c748e5946", "node_type": "1", "metadata": {}, "hash": "5e4769f35443ab876cfe424f636d215ca397620c45a238af1235c4b683127568", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 95513, "end_char_idx": 98604, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Some scientists referred to this as a "Planetary Phase of Civilization".\n\nRelated pages \n History of Africa\n History of America\n History of Asia\n History of Australia\n History of Europe\n History of the Earth\n\nReferences\n\nFurther reading \n \nEnglish translation by Paul G. Bahn from the French edition La Grotte Chauvet\n \n \nTranslation of La Grotte Chauvet, l\'art des origins, Éditions du Seuil, 2001\n\nOther websites \n Universal Concise History of the World, 1832 Full text, free to read, American book on the history of the world with the intriguing perspective of 1832 America.\n WWW-VL: World History at European University Institute\n Five Epochs of Civilization A scheme of organization which divides world history into five epochs marked by changes in communication technology\n World history -Citizendium\n\n+\nFormer good articles', 'vector_id': 11672}}]}}


    /var/folders/vz/v2f6_x6s0kg51j2vbm5rlhww0000gn/T/ipykernel_27262/2105931364.py:1: DeprecationWarning: The 'body' parameter is deprecated and will be removed in a future version. Instead use individual parameters.
      print(client.search(index="wikipedia_vector_index", body={


## Encode a question with OpenAI embedding model

To perform semantic search, we need to encode queries with the same embedding model used to encode the documents at index time.
In this example, we need to use the `text-embedding-3-small` model.

You'll need your OpenAI [API key](https://platform.openai.com/account/api-keys) to generate the embeddings.


```python
# Get OpenAI API key
OPENAI_API_KEY = getpass("Enter OpenAI API key")

# Set API key
openai.api_key = OPENAI_API_KEY

# Define model
EMBEDDING_MODEL = "text-embedding-3-small"

# Define question
question = 'Is the Atlantic the biggest ocean in the world?'

# Create embedding
question_embedding = openai.Embedding.create(input=question, model=EMBEDDING_MODEL)

```

## Run semantic search queries

Now we're ready to run queries against our Elasticsearch index using our encoded question. We'll be doing a k-nearest neighbors search, using the Elasticsearch [kNN query](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html) option.

First, we define a small function to pretty print the results.


```python
# Function to pretty print Elasticsearch results

def pretty_response(response):
    for hit in response['hits']['hits']:
        id = hit['_id']
        score = hit['_score']
        title = hit['_source']['title']
        text = hit['_source']['text']
        pretty_output = (f"\nID: {id}\nTitle: {title}\nSummary: {text}\nScore: {score}")
        print(pretty_output)
```

Now let's run our `kNN` query.


```python
response = client.search(
  index = "wikipedia_vector_index",
  knn={
      "field": "content_vector",
      "query_vector":  question_embedding["data"][0]["embedding"],
      "k": 10,
      "num_candidates": 100
    }
)
pretty_response(response)
```

    
    ID: 1936
    Title: Atlantic Ocean
    Summary: The Atlantic Ocean is the world's second largest ocean.  It covers a total area of about .


key: _node_content
string_value:
{"id_": "20f70211-e5c3-43bb-986b-c17c748e5946", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "046f28ee-41f0-4f03-86c3-32dc231b6890", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "43b0a7f3f32bb44044883d80a4d6bd30f06313df00cb68a4ddd51eb154cc1807", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "13dbdfee-c2e8-4a15-85ec-de794bf708cf", "node_type": "1", "metadata": {}, "hash": "2f2564641505934a9d467e0e0ee6d272890f7d36cec384ac92874da5e5c1f52d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 97040, "end_char_idx": 100309, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
# Get OpenAI API key
OPENAI_API_KEY = getpass("Enter OpenAI API key")

# Set API key
openai.api_key = OPENAI_API_KEY

# Define model
EMBEDDING_MODEL = "text-embedding-3-small"

# Define question
question = 'Is the Atlantic the biggest ocean in the world?'

# Create embedding
question_embedding = openai.Embedding.create(input=question, model=EMBEDDING_MODEL)

```

## Run semantic search queries

Now we're ready to run queries against our Elasticsearch index using our encoded question. We'll be doing a k-nearest neighbors search, using the Elasticsearch [kNN query](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html) option.

First, we define a small function to pretty print the results.


```python
# Function to pretty print Elasticsearch results

def pretty_response(response):
    for hit in response['hits']['hits']:
        id = hit['_id']
        score = hit['_score']
        title = hit['_source']['title']
        text = hit['_source']['text']
        pretty_output = (f"\nID: {id}\nTitle: {title}\nSummary: {text}\nScore: {score}")
        print(pretty_output)
```

Now let's run our `kNN` query.


```python
response = client.search(
  index = "wikipedia_vector_index",
  knn={
      "field": "content_vector",
      "query_vector":  question_embedding["data"][0]["embedding"],
      "k": 10,
      "num_candidates": 100
    }
)
pretty_response(response)
```

    
    ID: 1936
    Title: Atlantic Ocean
    Summary: The Atlantic Ocean is the world's second largest ocean.  It covers a total area of about . It covers about 20 percent of the Earth's surface. It is named after the god Atlas from Greek mythology.
    
    Geologic history 
    The Atlantic formed when the Americas moved west from Eurasia and Africa. This began sometime in the Cretaceous period, roughly 135 million years ago. It was part of the break-up of the supercontinent Pangaea.
    
    The east coast of South America is shaped somewhat like the west coast of Africa, and this gave a clue that continents moved over long periods of time (continental drift). The Atlantic Ocean is still growing now, because of sea-floor spreading from the mid-Atlantic Ridge, while the Pacific Ocean is said to be shrinking because the sea floor is folding under itself or subducting into the mantle.
    
    Geography
    The Atlantic Ocean is bounded on the west by North and South America. It connects to the Arctic Ocean through the Denmark Strait, Greenland Sea, Norwegian Sea and Barents Sea. It connects with the Mediterranean Sea through the Strait of Gibraltar.
    
    In the southeast, the Atlantic merges into the Indian Ocean. The 20° East meridian defines its border.
    
    In the southwest, the Drake Passage connects it to the Pacific Ocean. The Panama Canal links the Atlantic and Pacific.
    
    The Atlantic Ocean is second in size to the Pacific. It occupies an area of about . The volume of the Atlantic, along with its adjacent seas (the seas next to it), is 354,700,000 cubic kilometres.
    
    The average depth of the Atlantic, along with its adjacent seas, is . The greatest depth is Milwaukee Deep near Puerto Rico, where the Ocean is  deep.
    
    Gulf Stream 
    The Atlantic Ocean has important ocean currents.


key: _node_content
string_value:
{"id_": "13dbdfee-c2e8-4a15-85ec-de794bf708cf", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "20f70211-e5c3-43bb-986b-c17c748e5946", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "ba48a28a6b65a0cf9fa43fb941c99f221726ed6037ce0117659a623b9f8e7b96", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "94d270e6-630f-4431-83bd-44b0114e58a6", "node_type": "1", "metadata": {}, "hash": "904d73e910b86bc53696b281c82c723e3c1f2118e316319897539ce73cfe3ee8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 98571, "end_char_idx": 102074, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It covers a total area of about . It covers about 20 percent of the Earth's surface. It is named after the god Atlas from Greek mythology.
    
    Geologic history 
    The Atlantic formed when the Americas moved west from Eurasia and Africa. This began sometime in the Cretaceous period, roughly 135 million years ago. It was part of the break-up of the supercontinent Pangaea.
    
    The east coast of South America is shaped somewhat like the west coast of Africa, and this gave a clue that continents moved over long periods of time (continental drift). The Atlantic Ocean is still growing now, because of sea-floor spreading from the mid-Atlantic Ridge, while the Pacific Ocean is said to be shrinking because the sea floor is folding under itself or subducting into the mantle.
    
    Geography
    The Atlantic Ocean is bounded on the west by North and South America. It connects to the Arctic Ocean through the Denmark Strait, Greenland Sea, Norwegian Sea and Barents Sea. It connects with the Mediterranean Sea through the Strait of Gibraltar.
    
    In the southeast, the Atlantic merges into the Indian Ocean. The 20° East meridian defines its border.
    
    In the southwest, the Drake Passage connects it to the Pacific Ocean. The Panama Canal links the Atlantic and Pacific.
    
    The Atlantic Ocean is second in size to the Pacific. It occupies an area of about . The volume of the Atlantic, along with its adjacent seas (the seas next to it), is 354,700,000 cubic kilometres.
    
    The average depth of the Atlantic, along with its adjacent seas, is . The greatest depth is Milwaukee Deep near Puerto Rico, where the Ocean is  deep.
    
    Gulf Stream 
    The Atlantic Ocean has important ocean currents.  One of these, called the Gulf Stream, flows across the North Atlantic.  Water gets heated by the sun in the Caribbean Sea and then moves northwest toward the North Pole.  This makes France, the British Isles, Iceland, and Norway in Europe much warmer in winter than Newfoundland and Nova Scotia in Canada. Without the Gulf Stream, the climates of northeast Canada and northwest Europe might be the same, because these places are about the same distance from the North Pole.
    
    There are currents in the South Atlantic too, but the shape of this sea means that it has less effect on South Africa.
    
    Geology 
    The main feature of the Atlantic Ocean's seabed is a large underwater mountain chain called the Mid-Atlantic Ridge.  It runs from north to south under the Ocean. This is at the boundary of four tectonic plates: Eurasian, North American, South American and African. The ridge extends from Iceland in the north to about 58° south.
    
    The salinity of the surface waters of the open ocean ranges from 3337 parts per thousand and varies with latitude and season.
    
    References
    
    Other websites 
    LA Times special Altered Oceans 
    Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
    National Oceanic and Atmospheric Administration 
    NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
    
    www.cartage.org.lb 
    www.mnsu.edu
    Score: 0.93642545
    
    ID: 1975
    Title: Pacific Ocean
    Summary: The Pacific Ocean is the body of water between Asia and Australia in the west, the Americas in the east, the Southern Ocean to the south, and the Arctic Ocean to the north. It is the largest named ocean and it covers one-third of the surface of the entire world.


key: _node_content
string_value:
{"id_": "94d270e6-630f-4431-83bd-44b0114e58a6", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "13dbdfee-c2e8-4a15-85ec-de794bf708cf", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "2abe98d76dd5bd367dc1ca0277ac61093b982f1af54674973d650cd1f5aac37b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "45d48731-7d77-48f3-8bed-b69d226d59d7", "node_type": "1", "metadata": {}, "hash": "df951f996c309dacaa87c7a2b8a4608f00ab68bfb1348dcd98eab31931794925", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 100244, "end_char_idx": 103693, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Gulf Stream 
    The Atlantic Ocean has important ocean currents.  One of these, called the Gulf Stream, flows across the North Atlantic.  Water gets heated by the sun in the Caribbean Sea and then moves northwest toward the North Pole.  This makes France, the British Isles, Iceland, and Norway in Europe much warmer in winter than Newfoundland and Nova Scotia in Canada. Without the Gulf Stream, the climates of northeast Canada and northwest Europe might be the same, because these places are about the same distance from the North Pole.
    
    There are currents in the South Atlantic too, but the shape of this sea means that it has less effect on South Africa.
    
    Geology 
    The main feature of the Atlantic Ocean's seabed is a large underwater mountain chain called the Mid-Atlantic Ridge.  It runs from north to south under the Ocean. This is at the boundary of four tectonic plates: Eurasian, North American, South American and African. The ridge extends from Iceland in the north to about 58° south.
    
    The salinity of the surface waters of the open ocean ranges from 3337 parts per thousand and varies with latitude and season.
    
    References
    
    Other websites 
    LA Times special Altered Oceans 
    Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
    National Oceanic and Atmospheric Administration 
    NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
    
    www.cartage.org.lb 
    www.mnsu.edu
    Score: 0.93642545
    
    ID: 1975
    Title: Pacific Ocean
    Summary: The Pacific Ocean is the body of water between Asia and Australia in the west, the Americas in the east, the Southern Ocean to the south, and the Arctic Ocean to the north. It is the largest named ocean and it covers one-third of the surface of the entire world. It joins the Atlantic Ocean at a line drawn  south from Cape Horn, Chile/Argentina to Antarctica, and joins the Indian Ocean at a line drawn  south from Tasmania, Australia to Antarctica.
    
    As the Atlantic slowly gets wider, the Pacific is slowly shrinking. It does this by folding the sea floor in towards the centre of the Earth - this is called subduction.  This bumping and grinding is hard so there are many earthquakes and volcanoes when the pressure builds up and is quickly released as large explosions of hot rocks and dust. When an earthquake happens under the sea, the quick jerk causes a tsunami. This is why tsunamis are more common around the edge of the Pacific than anywhere else. Many of the Earth's volcanoes are either islands in the Pacific, or are on continents within a few hundred kilometers of the ocean's edge. Plate tectonics are another reason which makes Pacific Ocean smaller.
    
    Other websites 
    
     EPIC Pacific Ocean Data Collection Viewable on-line collection of observational data
     NOAA In-situ Ocean Data Viewer  plot and download ocean observations
     NOAA PMEL Argo profiling floats Realtime Pacific Ocean data
     NOAA TAO El Niño data Realtime Pacific Ocean El Niño buoy data
     NOAA Ocean Surface Current Analyses – Realtime (OSCAR) Near-realtime Pacific Ocean Surface Currents derived from satellite altimeter and scatterometer data
    Score: 0.9178456
    
    ID: 11124
    Title: List of seas
    Summary: The sea is the interconnected system of all the Earth's oceanic waters, including the Atlantic, Pacific, Indian, Southern and Arctic Oceans.


key: _node_content
string_value:
{"id_": "45d48731-7d77-48f3-8bed-b69d226d59d7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "94d270e6-630f-4431-83bd-44b0114e58a6", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5ee2428f5710c70f874a9ac901836e418efba102e7ed6e558e79fe7c77eb84c5", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "11d03ec7-3a0f-4523-b3de-ef16e7bd5102", "node_type": "1", "metadata": {}, "hash": "2cf315ee0a733b3751eaa0364be710d9ff210de1e03066d15bdd37df8f320502", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 101985, "end_char_idx": 104411, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It is the largest named ocean and it covers one-third of the surface of the entire world. It joins the Atlantic Ocean at a line drawn  south from Cape Horn, Chile/Argentina to Antarctica, and joins the Indian Ocean at a line drawn  south from Tasmania, Australia to Antarctica.
    
    As the Atlantic slowly gets wider, the Pacific is slowly shrinking. It does this by folding the sea floor in towards the centre of the Earth - this is called subduction.  This bumping and grinding is hard so there are many earthquakes and volcanoes when the pressure builds up and is quickly released as large explosions of hot rocks and dust. When an earthquake happens under the sea, the quick jerk causes a tsunami. This is why tsunamis are more common around the edge of the Pacific than anywhere else. Many of the Earth's volcanoes are either islands in the Pacific, or are on continents within a few hundred kilometers of the ocean's edge. Plate tectonics are another reason which makes Pacific Ocean smaller.
    
    Other websites 
    
     EPIC Pacific Ocean Data Collection Viewable on-line collection of observational data
     NOAA In-situ Ocean Data Viewer  plot and download ocean observations
     NOAA PMEL Argo profiling floats Realtime Pacific Ocean data
     NOAA TAO El Niño data Realtime Pacific Ocean El Niño buoy data
     NOAA Ocean Surface Current Analyses – Realtime (OSCAR) Near-realtime Pacific Ocean Surface Currents derived from satellite altimeter and scatterometer data
    Score: 0.9178456
    
    ID: 11124
    Title: List of seas
    Summary: The sea is the interconnected system of all the Earth's oceanic waters, including the Atlantic, Pacific, Indian, Southern and Arctic Oceans. However, the word "sea" can also be used for many specific, much smaller bodies of seawater, such as the North Sea or the Red Sea.There are 78 seas in the world
    
    List of seas, by ocean
    
    Pacific Ocean 
     Bering Sea
     Gulf of Alaska
     Seck Sea (Gulf of California)
     Sea of Okhotsk
     Sea of Japan
     Seto Inland Sea
     East China Sea
     South China Sea
     Beibu Gulf
     Sulu Sea
     Celebes Sea
     Bohol Sea (Mindanao Sea)
     Philippine Sea
     Flores Sea
     Banda Sea
     Arafura Sea
     Tasman Sea
     Yellow Sea
     Bohai Sea
     Coral Sea
     Gulf of Carpentaria
    
    Atlantic Ocean 
     Hudson Bay
     James Bay
     Baffin Bay init fam
     Gulf of St.


key: _node_content
string_value:
{"id_": "11d03ec7-3a0f-4523-b3de-ef16e7bd5102", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "45d48731-7d77-48f3-8bed-b69d226d59d7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "19ffa9bdaea2a13dafdae3f8ecc5531db55d7e6d4545a563a5e854389d6e3cdd", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "cb41c633-e2af-44c9-b79c-61e5cacf7526", "node_type": "1", "metadata": {}, "hash": "1c689b87512a50af6cbd0f0c64411e34f3093c36a3ce18972c7eeeebd7325659", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 102779, "end_char_idx": 105708, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Many of the Earth's volcanoes are either islands in the Pacific, or are on continents within a few hundred kilometers of the ocean's edge. Plate tectonics are another reason which makes Pacific Ocean smaller.
    
    Other websites 
    
     EPIC Pacific Ocean Data Collection Viewable on-line collection of observational data
     NOAA In-situ Ocean Data Viewer  plot and download ocean observations
     NOAA PMEL Argo profiling floats Realtime Pacific Ocean data
     NOAA TAO El Niño data Realtime Pacific Ocean El Niño buoy data
     NOAA Ocean Surface Current Analyses – Realtime (OSCAR) Near-realtime Pacific Ocean Surface Currents derived from satellite altimeter and scatterometer data
    Score: 0.9178456
    
    ID: 11124
    Title: List of seas
    Summary: The sea is the interconnected system of all the Earth's oceanic waters, including the Atlantic, Pacific, Indian, Southern and Arctic Oceans. However, the word "sea" can also be used for many specific, much smaller bodies of seawater, such as the North Sea or the Red Sea.There are 78 seas in the world
    
    List of seas, by ocean
    
    Pacific Ocean 
     Bering Sea
     Gulf of Alaska
     Seck Sea (Gulf of California)
     Sea of Okhotsk
     Sea of Japan
     Seto Inland Sea
     East China Sea
     South China Sea
     Beibu Gulf
     Sulu Sea
     Celebes Sea
     Bohol Sea (Mindanao Sea)
     Philippine Sea
     Flores Sea
     Banda Sea
     Arafura Sea
     Tasman Sea
     Yellow Sea
     Bohai Sea
     Coral Sea
     Gulf of Carpentaria
    
    Atlantic Ocean 
     Hudson Bay
     James Bay
     Baffin Bay init fam
     Gulf of St. Lawrence
     Gulf of Guinea
     Caribbean Sea
     Gulf of Mexico
     Sargasso Sea
     North Sea
     Baltic Sea
     Gulf of Bothnia
     Irish Sea
     Celtic Sea
     English Channel
     Mediterranean Sea
     Adriatic Sea
     Aegean Sea
     Black Sea
     Sea of Azov
     Ionian Sea
     Ligurian Sea
     Mirtoon Sea
     Tyrrhenian Sea
     Gulf of Sidra
     Sea of Marmara
     Sea of Crete
    
    Indian Ocean 
     Red Sea
     Gulf of Aden
     Persian Gulf
     Gulf of Oman
     Arabian Sea
     Bay of Bengal
     Gulf of Thailand
     Java Sea
     Timor Sea
     Gulf of Kutch
     Gulf of Khambhat
    
    Arctic Ocean 
     Barents Sea
     Kara Sea
     Beaufort Sea
     Amundsen Gulf
     Greenland Sea
     Chukchi Sea
     Laptev Sea
     East Siberian Sea
    
    Southern Ocean 
     Amundsen Sea
     Weddell Sea
     Ross Sea
     Great Australian Bight
     Gulf St. Vincent
     Spencer Gulf
    
    Seas which have land around them (these are landlocked) 
     Aral Sea
     Caspian Sea
     Dead Sea
     Sea of Galilee (we call this a sea, but it is really a small freshwater lake)
     Salton Sea
    
    Seas which are not on Earth 
    Lunar maria are very big areas on the Moon. In the past, people thought they were water and called them "seas".


key: _node_content
string_value:
{"id_": "cb41c633-e2af-44c9-b79c-61e5cacf7526", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "11d03ec7-3a0f-4523-b3de-ef16e7bd5102", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "957ed00aaf096cab90370aae6744f302e9b7858ef538f233771c3c81e97702c8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2aba2028-0d0c-440a-a62c-2ede083c8dc5", "node_type": "1", "metadata": {}, "hash": "cc1af911824ec3ea9a16ea0bd4a0a478c8c6d707edbece09c47e033e19c6d29f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 104412, "end_char_idx": 107409, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Lawrence
     Gulf of Guinea
     Caribbean Sea
     Gulf of Mexico
     Sargasso Sea
     North Sea
     Baltic Sea
     Gulf of Bothnia
     Irish Sea
     Celtic Sea
     English Channel
     Mediterranean Sea
     Adriatic Sea
     Aegean Sea
     Black Sea
     Sea of Azov
     Ionian Sea
     Ligurian Sea
     Mirtoon Sea
     Tyrrhenian Sea
     Gulf of Sidra
     Sea of Marmara
     Sea of Crete
    
    Indian Ocean 
     Red Sea
     Gulf of Aden
     Persian Gulf
     Gulf of Oman
     Arabian Sea
     Bay of Bengal
     Gulf of Thailand
     Java Sea
     Timor Sea
     Gulf of Kutch
     Gulf of Khambhat
    
    Arctic Ocean 
     Barents Sea
     Kara Sea
     Beaufort Sea
     Amundsen Gulf
     Greenland Sea
     Chukchi Sea
     Laptev Sea
     East Siberian Sea
    
    Southern Ocean 
     Amundsen Sea
     Weddell Sea
     Ross Sea
     Great Australian Bight
     Gulf St. Vincent
     Spencer Gulf
    
    Seas which have land around them (these are landlocked) 
     Aral Sea
     Caspian Sea
     Dead Sea
     Sea of Galilee (we call this a sea, but it is really a small freshwater lake)
     Salton Sea
    
    Seas which are not on Earth 
    Lunar maria are very big areas on the Moon. In the past, people thought they were water and called them "seas". 
    
    Scientists think that there is liquid water under the ground on some moons, for example Europa.
    
    Scientists also think that there are liquid hydrocarbons on Titan. 
    
    Basic English 850 words
    
    Geography-related lists
    Score: 0.9160589
    
    ID: 2033
    Title: Southern Ocean
    Summary: The Southern Ocean is the ocean around Antarctica. It means the waters of the Atlantic, Pacific, and Indian Oceans around the continent of Antarctica. Since the 1770s geographers have discussed its limits. Nowadays, sixty degrees south latitude is often accepted. Some people call this ocean the Antarctic Ocean.
    
    The total area is 20,327,000 km², and the coastline length is 17,968 km.
    
    Other websites 
    
     Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
     The CIA World Factbook's  entry on the Southern Ocean
     The Fifth Ocean from Geography.About.com
     NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
     NOAA FAQ about the number of oceans 
    
     
    Geography of Antarctica
    Score: 0.90838027
    
    ID: 1978
    Title: Indian Ocean
    Summary: The Indian Ocean is the ocean surrounded by Asia to the north, Australia and the Pacific Ocean to the east, the Southern Ocean to the south, and Africa and the Atlantic Ocean to the west. It is named for the river Indus and Ancient India on its north shore. The Bay of Bengal, the Arabian Sea, the Persian Gulf and the Red Sea are all parts of this ocean.
    
    The deepest point in the Indian Ocean is in the Java Trench near the Sunda Islands in the east, 7500 m (25,344 feet) deep. The average depth is 3,890 m (12,762 ft).


key: _node_content
string_value:
{"id_": "2aba2028-0d0c-440a-a62c-2ede083c8dc5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "cb41c633-e2af-44c9-b79c-61e5cacf7526", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a5bbe994e3f09663d98f2036befcae2196c9fd078888c34e1182cf4b13258157", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5a5abc24-eecc-4bf8-8ea0-9abc2de8a74e", "node_type": "1", "metadata": {}, "hash": "f10a87c9fa8ce80cef04892885deff440e96601738b30e1c04c475dcc6280a3f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 105719, "end_char_idx": 108991, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Scientists think that there is liquid water under the ground on some moons, for example Europa.
    
    Scientists also think that there are liquid hydrocarbons on Titan. 
    
    Basic English 850 words
    
    Geography-related lists
    Score: 0.9160589
    
    ID: 2033
    Title: Southern Ocean
    Summary: The Southern Ocean is the ocean around Antarctica. It means the waters of the Atlantic, Pacific, and Indian Oceans around the continent of Antarctica. Since the 1770s geographers have discussed its limits. Nowadays, sixty degrees south latitude is often accepted. Some people call this ocean the Antarctic Ocean.
    
    The total area is 20,327,000 km², and the coastline length is 17,968 km.
    
    Other websites 
    
     Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
     The CIA World Factbook's  entry on the Southern Ocean
     The Fifth Ocean from Geography.About.com
     NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
     NOAA FAQ about the number of oceans 
    
     
    Geography of Antarctica
    Score: 0.90838027
    
    ID: 1978
    Title: Indian Ocean
    Summary: The Indian Ocean is the ocean surrounded by Asia to the north, Australia and the Pacific Ocean to the east, the Southern Ocean to the south, and Africa and the Atlantic Ocean to the west. It is named for the river Indus and Ancient India on its north shore. The Bay of Bengal, the Arabian Sea, the Persian Gulf and the Red Sea are all parts of this ocean.
    
    The deepest point in the Indian Ocean is in the Java Trench near the Sunda Islands in the east, 7500 m (25,344 feet) deep. The average depth is 3,890 m (12,762 ft). The Indian Ocean is the third largest ocean, 28,350,000 square miles in size. The majority is in the southern hemisphere.
    
    Other websites 
    
     Maps of the indian Ocean
     Océan Indien in easy French
     NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
     The Indian Ocean in World History: Educational Website Interactive resource from the Sultan Qaboos Cultural Center
     The Regional Tuna Tagging Project-Indian Ocean with details of the importance of Tuna in the Indian Ocean.. 
     Detailed maps of the Indian Ocean
     The Indian Ocean Trade: A Classroom Simulation 
     CIA - The World Factbook, Oceans: Indian Ocean
    Score: 0.90745246
    
    ID: 1980
    Title: Arctic Ocean
    Summary: The Arctic Ocean is the ocean around the North Pole. The most northern parts of Eurasia and North America are around the Arctic Ocean. Thick pack ice and snow cover almost all of this ocean in winter, and most of it in summer. An icebreaker or a nuclear-powered submarine can use the Northwest Passage through the Arctic Ocean to go between the Pacific and Atlantic oceans.
    
    The ocean's area is about 14.056 million km2, which is the smallest of the world's 5 oceans, and it has  of coastline. The central surface covered by ice about  thick. The biology there is quite special. Endangered species there include walruses, whales and polar bears. Year by year the Arctic Ocean is becoming less icy, as a result of global warming.
    
    The average depth of the Arctic Ocean is . The deepest point is in the Eurasian Basin, at .


key: _node_content
string_value:
{"id_": "5a5abc24-eecc-4bf8-8ea0-9abc2de8a74e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2aba2028-0d0c-440a-a62c-2ede083c8dc5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "4f5431bc445b34b80f0360f57de0a9e3900befcbdce22e6d7c4940b39e5ee285", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "fc306103-6dc2-4926-b2c9-f12047e15555", "node_type": "1", "metadata": {}, "hash": "58fd976add5fc61182d5854d3b12c4d9a6a6f6ba53691dc690b9eb874eee340b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 107368, "end_char_idx": 110465, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The average depth is 3,890 m (12,762 ft). The Indian Ocean is the third largest ocean, 28,350,000 square miles in size. The majority is in the southern hemisphere.
    
    Other websites 
    
     Maps of the indian Ocean
     Océan Indien in easy French
     NOAA In-situ Ocean Data Viewer  Plot and download ocean observations
     The Indian Ocean in World History: Educational Website Interactive resource from the Sultan Qaboos Cultural Center
     The Regional Tuna Tagging Project-Indian Ocean with details of the importance of Tuna in the Indian Ocean.. 
     Detailed maps of the Indian Ocean
     The Indian Ocean Trade: A Classroom Simulation 
     CIA - The World Factbook, Oceans: Indian Ocean
    Score: 0.90745246
    
    ID: 1980
    Title: Arctic Ocean
    Summary: The Arctic Ocean is the ocean around the North Pole. The most northern parts of Eurasia and North America are around the Arctic Ocean. Thick pack ice and snow cover almost all of this ocean in winter, and most of it in summer. An icebreaker or a nuclear-powered submarine can use the Northwest Passage through the Arctic Ocean to go between the Pacific and Atlantic oceans.
    
    The ocean's area is about 14.056 million km2, which is the smallest of the world's 5 oceans, and it has  of coastline. The central surface covered by ice about  thick. The biology there is quite special. Endangered species there include walruses, whales and polar bears. Year by year the Arctic Ocean is becoming less icy, as a result of global warming.
    
    The average depth of the Arctic Ocean is . The deepest point is in the Eurasian Basin, at .
    
    Geography 
    The Arctic Ocean covers an area of about 14,056,000 km2.  The coastline is 45,390 km (28,200 mi) long It is surrounded by Eurasia, North America, Greenland, and by several islands.
    
    It is generally taken to include Baffin Bay, Barents Sea, Beaufort Sea, Chukchi Sea, East Siberian Sea, Greenland Sea, Hudson Bay, Hudson Strait, Kara Sea, Laptev Sea, White Sea and other bodies of water. It is connected to the Pacific Ocean by the Bering Strait  and to the Atlantic Ocean through the Greenland Sea and Labrador Sea.
    
    Countries bordering the Arctic Ocean are: Russia, Norway, Iceland, Greenland, Canada and the United States.
    
    Climate 
    The Arctic Ocean is in a polar climate. Winters are characterized by the polar night, cold and stable weather conditions, and clear skies.
    
    The temperature of the surface of the Arctic Ocean is fairly constant, near the freezing point of seawater.  Arctic Ocean consists of saltwater but its salinity is less than other oceans. The temperature must reach −1.8 °C (28.8 °F) before freezing occurs.
    
    Ice covers most of the Arctic Ocean. It covers almost the whole ocean in late winter and the majority of the ocean in late summer. Much of the Arctic ice pack is covered in snow for about 10 months of the year. The maximum snow cover is in March or April — about 20 to 50 cm (7.9 to 19.7 in).
    
    The climate of the Arctic region has varied significantly in the past.


key: _node_content
string_value:
{"id_": "fc306103-6dc2-4926-b2c9-f12047e15555", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5a5abc24-eecc-4bf8-8ea0-9abc2de8a74e", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a88ee825730923afa7f62a96dd4a6fe6ab5b5bdd831e0e0be3e79b873e14587b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "3dd26252-ef11-4d31-a51b-6f0e58971706", "node_type": "1", "metadata": {}, "hash": "069eeb8643168ea547ff11f206b5f37821f6a2d827e8352362faab0a8b3bb1e2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 108900, "end_char_idx": 112205, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The average depth of the Arctic Ocean is . The deepest point is in the Eurasian Basin, at .
    
    Geography 
    The Arctic Ocean covers an area of about 14,056,000 km2.  The coastline is 45,390 km (28,200 mi) long It is surrounded by Eurasia, North America, Greenland, and by several islands.
    
    It is generally taken to include Baffin Bay, Barents Sea, Beaufort Sea, Chukchi Sea, East Siberian Sea, Greenland Sea, Hudson Bay, Hudson Strait, Kara Sea, Laptev Sea, White Sea and other bodies of water. It is connected to the Pacific Ocean by the Bering Strait  and to the Atlantic Ocean through the Greenland Sea and Labrador Sea.
    
    Countries bordering the Arctic Ocean are: Russia, Norway, Iceland, Greenland, Canada and the United States.
    
    Climate 
    The Arctic Ocean is in a polar climate. Winters are characterized by the polar night, cold and stable weather conditions, and clear skies.
    
    The temperature of the surface of the Arctic Ocean is fairly constant, near the freezing point of seawater.  Arctic Ocean consists of saltwater but its salinity is less than other oceans. The temperature must reach −1.8 °C (28.8 °F) before freezing occurs.
    
    Ice covers most of the Arctic Ocean. It covers almost the whole ocean in late winter and the majority of the ocean in late summer. Much of the Arctic ice pack is covered in snow for about 10 months of the year. The maximum snow cover is in March or April — about 20 to 50 cm (7.9 to 19.7 in).
    
    The climate of the Arctic region has varied significantly in the past. As recently as 55 million years ago, during the eocene epoch, the region reached an average annual temperature of 10–20 °C (50–68 °F). The surface waters of the Arctic Ocean warmed enough to support tropical lifeforms.
    
    Animal and plant life 
    Endangered marine species in the Arctic Ocean include walruses and whales. The area has a fragile ecosystem. The Arctic Ocean has relatively little plant life except for phytoplankton.  Phytoplankton are a crucial part of the ocean. They feed on nutrients from rivers and the currents of the Atlantic and Pacific oceans.
    
    References
    
    Other websites 
    
     The Hidden Ocean Arctic 2005 Daily logs, photos and video from exploration mission.
     Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
     Arctic Council
     The Northern Forum
     Arctic Environmental Atlas  Interactive map
     NOAA Arctic Theme Page
     
     Daily Arctic Ocean Rawinsonde Data from Soviet Drifting Ice Stations (1954–1990) at NSIDC
     Arctic time series: The Unaami Data collection 
     NOAA North Pole Web Cam Images from Web Cams deployed in spring on an ice floe
     NOAA Near-realtime North Pole Weather Data Data from instruments deployed on an ice floe
     Search for Arctic Life Heats Up by Stephen Leahy
     International Polar Foundation
     National Snow and Ice Data Center – Daily report of Arctic ice cover based on satellite data
     Marine Biodiversity Wiki 
    
    Oceans
    Arctic
    Score: 0.9073483
    
    ID: 15220
    Title: Caribbean Sea
    Summary: The Caribbean Sea is a tropical sea in the center of the Caribbean area. The body of water is part of the Atlantic Ocean. The sea is southeast of the Gulf of Mexico.


key: _node_content
string_value:
{"id_": "3dd26252-ef11-4d31-a51b-6f0e58971706", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "fc306103-6dc2-4926-b2c9-f12047e15555", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "89b74f965a0f3a7bb4adade388a50c2536b26800d7a64aee5b7f9efb3fc6ab06", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b9309548-75fd-4ab6-ac80-5a8f0140e503", "node_type": "1", "metadata": {}, "hash": "ff35c3953297b5358ae5910a844d70491c2ef547193e0d008b05bebf11a106ee", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 110466, "end_char_idx": 113837, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
As recently as 55 million years ago, during the eocene epoch, the region reached an average annual temperature of 10–20 °C (50–68 °F). The surface waters of the Arctic Ocean warmed enough to support tropical lifeforms.
    
    Animal and plant life 
    Endangered marine species in the Arctic Ocean include walruses and whales. The area has a fragile ecosystem. The Arctic Ocean has relatively little plant life except for phytoplankton.  Phytoplankton are a crucial part of the ocean. They feed on nutrients from rivers and the currents of the Atlantic and Pacific oceans.
    
    References
    
    Other websites 
    
     The Hidden Ocean Arctic 2005 Daily logs, photos and video from exploration mission.
     Oceanography Image of the Day, from the Woods Hole Oceanographic Institution
     Arctic Council
     The Northern Forum
     Arctic Environmental Atlas  Interactive map
     NOAA Arctic Theme Page
     
     Daily Arctic Ocean Rawinsonde Data from Soviet Drifting Ice Stations (1954–1990) at NSIDC
     Arctic time series: The Unaami Data collection 
     NOAA North Pole Web Cam Images from Web Cams deployed in spring on an ice floe
     NOAA Near-realtime North Pole Weather Data Data from instruments deployed on an ice floe
     Search for Arctic Life Heats Up by Stephen Leahy
     International Polar Foundation
     National Snow and Ice Data Center – Daily report of Arctic ice cover based on satellite data
     Marine Biodiversity Wiki 
    
    Oceans
    Arctic
    Score: 0.9073483
    
    ID: 15220
    Title: Caribbean Sea
    Summary: The Caribbean Sea is a tropical sea in the center of the Caribbean area. The body of water is part of the Atlantic Ocean. The sea is southeast of the Gulf of Mexico. The Caribbean Sea has many islands, which are popular among North American tourists because of their tropical climate. The Caribbean Sea is famous around the world as a tourist destination.
    
    History 
    Christopher Columbus came across a group of islands in the Caribbean region. When he did so, he thought he had reached another part of the world. Because of this, he named the islands the ‘West Indies’. However, later it was realized that he found an entire region. It still had its natural resources. The name ‘Caribbean’ was later given to it by the Amerindian tribe, the Caribs. That is how it got its name: the Caribbean Sea.
    
    This entire region covers an area of 1,063,000 sq. miles. It covers from Mexico to the boundaries of South America.
    
    This sea is just as deep as it is wide. Its deepest point is believed to be even lower than 25,220 ft, 7,686 m. That makes this point one of the lowest points on the surface of the earth, and the Caribbean Sea one of the deepest seas in the world.
    
    Other websites 
    
    Seas of the Atlantic Ocean
    Score: 0.90673447
    
    ID: 21206
    Title: Irish Sea
    Summary: The Irish Sea (sometimes called the Manx Sea) is a body of water that separates Ireland and Great Britain. It is known to be one of the most polluted seas in the world including the North Sea and the Mediterranean Sea. The sea is important to regional trade, shipping and fishing.  It is a source of power generation in the form of wind power and nuclear plants. Annual traffic between Great Britain and Ireland amounts to over 12 million passengers and  of traded goods.


key: _node_content
string_value:
{"id_": "b9309548-75fd-4ab6-ac80-5a8f0140e503", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "3dd26252-ef11-4d31-a51b-6f0e58971706", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b6fecdc8e655f8978017a17797e441e3448315e3a722eaef6cf3c18867726513", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "5a54131b-b0b8-4b56-8bca-b4a7b5f1b484", "node_type": "1", "metadata": {}, "hash": "60cdad903303d00441c0def74c5a3977bd6e7477f2176e85f5df24fa26b90feb", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 112162, "end_char_idx": 115319, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
The sea is southeast of the Gulf of Mexico. The Caribbean Sea has many islands, which are popular among North American tourists because of their tropical climate. The Caribbean Sea is famous around the world as a tourist destination.
    
    History 
    Christopher Columbus came across a group of islands in the Caribbean region. When he did so, he thought he had reached another part of the world. Because of this, he named the islands the ‘West Indies’. However, later it was realized that he found an entire region. It still had its natural resources. The name ‘Caribbean’ was later given to it by the Amerindian tribe, the Caribs. That is how it got its name: the Caribbean Sea.
    
    This entire region covers an area of 1,063,000 sq. miles. It covers from Mexico to the boundaries of South America.
    
    This sea is just as deep as it is wide. Its deepest point is believed to be even lower than 25,220 ft, 7,686 m. That makes this point one of the lowest points on the surface of the earth, and the Caribbean Sea one of the deepest seas in the world.
    
    Other websites 
    
    Seas of the Atlantic Ocean
    Score: 0.90673447
    
    ID: 21206
    Title: Irish Sea
    Summary: The Irish Sea (sometimes called the Manx Sea) is a body of water that separates Ireland and Great Britain. It is known to be one of the most polluted seas in the world including the North Sea and the Mediterranean Sea. The sea is important to regional trade, shipping and fishing.  It is a source of power generation in the form of wind power and nuclear plants. Annual traffic between Great Britain and Ireland amounts to over 12 million passengers and  of traded goods.
    
    Economics 
    It covers  and at its deepest point is  deep. In 2008, about  of fish were caught. Shell fish made up three quarters of this amount.
    
    The Irish Sea has 17 active oil and gas drilling platforms. It is estimated there are about 1.6 billion barrels of oil in the Barryroe oil field alone.
    
    Sealife 
    At least thirty species of shark can be found in the Irish Sea at different times. These include the basking, thresher, blue, mako and porbeagle sharks. There are about 12 species of Dolphin, porpoise and whales in the Irish Sea. These include the common dolphin, bottlenose dolphin and the harbor porpoise.
    
    References 
    
    Seas of the Atlantic Ocean
    Ireland
    Geography of the United Kingdom
    Score: 0.90410626
    
    ID: 6308
    Title: North Sea
    Summary: The North Sea is a sea that is part of the Atlantic Ocean in northern Europe. The North Sea is between Norway and Denmark in the east, Scotland and England in the west, Germany, the Netherlands, Belgium and France in the south.
    
    Borders 
    The Skagerrak connects the North Sea to the Baltic Sea. In the south, the North Sea becomes the English Channel, a sea between England and France. This is called the Dover Straits and is very busy with ships.
    
    The border between the North Sea and the Skagerrak is at an imagined line between Lindesnes in Norway, and Hanstholm in Denmark. In the North, the North sea is open towards the Atlantic.


key: _node_content
string_value:
{"id_": "5a54131b-b0b8-4b56-8bca-b4a7b5f1b484", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b9309548-75fd-4ab6-ac80-5a8f0140e503", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "abca709d64d90a514e6b726fd4ded5cf6ef13d1a23ece8a1329abded08585d67", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c516abc2-b1cc-4f9c-877d-64c2d8c944da", "node_type": "1", "metadata": {}, "hash": "41162d1daf3ffff5e507f9748ab03a7cc8266ad33dce7bd3c1cfdcd6ce1d96f2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 113648, "end_char_idx": 116591, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
It is a source of power generation in the form of wind power and nuclear plants. Annual traffic between Great Britain and Ireland amounts to over 12 million passengers and  of traded goods.
    
    Economics 
    It covers  and at its deepest point is  deep. In 2008, about  of fish were caught. Shell fish made up three quarters of this amount.
    
    The Irish Sea has 17 active oil and gas drilling platforms. It is estimated there are about 1.6 billion barrels of oil in the Barryroe oil field alone.
    
    Sealife 
    At least thirty species of shark can be found in the Irish Sea at different times. These include the basking, thresher, blue, mako and porbeagle sharks. There are about 12 species of Dolphin, porpoise and whales in the Irish Sea. These include the common dolphin, bottlenose dolphin and the harbor porpoise.
    
    References 
    
    Seas of the Atlantic Ocean
    Ireland
    Geography of the United Kingdom
    Score: 0.90410626
    
    ID: 6308
    Title: North Sea
    Summary: The North Sea is a sea that is part of the Atlantic Ocean in northern Europe. The North Sea is between Norway and Denmark in the east, Scotland and England in the west, Germany, the Netherlands, Belgium and France in the south.
    
    Borders 
    The Skagerrak connects the North Sea to the Baltic Sea. In the south, the North Sea becomes the English Channel, a sea between England and France. This is called the Dover Straits and is very busy with ships.
    
    The border between the North Sea and the Skagerrak is at an imagined line between Lindesnes in Norway, and Hanstholm in Denmark. In the North, the North sea is open towards the Atlantic. The border between the two is an imagined line from Northern Scotland, to Shetland, and then to Ålesund in Norway. According to the Oslo-Paris Treaty of 1962 it is a bit more to the west and the north though. The treaty puts it at 5° East longitude, and 62° North latitude. That is at the parallel of the Geirangerfjord in Norway.
    
    Various statistical data 
    On average, the North Sea has a depth of only 94 meters. About 80 million people live near the North Sea, at most 150 km away from the coast. Together with the English Channel in the south, the southern North Sea is the busiest body of water in the world.
    
    Rivers that drain into it 
    Well-known rivers that drain into the North Sea include the Tay (at Dundee), the Forth (at Edinburgh), the Tyne (South Shields), the Wear (at Sunderland),  the Tees (near Middlesbrough),  the Elbe (at Cuxhaven), the Weser (at Bremerhaven), the Rhine and Meuse or Maas (at Rotterdam), the Scheldt (at Flushing or Vlissingen), the Thames, and the Humber (at Hull), and the river Nairn (at Nairn)
    
    The Kiel Canal, one of the world's busiest artificial waterways, connects the North Sea with the Baltic.
    
    Name 
    Its name comes from its relationship to the land of the Frisians (see Frisia).


key: _node_content
string_value:
{"id_": "c516abc2-b1cc-4f9c-877d-64c2d8c944da", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "5a54131b-b0b8-4b56-8bca-b4a7b5f1b484", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "7813ca36cffc327583997b3d84bc4d96ef91b3e996b7b39200fdaa015ca46508", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2b93294e-da25-444f-9fdc-97c4422d65ef", "node_type": "1", "metadata": {}, "hash": "605c59ac0f82961b26e29fa317be67cf489f653df99e092ca176ba96bda7d0b8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 115062, "end_char_idx": 118007, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "2b93294e-da25-444f-9fdc-97c4422d65ef", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c516abc2-b1cc-4f9c-877d-64c2d8c944da", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "b880c6c9899656fc79ec93d1b41ec40c83fd55d5a34dca36e90c0f37a9488eb6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c9eeab8c-ce30-4483-bbc4-0c573af07bc6", "node_type": "1", "metadata": {}, "hash": "c7c797e1ca8c93a8cd41c850ca659ca16ad75c03a81fd8f64ae522b7ff603b3d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 116503, "end_char_idx": 119698, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: _node_content
string_value:
{"id_": "c9eeab8c-ce30-4483-bbc4-0c573af07bc6", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2b93294e-da25-444f-9fdc-97c4422d65ef", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9764401ac78f9615d27a3e459bbc67dd40cfeddb29c882af9d46fe281baad2a7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "62e42cf1-40d6-44e7-ad94-4abac4984220", "node_type": "1", "metadata": {}, "hash": "950495248c5838f3cb91ed24cc11ae9d0ec2c8c068a7baf9363a5fd377096fe1", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 117959, "end_char_idx": 121266, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
In the southern part, there are many sand banks.
    
    Looking at the satellite picture it is easy to see the geographic divisions of the North Sea:
    a generally shallow southern North Sea
    the central North Sea
     the northern North Sea, with the Norwegian Trench, near the Skagerrak.
    
    The southern north sea is composed of the Southern Bight, before the coast of Belgium and the Netherlands and the German Bight before the coastline of Germany. The Dogger Bank is the limit between the southern and central parts. The Waddenzee runs all the way from Den Helder in the Netherlands to Esbjerg in Denmark.
    
    The Dogger Bank covers an area about half the size of the Netherlands. There, the North Sea has a depth of between 13 and 20 metres only. The area is very famous for fishing. With some storms there are even waves breaking there.
    
    The Norwegian Trench has an average depth of around 250 to 300 metres; at the entrance to the Skagerrak, the depth increases up to 725 meters. Along the trench is the Norwegian Current, which brings most of the waters of the North Sea into the Atlantic Ocean. Also, most of the waters of the Baltic Sea flow northwards here.
    
    About 200 km east of the Scottish city of Dundee there are more trenches, known collectively as the Devil's hole. Generally, the water is about 90 meters deep there. The trenches very often are only a few kilometers in length. In these trenches, the depth increases to up to 230 meters.
    
    In the Dover Strait the water is about 30 meters deep. At the end of the English Channel, this depth increases to about 100 meters.
    
    History 
    In the last ice age the North Sea was covered by large areas of ice called glaciers. About 20,000 years ago the ice melted and the North Sea was formed (made).
    
    North Sea oil 
    In the 1960s, geologists found large areas of oil and natural gas under the North Sea. Most of the oil fields are owned by the United Kingdom and Norway but some belong to Denmark, the Netherlands and Germany. Drilling began in the 1960s and led to a famous argument between England and Scotland about how the revenue (money) from the oil should be spent.
    
    Animal life 
    
    People have been fishing in the North Sea for thousands of years. However, so many fish are now caught there that new ones may not be able to grow fast enough to keep the fishery going.
    
    Terns, Atlantic puffins, razorbills, kittiwakes and other seabirds live on the North Sea coast. Many coastal areas are protected nature reserves.
    
    Other websites
    
    Seas of the Atlantic Ocean
    Bodies of water of Europe
    Score: 0.9022158
    
    ID: 6278
    Title: Atlantis
    Summary: Atlantis is a name for a fictional large island or small continent that was (in the legend) in the Atlantic Ocean many years before it sank into the depth of the sea .
    
    The name Atlantis first appears in the writings of Herodotus - he describes the western ocean as "Sea of Atlantis." Then, one generation later, Atlantis is described in detail in the stories Timaeus and Critias by the Greek philosopher Plato.  He used this story to help explain his ideas about government and philosophy. Plato was the only ancient writer who wrote specific things about Atlantis.


key: _node_content
string_value:
{"id_": "62e42cf1-40d6-44e7-ad94-4abac4984220", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c9eeab8c-ce30-4483-bbc4-0c573af07bc6", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "15e16c0b166ca0381c74d0f02f2f8a94fb1baafc8c5d2a37297f2b2f850fd068", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b44fdfb4-c728-4912-abb7-0817051d55f9", "node_type": "1", "metadata": {}, "hash": "371f543cd68823cef3b01eae101715f8ad0882191dea3b12417589611e05f93d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 119515, "end_char_idx": 123004, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
At the end of the English Channel, this depth increases to about 100 meters.
    
    History 
    In the last ice age the North Sea was covered by large areas of ice called glaciers. About 20,000 years ago the ice melted and the North Sea was formed (made).
    
    North Sea oil 
    In the 1960s, geologists found large areas of oil and natural gas under the North Sea. Most of the oil fields are owned by the United Kingdom and Norway but some belong to Denmark, the Netherlands and Germany. Drilling began in the 1960s and led to a famous argument between England and Scotland about how the revenue (money) from the oil should be spent.
    
    Animal life 
    
    People have been fishing in the North Sea for thousands of years. However, so many fish are now caught there that new ones may not be able to grow fast enough to keep the fishery going.
    
    Terns, Atlantic puffins, razorbills, kittiwakes and other seabirds live on the North Sea coast. Many coastal areas are protected nature reserves.
    
    Other websites
    
    Seas of the Atlantic Ocean
    Bodies of water of Europe
    Score: 0.9022158
    
    ID: 6278
    Title: Atlantis
    Summary: Atlantis is a name for a fictional large island or small continent that was (in the legend) in the Atlantic Ocean many years before it sank into the depth of the sea .
    
    The name Atlantis first appears in the writings of Herodotus - he describes the western ocean as "Sea of Atlantis." Then, one generation later, Atlantis is described in detail in the stories Timaeus and Critias by the Greek philosopher Plato.  He used this story to help explain his ideas about government and philosophy. Plato was the only ancient writer who wrote specific things about Atlantis.
    
    According to Plato, the Atlanteans lived 9000 years before his own time and were half human and half god.  They created a very good human society.  When they stopped being good people and did bad things, the gods sent earthquakes and fire to destroy Atlantis.
    
    Many scholars think Plato could have been thinking of a real place when he wrote about Atlantis.  Many, many people have thought of many, many places where the real place that inspired Atlantis could have been.  For example, there was a Minoan kingdom on the island of Santorini.  The Minoan kingdom was very powerful thousands of years before Plato, and their society was damaged when a volcano erupted on their island. According to Plato, Atlantis was very large, as big as North Africa, so it should not have been hard to find.
    
    After the discovery of the Americas, some people in Europe thought they might be Atlantis. However, after Plato, the idea of Atlantis was mostly forgotten until 1882, when a writer named Ignatius Donnelly wrote a book saying that Atlantis was real and that the culture of Atlantis had started many other ancient cultures, such as the Egyptian and Mayan.  Then other people became interested in Atlantis. 
    
    Atlantis has appeared in many works of fiction.  In Marvel Comics, Atlantis is at the bottom of the ocean and exists in modern times, with people who breathe water.  Other works of fiction use Atlantis as background.  For example, Robert E. Howard set his Conan the Barbarian stories in a fictional time called the Hyborian Age, which began with the destruction of Atlantis and ended when real written history started.
    
    References
    
    Greek mythology
    Ancient history
    Score: 0.90082896


key: _node_content
string_value:
{"id_": "b44fdfb4-c728-4912-abb7-0817051d55f9", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "fec9d699-ef06-4f9a-a5c3-95ca3aac39a4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d87607119004b9428a3ad29eef529770faa5b7d1140477aff6167fabb456d032", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "62e42cf1-40d6-44e7-ad94-4abac4984220", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb", "title": "Semantic search using Elasticsearch and OpenAI", "tokens": 28076, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "5df77cf178769b11c2cb85135f1c36a4093436f1ad8b56927e7d0987dab6ff3c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 121113, "end_char_idx": 123755, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
He used this story to help explain his ideas about government and philosophy. Plato was the only ancient writer who wrote specific things about Atlantis.
    
    According to Plato, the Atlanteans lived 9000 years before his own time and were half human and half god.  They created a very good human society.  When they stopped being good people and did bad things, the gods sent earthquakes and fire to destroy Atlantis.
    
    Many scholars think Plato could have been thinking of a real place when he wrote about Atlantis.  Many, many people have thought of many, many places where the real place that inspired Atlantis could have been.  For example, there was a Minoan kingdom on the island of Santorini.  The Minoan kingdom was very powerful thousands of years before Plato, and their society was damaged when a volcano erupted on their island. According to Plato, Atlantis was very large, as big as North Africa, so it should not have been hard to find.
    
    After the discovery of the Americas, some people in Europe thought they might be Atlantis. However, after Plato, the idea of Atlantis was mostly forgotten until 1882, when a writer named Ignatius Donnelly wrote a book saying that Atlantis was real and that the culture of Atlantis had started many other ancient cultures, such as the Egyptian and Mayan.  Then other people became interested in Atlantis. 
    
    Atlantis has appeared in many works of fiction.  In Marvel Comics, Atlantis is at the bottom of the ocean and exists in modern times, with people who breathe water.  Other works of fiction use Atlantis as background.  For example, Robert E. Howard set his Conan the Barbarian stories in a fictional time called the Hyborian Age, which began with the destruction of Atlantis and ended when real written history started.
    
    References
    
    Greek mythology
    Ancient history
    Score: 0.90082896


## Next steps

Success! Now you know how to use Elasticsearch as a vector database to store embeddings, encode queries by calling the OpenAI `embeddings` endpoint, and run semantic search.

Play around with different queries, and if you want to try with your own data, you can experiment with different embedding models.

ℹ️ Check out our other notebook [Retrieval augmented generation using Elasticsearch and OpenAI](openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb). That notebook builds on this example to demonstrate how to use Elasticsearch together with the OpenAI [chat completions](https://platform.openai.com/docs/api-reference/chat) API for retrieval augmented generation (RAG).


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/README.ipynb


key: _node_content
string_value:
{"id_": "ee0ab3b0-b588-4663-87e7-e5a217f35b58", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/README.ipynb", "title": "Elasticsearch", "tokens": 324, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "eccf632c-84a9-4027-b011-083b77cdd06a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/README.ipynb", "title": "Elasticsearch", "tokens": 324, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "3ccc242a5ae33cbbbe82550bb4e72bf37f8cae1f40dae977af4d5ee5e6854e57", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 1523, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Elasticsearch

Elasticsearch is a popular search/analytics engine and [vector database](https://www.elastic.co/elasticsearch/vector-database).
Elasticsearch offers an efficient way to create, store, and search vector embeddings at scale.

For technical details, refer to the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/knn-search.html).

The [`elasticsearch-labs`](https://github.com/elastic/elasticsearch-labs) repo contains executable Python notebooks, sample apps, and resources for testing out the Elastic platform.

## OpenAI cookbook notebooks 📒

Check out our notebooks in this repo for working with OpenAI, using Elasticsearch as your vector database.

### [Semantic search](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-semantic-search.ipynb)

In this notebook you'll learn how to:

 - Index the OpenAI Wikipedia embeddings dataset into Elasticsearch
 - Encode a question with the `openai ada-02` model
 - Perform a semantic search

<hr>


### [Retrieval augmented generation](https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/elasticsearch/elasticsearch-retrieval-augmented-generation.ipynb)

This notebooks builds on the semantic search notebook by:

- Selecting the top hit from a semantic search
- Sending that result to the OpenAI [Chat Completions](https://platform.openai.com/docs/guides/gpt/chat-completions-api) API endpoint for retrieval augmented generation (RAG)


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb


key: _node_content
string_value:
{"id_": "23a4b8a7-6879-4bd0-a736-b868ccda75df", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "6450d870-7012-41fb-93d0-f473aab88e31", "node_type": "1", "metadata": {}, "hash": "06492cb4e28eb7624892892d6d68018f8446b0cb096537fa7494070ac60c524a", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2459, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Getting Started with Milvus and OpenAI
### Finding your next book

In this notebook we will be going over generating embeddings of book descriptions with OpenAI and using those embeddings within Milvus to find relevant books. The dataset in this example is sourced from HuggingFace datasets, and contains a little over 1 million title-description pairs.

Lets begin by first downloading the required libraries for this notebook:
- `openai` is used for communicating with the OpenAI embedding service
- `pymilvus` is used for communicating with the Milvus server
- `datasets` is used for downloading the dataset
- `tqdm` is used for the progress bars



```python
! pip install openai pymilvus datasets tqdm
```

    Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
    Requirement already satisfied: openai in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (0.27.2)
    Requirement already satisfied: pymilvus in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (2.2.2)
    Requirement already satisfied: datasets in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (2.10.1)
    Requirement already satisfied: tqdm in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (4.64.1)
    Requirement already satisfied: aiohttp in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from openai) (3.8.4)
    Requirement already satisfied: requests>=2.20 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from openai) (2.28.2)
    Requirement already satisfied: pandas>=1.2.4 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.5.3)
    Requirement already satisfied: ujson<=5.4.0,>=2.0.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (5.1.0)
    Requirement already satisfied: mmh3<=3.0.0,>=2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (3.0.0)
    Requirement already satisfied: grpcio<=1.48.0,>=1.47.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.47.2)
    Requirement already satisfied: grpcio-tools<=1.48.0,>=1.47.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.47.2)
    Requirement already satisfied: huggingface-hub<1.0.0,>=0.2.


key: _node_content
string_value:
{"id_": "6450d870-7012-41fb-93d0-f473aab88e31", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "23a4b8a7-6879-4bd0-a736-b868ccda75df", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "9c601a39a53550cc6af76cd384ae644270ad2b4508a3fe8621ac5be32df5146b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1044431f-5597-48b3-bfa5-048c5e18c980", "node_type": "1", "metadata": {}, "hash": "1914189bad3bde2e9962fc2a932a36f835248724c335242ee74d4ee0a453b450", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1413, "end_char_idx": 3673, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
9/site-packages (from openai) (3.8.4)
    Requirement already satisfied: requests>=2.20 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from openai) (2.28.2)
    Requirement already satisfied: pandas>=1.2.4 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.5.3)
    Requirement already satisfied: ujson<=5.4.0,>=2.0.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (5.1.0)
    Requirement already satisfied: mmh3<=3.0.0,>=2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (3.0.0)
    Requirement already satisfied: grpcio<=1.48.0,>=1.47.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.47.2)
    Requirement already satisfied: grpcio-tools<=1.48.0,>=1.47.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pymilvus) (1.47.2)
    Requirement already satisfied: huggingface-hub<1.0.0,>=0.2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.12.1)
    Requirement already satisfied: dill<0.3.7,>=0.3.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.3.6)
    Requirement already satisfied: xxhash in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (3.2.0)
    Requirement already satisfied: pyyaml>=5.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (5.4.1)
    Requirement already satisfied: fsspec[http]>=2021.11.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (2023.1.0)
    Requirement already satisfied: packaging in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (23.0)
    Requirement already satisfied: numpy>=1.17 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (1.23.5)
    Requirement already satisfied: multiprocess in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.70.14)
    Requirement already satisfied: pyarrow>=6.0.


key: _node_content
string_value:
{"id_": "1044431f-5597-48b3-bfa5-048c5e18c980", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "6450d870-7012-41fb-93d0-f473aab88e31", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "8733fe2b0a150975b3ceeb017a1e3fcc64d39a7665e3da0b7d5f142e372acb94", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ceabbea9-983b-4499-99cb-844bf8b9ad2c", "node_type": "1", "metadata": {}, "hash": "46c75ed86932c21eb33afb9e09364fe69884d55a7fe06e60118276bb47930b17", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2449, "end_char_idx": 4807, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
0.0,>=0.2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.12.1)
    Requirement already satisfied: dill<0.3.7,>=0.3.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.3.6)
    Requirement already satisfied: xxhash in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (3.2.0)
    Requirement already satisfied: pyyaml>=5.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (5.4.1)
    Requirement already satisfied: fsspec[http]>=2021.11.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (2023.1.0)
    Requirement already satisfied: packaging in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (23.0)
    Requirement already satisfied: numpy>=1.17 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (1.23.5)
    Requirement already satisfied: multiprocess in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.70.14)
    Requirement already satisfied: pyarrow>=6.0.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (10.0.1)
    Requirement already satisfied: responses<0.19 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.18.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: frozenlist>=1.1.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.3.3)
    Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (4.0.2)
    Requirement already satisfied: yarl<2.0,>=1.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.8.2)
    Requirement already satisfied: aiosignal>=1.1.2 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.3.1)
    Requirement already satisfied: charset-normalizer<4.0,


key: _node_content
string_value:
{"id_": "ceabbea9-983b-4499-99cb-844bf8b9ad2c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1044431f-5597-48b3-bfa5-048c5e18c980", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "94e2a332debd5ab0cfdd55286b654d6ef4bd839a3aa393f41e176eef18f49405", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "0dcf694d-8620-4598-baee-a66871dedb59", "node_type": "1", "metadata": {}, "hash": "4da2aa5353fd988236f6544bdb8a58694d0ee9968c021da33abb12725e30a962", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3618, "end_char_idx": 5803, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
70.14)
    Requirement already satisfied: pyarrow>=6.0.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (10.0.1)
    Requirement already satisfied: responses<0.19 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from datasets) (0.18.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (6.0.4)
    Requirement already satisfied: frozenlist>=1.1.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.3.3)
    Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (4.0.2)
    Requirement already satisfied: yarl<2.0,>=1.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.8.2)
    Requirement already satisfied: aiosignal>=1.1.2 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (1.3.1)
    Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (3.0.1)
    Requirement already satisfied: attrs>=17.3.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (22.2.0)
    Requirement already satisfied: six>=1.5.2 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio<=1.48.0,>=1.47.0->pymilvus) (1.16.0)
    Requirement already satisfied: protobuf<4.0dev,>=3.12.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio-tools<=1.48.0,>=1.47.0->pymilvus) (3.20.1)
    Requirement already satisfied: setuptools in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio-tools<=1.48.0,>=1.47.0->pymilvus) (65.6.3)
    Requirement already satisfied: filelock in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (3.9.


key: _node_content
string_value:
{"id_": "0dcf694d-8620-4598-baee-a66871dedb59", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ceabbea9-983b-4499-99cb-844bf8b9ad2c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "dd537787c71fcc19ef4f957f8152d92e56e126a592f6545eb9e3c3852ffedd63", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d85b2586-c759-42f9-93cb-266919b90095", "node_type": "1", "metadata": {}, "hash": "1a1942cd96e5021e536f597c3dd3719b75122376850b3d2df47b71ae7a38311c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4702, "end_char_idx": 6854, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
9/site-packages (from aiohttp->openai) (1.3.1)
    Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (3.0.1)
    Requirement already satisfied: attrs>=17.3.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from aiohttp->openai) (22.2.0)
    Requirement already satisfied: six>=1.5.2 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio<=1.48.0,>=1.47.0->pymilvus) (1.16.0)
    Requirement already satisfied: protobuf<4.0dev,>=3.12.0 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio-tools<=1.48.0,>=1.47.0->pymilvus) (3.20.1)
    Requirement already satisfied: setuptools in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from grpcio-tools<=1.48.0,>=1.47.0->pymilvus) (65.6.3)
    Requirement already satisfied: filelock in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (3.9.0)
    Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (4.5.0)
    Requirement already satisfied: python-dateutil>=2.8.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pandas>=1.2.4->pymilvus) (2.8.2)
    Requirement already satisfied: pytz>=2020.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pandas>=1.2.4->pymilvus) (2022.7.1)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (1.26.14)
    Requirement already satisfied: idna<4,>=2.5 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: certifi>=2017.4.17 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (2022.12.7)


key: _node_content
string_value:
{"id_": "d85b2586-c759-42f9-93cb-266919b90095", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0dcf694d-8620-4598-baee-a66871dedb59", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "476667c7b57089f5854f36efeecbdf00a5829a381eda15ee4728b12b19fc16ce", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "cc66812c-3f15-4eaf-bd98-76627654df77", "node_type": "1", "metadata": {}, "hash": "5415167b94e7c32b7afa65668a089e588b6f7c38fc6be8715f6efb74c8028af5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5775, "end_char_idx": 7953, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
0.0,>=0.2.0->datasets) (3.9.0)
    Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (4.5.0)
    Requirement already satisfied: python-dateutil>=2.8.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pandas>=1.2.4->pymilvus) (2.8.2)
    Requirement already satisfied: pytz>=2020.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from pandas>=1.2.4->pymilvus) (2022.7.1)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (1.26.14)
    Requirement already satisfied: idna<4,>=2.5 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (3.4)
    Requirement already satisfied: certifi>=2017.4.17 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (2022.12.7)


With the required packages installed we can get started. Lets begin by launching the Milvus service. The file being run is the `docker-compose.yaml` found in the folder of this file. This command launches a Milvus standalone instance which we will use for this test.  


```python
! docker compose up -d
```

    [1A[1B[0G[?25l[+] Running 0/0
    [37m ⠋ Network milvus  Creating                                                0.1s
    [0m[?25h[1A[1A[0G[?25l[34m[+] Running 1/1[0m
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠋ Container milvus-minio  Creating                                        0.1s
    [0m[37m ⠋ Container milvus-etcd   Creating                                        0.1s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠙ Container milvus-minio  Creating                                        0.2s
    [0m[37m ⠙ Container milvus-etcd   Creating                                        0.


key: _node_content
string_value:
{"id_": "cc66812c-3f15-4eaf-bd98-76627654df77", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d85b2586-c759-42f9-93cb-266919b90095", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "eb4b9223653056f44afa3fcc193b567c08905a5bef5a93c9a4f69ce3cedc4421", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "360a8020-2d5e-48af-b1c6-622426e960e1", "node_type": "1", "metadata": {}, "hash": "7bdb1af130e6059b0ff6642114df955c9b274d9d93a80c6c10d2e2fdb0f0b511", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6663, "end_char_idx": 9052, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
20->openai) (3.4)
    Requirement already satisfied: certifi>=2017.4.17 in /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages (from requests>=2.20->openai) (2022.12.7)


With the required packages installed we can get started. Lets begin by launching the Milvus service. The file being run is the `docker-compose.yaml` found in the folder of this file. This command launches a Milvus standalone instance which we will use for this test.  


```python
! docker compose up -d
```

    [1A[1B[0G[?25l[+] Running 0/0
    [37m ⠋ Network milvus  Creating                                                0.1s
    [0m[?25h[1A[1A[0G[?25l[34m[+] Running 1/1[0m
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠋ Container milvus-minio  Creating                                        0.1s
    [0m[37m ⠋ Container milvus-etcd   Creating                                        0.1s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠙ Container milvus-minio  Creating                                        0.2s
    [0m[37m ⠙ Container milvus-etcd   Creating                                        0.2s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠹ Container milvus-minio  Creating                                        0.3s
    [0m[37m ⠹ Container milvus-etcd   Creating                                        0.3s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[34m[+] Running 3/3[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[37m ⠋ Container milvus-standalone  Creating                                   0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.


key: _node_content
string_value:
{"id_": "360a8020-2d5e-48af-b1c6-622426e960e1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "cc66812c-3f15-4eaf-bd98-76627654df77", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "a4dad88ab857f9814e95e76c4064c58567279d6267a5656b3144bd30fbeb8b61", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a05d9495-2503-4790-b902-8c421cd0f8e5", "node_type": "1", "metadata": {}, "hash": "e8678964bf9efc89c45e91c4505bbb042892724d8c8af5d77623f24c2f1be52f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7860, "end_char_idx": 10159, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
2s
    [0m[37m ⠙ Container milvus-etcd   Creating                                        0.2s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠹ Container milvus-minio  Creating                                        0.3s
    [0m[37m ⠹ Container milvus-etcd   Creating                                        0.3s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[34m[+] Running 3/3[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[37m ⠋ Container milvus-standalone  Creating                                   0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[37m ⠙ Container milvus-standalone  Creating                                   0.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[34m[+] Running 4/4[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.7s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.


key: _node_content
string_value:
{"id_": "a05d9495-2503-4790-b902-8c421cd0f8e5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "360a8020-2d5e-48af-b1c6-622426e960e1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "0ea5f3f0b281f36c5f9ce2dd4acec81dcd7d9021afa1f9c405190d312f3906a9", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "558899d9-76bf-4a50-9cfc-6b8526bacc65", "node_type": "1", "metadata": {}, "hash": "eb6f5da275b7f3a26f956adbaf9d3afbd6586dbeded10756042d1530336fb76c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 8959, "end_char_idx": 11294, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[37m ⠙ Container milvus-standalone  Creating                                   0.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[34m[+] Running 4/4[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Created                                    0.3s
    [0m[34m ⠿ Container milvus-etcd        Created                                    0.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.7s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.8s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.8s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.9s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.9s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.0s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.0s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.


key: _node_content
string_value:
{"id_": "558899d9-76bf-4a50-9cfc-6b8526bacc65", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a05d9495-2503-4790-b902-8c421cd0f8e5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "18bded39e0aa33e18a5390c3718faa974e6e72e09e182608e3a7454b724acd60", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "3c771b19-b99a-4ffe-917c-5e4684335707", "node_type": "1", "metadata": {}, "hash": "bf1286a19ffe9725d1ba01871187cd70011f94f3b21e157d59542dff6c1f682d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 10159, "end_char_idx": 12485, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.8s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.8s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.9s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.9s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.0s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.0s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.1s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.2s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.2s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.3s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.


key: _node_content
string_value:
{"id_": "3c771b19-b99a-4ffe-917c-5e4684335707", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "558899d9-76bf-4a50-9cfc-6b8526bacc65", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d5c39921262d12dcc092783deb81f5f2a6fab2af30ed5ca6f2f9d0cdc7c10cea", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "22ab83f9-b650-45eb-abff-9a093af3666d", "node_type": "1", "metadata": {}, "hash": "17ec910260dae1b14dfd584803e3d46bedaebee40cd4f48eeed34ca95e088494", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11294, "end_char_idx": 13583, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.1s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.2s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.2s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.3s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.4s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.4s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.5s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.5s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.


key: _node_content
string_value:
{"id_": "22ab83f9-b650-45eb-abff-9a093af3666d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "3c771b19-b99a-4ffe-917c-5e4684335707", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "7fd28c556ce150f2176b23967f601bf1f3324622e7af23619fefdbf1d62385c9", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "6d4d6433-8ba1-4d26-9c49-4eb1a8b834a2", "node_type": "1", "metadata": {}, "hash": "bf042a2918ca20e898f34868df9eacdec658c58d4ae59ca5730204594dde648e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 12392, "end_char_idx": 14718, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.4s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.4s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.5s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.5s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.6s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.6s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.7s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.


key: _node_content
string_value:
{"id_": "6d4d6433-8ba1-4d26-9c49-4eb1a8b834a2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "22ab83f9-b650-45eb-abff-9a093af3666d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "dc655dc81b9c0a06b1571c2ef46c4f20a88b0234ea0e9173b64cb16fc5c00ea9", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "48a35d7d-2beb-4ec7-aac9-201cd095c9f8", "node_type": "1", "metadata": {}, "hash": "8d268fe7d0a8c3be0754f09e2bccf3c6ac354898bfb6fedc3b950413d1ad2323", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 13583, "end_char_idx": 15909, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.6s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.6s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.7s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   1.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.6s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.7s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.


key: _node_content
string_value:
{"id_": "48a35d7d-2beb-4ec7-aac9-201cd095c9f8", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "6d4d6433-8ba1-4d26-9c49-4eb1a8b834a2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f3dd18e087e68997e59e7e2447de9262ba129d3123dd6eb9d90c6d34e04522f2", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "509664d5-e507-4fc8-a47d-4b530b9229be", "node_type": "1", "metadata": {}, "hash": "3e2977280cd28d7fa7e1b24787db545d4599067503f3356d1caa1d1542bda64c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 14718, "end_char_idx": 17007, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.6s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.7s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.8s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.9s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.0s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.


key: _node_content
string_value:
{"id_": "509664d5-e507-4fc8-a47d-4b530b9229be", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "48a35d7d-2beb-4ec7-aac9-201cd095c9f8", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "d6d3c429ea228af08b36480a85c4320a4009485c4228a16da79907a8fb818bca", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "43250f85-b112-4aa1-bc08-409d1763b9f2", "node_type": "1", "metadata": {}, "hash": "9309ef64349f694afa09c212c583b5efa2ba3d49c36a94789e4fc08dbf492580", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 15816, "end_char_idx": 18142, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.8s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.9s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.0s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.


key: _node_content
string_value:
{"id_": "43250f85-b112-4aa1-bc08-409d1763b9f2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "509664d5-e507-4fc8-a47d-4b530b9229be", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "81d6c4eae99e9aaa4226303cd445a9273c0e934ef57da9a066b3766a7abda628", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "18015480-8d5d-4af9-b5eb-2200f5f9ca83", "node_type": "1", "metadata": {}, "hash": "cd6a25a4008bb799de34b7b2aed8e7ba076baebef74e035947608d09829000e5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 17007, "end_char_idx": 19333, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.4s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.5s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.


key: _node_content
string_value:
{"id_": "18015480-8d5d-4af9-b5eb-2200f5f9ca83", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "43250f85-b112-4aa1-bc08-409d1763b9f2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f0204e1233be62ccdfa283292291a06e1f39cde09c7db538031594cfba065c77", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1f3b4922-785d-4e52-b74e-27d196305b99", "node_type": "1", "metadata": {}, "hash": "832cfc1551aa5bda8d7e38921519a3dd0f98d2718e0a441b848d5ff462b69286", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 18142, "end_char_idx": 20354, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.4s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.5s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.6s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[34m[+] Running 4/4[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[34m ⠿ Container milvus-standalone  Started                                    2.6s
    [0m[?25h

With Milvus running we can setup our global variables:
- HOST: The Milvus host address
- PORT: The Milvus port number
- COLLECTION_NAME: What to name the collection within Milvus
- DIMENSION: The dimension of the embeddings
- OPENAI_ENGINE: Which embedding model to use
- openai.api_key: Your OpenAI account key
- INDEX_PARAM: The index settings to use for the collection
- QUERY_PARAM: The search parameters to use
- BATCH_SIZE: How many texts to embed and insert at once


key: _node_content
string_value:
{"id_": "1f3b4922-785d-4e52-b74e-27d196305b99", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "18015480-8d5d-4af9-b5eb-2200f5f9ca83", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "8f72f9c033210a8303c34c4c76e547db7f50c5077d8260c471abd005563ded46", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "7a3dac71-7203-408a-ba91-8a315bc91ee9", "node_type": "1", "metadata": {}, "hash": "1abbc5898e8e72a42d68672542d0a87cb2b137912e6398699b5efd5e678e48a2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 19147, "end_char_idx": 21735, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   2.6s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[34m[+] Running 4/4[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.8s
    [0m[34m ⠿ Container milvus-etcd        Started                                    1.7s
    [0m[34m ⠿ Container milvus-standalone  Started                                    2.6s
    [0m[?25h

With Milvus running we can setup our global variables:
- HOST: The Milvus host address
- PORT: The Milvus port number
- COLLECTION_NAME: What to name the collection within Milvus
- DIMENSION: The dimension of the embeddings
- OPENAI_ENGINE: Which embedding model to use
- openai.api_key: Your OpenAI account key
- INDEX_PARAM: The index settings to use for the collection
- QUERY_PARAM: The search parameters to use
- BATCH_SIZE: How many texts to embed and insert at once


```python
import openai

HOST = 'localhost'
PORT = 19530
COLLECTION_NAME = 'book_search'
DIMENSION = 1536
OPENAI_ENGINE = 'text-embedding-3-small'
openai.api_key = 'sk-your_key'

INDEX_PARAM = {
    'metric_type':'L2',
    'index_type':"HNSW",
    'params':{'M': 8, 'efConstruction': 64}
}

QUERY_PARAM = {
    "metric_type": "L2",
    "params": {"ef": 64},
}

BATCH_SIZE = 1000
```

## Milvus
This segment deals with Milvus and setting up the database for this use case. Within Milvus we need to setup a collection and index the collection. 


```python
from pymilvus import connections, utility, FieldSchema, Collection, CollectionSchema, DataType

# Connect to Milvus Database
connections.connect(host=HOST, port=PORT)
```


```python
# Remove collection if it already exists
if utility.has_collection(COLLECTION_NAME):
    utility.drop_collection(COLLECTION_NAME)
```


```python
# Create collection which includes the id, title, and embedding.
fields = [
    FieldSchema(name='id', dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name='title', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='description', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, dim=DIMENSION)
]
schema = CollectionSchema(fields=fields)
collection = Collection(name=COLLECTION_NAME, schema=schema)
```


key: _node_content
string_value:
{"id_": "7a3dac71-7203-408a-ba91-8a315bc91ee9", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1f3b4922-785d-4e52-b74e-27d196305b99", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "177441aacb49217ab725e542264b2895f2e2e4bb4df5b8c0fec318ca9f0c9f08", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1a98da4a-702d-443c-a382-557fc0ab37b1", "node_type": "1", "metadata": {}, "hash": "d93d38d3b67deec649226f40e35679f6699d2d4b4b15c40bfd76378202153c2b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 20357, "end_char_idx": 23194, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
import openai

HOST = 'localhost'
PORT = 19530
COLLECTION_NAME = 'book_search'
DIMENSION = 1536
OPENAI_ENGINE = 'text-embedding-3-small'
openai.api_key = 'sk-your_key'

INDEX_PARAM = {
    'metric_type':'L2',
    'index_type':"HNSW",
    'params':{'M': 8, 'efConstruction': 64}
}

QUERY_PARAM = {
    "metric_type": "L2",
    "params": {"ef": 64},
}

BATCH_SIZE = 1000
```

## Milvus
This segment deals with Milvus and setting up the database for this use case. Within Milvus we need to setup a collection and index the collection. 


```python
from pymilvus import connections, utility, FieldSchema, Collection, CollectionSchema, DataType

# Connect to Milvus Database
connections.connect(host=HOST, port=PORT)
```


```python
# Remove collection if it already exists
if utility.has_collection(COLLECTION_NAME):
    utility.drop_collection(COLLECTION_NAME)
```


```python
# Create collection which includes the id, title, and embedding.
fields = [
    FieldSchema(name='id', dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name='title', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='description', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, dim=DIMENSION)
]
schema = CollectionSchema(fields=fields)
collection = Collection(name=COLLECTION_NAME, schema=schema)
```


```python
# Create the index on the collection and load it.
collection.create_index(field_name="embedding", index_params=INDEX_PARAM)
collection.load()
```

## Dataset
With Milvus up and running we can begin grabbing our data. Hugging Face Datasets is a hub that holds many different user datasets, and for this example we are using Skelebor's book dataset. This dataset contains title-description pairs for over 1 million books. We are going to embed each description and store it within Milvus along with its title. 


```python
import datasets

# Download the dataset and only use the `train` portion (file is around 800Mb)
dataset = datasets.load_dataset('Skelebor/book_titles_and_descriptions_en_clean', split='train')
```

    /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
      from .autonotebook import tqdm as notebook_tqdm
    Found cached dataset parquet (/Users/filiphaltmayer/.cache/huggingface/datasets/Skelebor___parquet/Skelebor--book_titles_and_descriptions_en_clean-3596935b1d8a7747/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)


## Insert the Data
Now that we have our data on our machine we can begin embedding it and inserting it into Milvus. The embedding function takes in text and returns the embeddings in a list format.


key: _node_content
string_value:
{"id_": "1a98da4a-702d-443c-a382-557fc0ab37b1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "7a3dac71-7203-408a-ba91-8a315bc91ee9", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "2b679c42aa6344889870b1e8d11fca23c7341e1b34ce68d2df2df83d90563bdc", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1238080f-fc68-4f72-86de-5848cdbba1e0", "node_type": "1", "metadata": {}, "hash": "d13b30ab535b0a2b14e01b7e717ae104124501dbdd2272e3aa3a78215b355d73", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 21738, "end_char_idx": 24719, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
# Create the index on the collection and load it.
collection.create_index(field_name="embedding", index_params=INDEX_PARAM)
collection.load()
```

## Dataset
With Milvus up and running we can begin grabbing our data. Hugging Face Datasets is a hub that holds many different user datasets, and for this example we are using Skelebor's book dataset. This dataset contains title-description pairs for over 1 million books. We are going to embed each description and store it within Milvus along with its title. 


```python
import datasets

# Download the dataset and only use the `train` portion (file is around 800Mb)
dataset = datasets.load_dataset('Skelebor/book_titles_and_descriptions_en_clean', split='train')
```

    /Users/filiphaltmayer/miniconda3/envs/haystack/lib/python3.9/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
      from .autonotebook import tqdm as notebook_tqdm
    Found cached dataset parquet (/Users/filiphaltmayer/.cache/huggingface/datasets/Skelebor___parquet/Skelebor--book_titles_and_descriptions_en_clean-3596935b1d8a7747/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)


## Insert the Data
Now that we have our data on our machine we can begin embedding it and inserting it into Milvus. The embedding function takes in text and returns the embeddings in a list format. 


```python
# Simple function that converts the texts to embeddings
def embed(texts):
    embeddings = openai.Embedding.create(
        input=texts,
        engine=OPENAI_ENGINE
    )
    return [x['embedding'] for x in embeddings['data']]

```

This next step does the actual inserting. Due to having so many datapoints, if you want to immidiately test it out you can stop the inserting cell block early and move along. Doing this will probably decrease the accuracy of the results due to less datapoints, but it should still be good enough. 


```python
from tqdm import tqdm

data = [
    [], # title
    [], # description
]

# Embed and insert in batches
for i in tqdm(range(0, len(dataset))):
    data[0].append(dataset[i]['title'])
    data[1].append(dataset[i]['description'])
    if len(data[0]) % BATCH_SIZE == 0:
        data.append(embed(data[1]))
        collection.insert(data)
        data = [[],[]]

# Embed and insert the remainder 
if len(data[0]) != 0:
    data.append(embed(data[1]))
    collection.insert(data)
    data = [[],[]]

```

      0%|          | 1999/1032335 [00:06<57:22, 299.31it/s]  



    ---------------------------------------------------------------------------

    KeyboardInterrupt                         Traceback (most recent call last)

    Cell In[18], line 13
         11 data[1].append(dataset[i]['description'])
         12 if len(data[0]) % BATCH_SIZE == 0:
    ---> 13     data.append(embed(data[1]))
         14     collection.insert(data)
         15     data = [[],[]]


key: _node_content
string_value:
{"id_": "1238080f-fc68-4f72-86de-5848cdbba1e0", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1a98da4a-702d-443c-a382-557fc0ab37b1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "f93ee46a07f19d85701ad559e08a5c7ba002a0d1a0efdaecad09ca357c8d4688", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "f9b72876-1b4a-4b5e-b8da-488ee722bc08", "node_type": "1", "metadata": {}, "hash": "d6ff75ce25211dd6029a9c3b99652e2a740920ea843d6467510bb82aa1daf864", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 23198, "end_char_idx": 25463, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
# Simple function that converts the texts to embeddings
def embed(texts):
    embeddings = openai.Embedding.create(
        input=texts,
        engine=OPENAI_ENGINE
    )
    return [x['embedding'] for x in embeddings['data']]

```

This next step does the actual inserting. Due to having so many datapoints, if you want to immidiately test it out you can stop the inserting cell block early and move along. Doing this will probably decrease the accuracy of the results due to less datapoints, but it should still be good enough. 


```python
from tqdm import tqdm

data = [
    [], # title
    [], # description
]

# Embed and insert in batches
for i in tqdm(range(0, len(dataset))):
    data[0].append(dataset[i]['title'])
    data[1].append(dataset[i]['description'])
    if len(data[0]) % BATCH_SIZE == 0:
        data.append(embed(data[1]))
        collection.insert(data)
        data = [[],[]]

# Embed and insert the remainder 
if len(data[0]) != 0:
    data.append(embed(data[1]))
    collection.insert(data)
    data = [[],[]]

```

      0%|          | 1999/1032335 [00:06<57:22, 299.31it/s]  



    ---------------------------------------------------------------------------

    KeyboardInterrupt                         Traceback (most recent call last)

    Cell In[18], line 13
         11 data[1].append(dataset[i]['description'])
         12 if len(data[0]) % BATCH_SIZE == 0:
    ---> 13     data.append(embed(data[1]))
         14     collection.insert(data)
         15     data = [[],[]]


    Cell In[17], line 3, in embed(texts)
          2 def embed(texts):
    ----> 3     embeddings = openai.Embedding.create(
          4         input=texts,
          5         engine=OPENAI_ENGINE
          6     )
          7     return [x['embedding'] for x in embeddings['data']]


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_resources/embedding.py:33, in Embedding.create(cls, *args, **kwargs)
         31 while True:
         32     try:
    ---> 33         response = super().create(*args, **kwargs)
         35         # If a user specifies base64, we'll just return the encoded string.
         36         # This is only for the default case.
         37         if not user_provided_encoding_format:


key: _node_content
string_value:
{"id_": "f9b72876-1b4a-4b5e-b8da-488ee722bc08", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1238080f-fc68-4f72-86de-5848cdbba1e0", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "3aed3aeb9c4346b8eac9f6064d9fbe60d0c474576c0effe62f028e2487981ec7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c6111576-3ed0-48ef-9da3-ee708ee5752a", "node_type": "1", "metadata": {}, "hash": "ced7d2c290925705047f42c8457ce4d95c177f15d40a2cbca5292a2c41486635", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 24321, "end_char_idx": 27437, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
---------------------------------------------------------------------------

    KeyboardInterrupt                         Traceback (most recent call last)

    Cell In[18], line 13
         11 data[1].append(dataset[i]['description'])
         12 if len(data[0]) % BATCH_SIZE == 0:
    ---> 13     data.append(embed(data[1]))
         14     collection.insert(data)
         15     data = [[],[]]


    Cell In[17], line 3, in embed(texts)
          2 def embed(texts):
    ----> 3     embeddings = openai.Embedding.create(
          4         input=texts,
          5         engine=OPENAI_ENGINE
          6     )
          7     return [x['embedding'] for x in embeddings['data']]


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_resources/embedding.py:33, in Embedding.create(cls, *args, **kwargs)
         31 while True:
         32     try:
    ---> 33         response = super().create(*args, **kwargs)
         35         # If a user specifies base64, we'll just return the encoded string.
         36         # This is only for the default case.
         37         if not user_provided_encoding_format:


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py:153, in EngineAPIResource.create(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)
        127 @classmethod
        128 def create(
        129     cls,
       (...)
        136     **params,
        137 ):
        138     (
        139         deployment_id,
        140         engine,
       (...)
        150         api_key, api_base, api_type, api_version, organization, **params
        151     )
    --> 153     response, _, api_key = requestor.request(
        154         "post",
        155         url,
        156         params=params,
        157         headers=headers,
        158         stream=stream,
        159         request_id=request_id,
        160         request_timeout=request_timeout,
        161     )
        163     if stream:
        164         # must be an iterator
        165         assert not isinstance(response, OpenAIResponse)


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_requestor.py:216, in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
        205 def request(
        206     self,
        207     method,
       (...)
        214     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
        215 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    --> 216     result = self.request_raw(
        217         method.lower(),
        218         url,
        219         params=params,
        220         supplied_headers=headers,
        221         files=files,
        222         stream=stream,
        223         request_id=request_id,
        224         request_timeout=request_timeout,
        225     )
        226     resp, got_stream = self._interpret_response(result, stream)
        227     return resp, got_stream, self.api_key


key: _node_content
string_value:
{"id_": "c6111576-3ed0-48ef-9da3-ee708ee5752a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "f9b72876-1b4a-4b5e-b8da-488ee722bc08", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "3de736c4e44152e84e0c3dca2e270f88ecbf06489c5bccce435ea8eaae0548bb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2f4c04bc-1003-45c5-924b-bf1b698b7943", "node_type": "1", "metadata": {}, "hash": "32c1ac354de1714a15cf950b771097a2137e20c99b78824acba78f4a1f3f4d83", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 26495, "end_char_idx": 29058, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_requestor.py:216, in APIRequestor.request(self, method, url, params, headers, files, stream, request_id, request_timeout)
        205 def request(
        206     self,
        207     method,
       (...)
        214     request_timeout: Optional[Union[float, Tuple[float, float]]] = None,
        215 ) -> Tuple[Union[OpenAIResponse, Iterator[OpenAIResponse]], bool, str]:
    --> 216     result = self.request_raw(
        217         method.lower(),
        218         url,
        219         params=params,
        220         supplied_headers=headers,
        221         files=files,
        222         stream=stream,
        223         request_id=request_id,
        224         request_timeout=request_timeout,
        225     )
        226     resp, got_stream = self._interpret_response(result, stream)
        227     return resp, got_stream, self.api_key


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_requestor.py:516, in APIRequestor.request_raw(self, method, url, params, supplied_headers, files, stream, request_id, request_timeout)
        514     _thread_context.session = _make_session()
        515 try:
    --> 516     result = _thread_context.session.request(
        517         method,
        518         abs_url,
        519         headers=headers,
        520         data=data,
        521         files=files,
        522         stream=stream,
        523         timeout=request_timeout if request_timeout else TIMEOUT_SECS,
        524     )
        525 except requests.exceptions.Timeout as e:
        526     raise error.Timeout("Request timed out: {}".format(e)) from e


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/sessions.py:587, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
        582 send_kwargs = {
        583     "timeout": timeout,
        584     "allow_redirects": allow_redirects,
        585 }
        586 send_kwargs.update(settings)
    --> 587 resp = self.send(prep, **send_kwargs)
        589 return resp


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/sessions.py:701, in Session.send(self, request, **kwargs)
        698 start = preferred_clock()
        700 # Send the request
    --> 701 r = adapter.send(request, **kwargs)
        703 # Total elapsed time of the request (approximately)
        704 elapsed = preferred_clock() - start


key: _node_content
string_value:
{"id_": "2f4c04bc-1003-45c5-924b-bf1b698b7943", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c6111576-3ed0-48ef-9da3-ee708ee5752a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "62c0094a00e8625ed86bad848696844321bcea3324fed88f65575f6cfe6a8cee", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9d0e1dbc-2464-4f27-bf73-17acc2c62445", "node_type": "1", "metadata": {}, "hash": "0f5f41412fbe5ff3d1983a3a4f67bb81332081a5e0d476b41cd5471547d7c3ee", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 27444, "end_char_idx": 29891, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/openai/api_requestor.py:516, in APIRequestor.request_raw(self, method, url, params, supplied_headers, files, stream, request_id, request_timeout)
        514     _thread_context.session = _make_session()
        515 try:
    --> 516     result = _thread_context.session.request(
        517         method,
        518         abs_url,
        519         headers=headers,
        520         data=data,
        521         files=files,
        522         stream=stream,
        523         timeout=request_timeout if request_timeout else TIMEOUT_SECS,
        524     )
        525 except requests.exceptions.Timeout as e:
        526     raise error.Timeout("Request timed out: {}".format(e)) from e


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/sessions.py:587, in Session.request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
        582 send_kwargs = {
        583     "timeout": timeout,
        584     "allow_redirects": allow_redirects,
        585 }
        586 send_kwargs.update(settings)
    --> 587 resp = self.send(prep, **send_kwargs)
        589 return resp


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/sessions.py:701, in Session.send(self, request, **kwargs)
        698 start = preferred_clock()
        700 # Send the request
    --> 701 r = adapter.send(request, **kwargs)
        703 # Total elapsed time of the request (approximately)
        704 elapsed = preferred_clock() - start


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/adapters.py:489, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
        487 try:
        488     if not chunked:
    --> 489         resp = conn.urlopen(
        490             method=request.method,
        491             url=url,
        492             body=request.body,
        493             headers=request.headers,
        494             redirect=False,
        495             assert_same_host=False,
        496             preload_content=False,
        497             decode_content=False,
        498             retries=self.max_retries,
        499             timeout=timeout,
        500         )
        502     # Send the request.
        503     else:
        504         if hasattr(conn, "proxy_pool"):


key: _node_content
string_value:
{"id_": "9d0e1dbc-2464-4f27-bf73-17acc2c62445", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2f4c04bc-1003-45c5-924b-bf1b698b7943", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "c72099acd4fbff3c80de864e973afd01a386cc6a99db9e8f9a7b219cb3956291", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d386bbca-edd3-443a-97c9-b99d8b20657c", "node_type": "1", "metadata": {}, "hash": "4c33a55a6575ff5ded176eeb4cc3934a59d14d4f93540625e72e1f42e6c0a3b0", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 28703, "end_char_idx": 31628, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/sessions.py:701, in Session.send(self, request, **kwargs)
        698 start = preferred_clock()
        700 # Send the request
    --> 701 r = adapter.send(request, **kwargs)
        703 # Total elapsed time of the request (approximately)
        704 elapsed = preferred_clock() - start


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/requests/adapters.py:489, in HTTPAdapter.send(self, request, stream, timeout, verify, cert, proxies)
        487 try:
        488     if not chunked:
    --> 489         resp = conn.urlopen(
        490             method=request.method,
        491             url=url,
        492             body=request.body,
        493             headers=request.headers,
        494             redirect=False,
        495             assert_same_host=False,
        496             preload_content=False,
        497             decode_content=False,
        498             retries=self.max_retries,
        499             timeout=timeout,
        500         )
        502     # Send the request.
        503     else:
        504         if hasattr(conn, "proxy_pool"):


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/urllib3/connectionpool.py:703, in HTTPConnectionPool.urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
        700     self._prepare_proxy(conn)
        702 # Make the request on the httplib connection object.
    --> 703 httplib_response = self._make_request(
        704     conn,
        705     method,
        706     url,
        707     timeout=timeout_obj,
        708     body=body,
        709     headers=headers,
        710     chunked=chunked,
        711 )
        713 # If we're going to release the connection in ``finally:``, then
        714 # the response doesn't need to know about the connection. Otherwise
        715 # it will also try to release it and we'll have a double-release
        716 # mess.
        717 response_conn = conn if not release_conn else None


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/urllib3/connectionpool.py:449, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
        444             httplib_response = conn.getresponse()
        445         except BaseException as e:
        446             # Remove the TypeError from the exception chain in
        447             # Python 3 (including for exceptions like SystemExit).
        448             # Otherwise it looks like a bug in the code.
    --> 449             six.raise_from(e, None)
        450 except (SocketTimeout, BaseSSLError, SocketError) as e:
        451     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)


    File <string>:3, in raise_from(value, from_value)


key: _node_content
string_value:
{"id_": "d386bbca-edd3-443a-97c9-b99d8b20657c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9d0e1dbc-2464-4f27-bf73-17acc2c62445", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "1f4c83096ae27d2fb38ca9e6b6b49ecd418e42a02687c9730e1f4cc90f75da87", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "adb7d01c-7e47-4234-9a08-225199189c58", "node_type": "1", "metadata": {}, "hash": "5b9413599f83f9778509110fc9e2752a98b353bf9474ce9d1b1d859e87c7b56c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 30853, "end_char_idx": 33404, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/urllib3/connectionpool.py:449, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
        444             httplib_response = conn.getresponse()
        445         except BaseException as e:
        446             # Remove the TypeError from the exception chain in
        447             # Python 3 (including for exceptions like SystemExit).
        448             # Otherwise it looks like a bug in the code.
    --> 449             six.raise_from(e, None)
        450 except (SocketTimeout, BaseSSLError, SocketError) as e:
        451     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)


    File <string>:3, in raise_from(value, from_value)


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/urllib3/connectionpool.py:444, in HTTPConnectionPool._make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
        441 except TypeError:
        442     # Python 3
        443     try:
    --> 444         httplib_response = conn.getresponse()
        445     except BaseException as e:
        446         # Remove the TypeError from the exception chain in
        447         # Python 3 (including for exceptions like SystemExit).
        448         # Otherwise it looks like a bug in the code.
        449         six.raise_from(e, None)


    File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:1377, in HTTPConnection.getresponse(self)
       1375 try:
       1376     try:
    -> 1377         response.begin()
       1378     except ConnectionError:
       1379         self.close()


    File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:320, in HTTPResponse.begin(self)
        318 # read until we get a non-100 response
        319 while True:
    --> 320     version, status, reason = self._read_status()
        321     if status != CONTINUE:
        322         break


    File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:281, in HTTPResponse._read_status(self)
        280 def _read_status(self):
    --> 281     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
        282     if len(line) > _MAXLINE:
        283         raise LineTooLong("status line")


    File ~/miniconda3/envs/haystack/lib/python3.9/socket.py:704, in SocketIO.readinto(self, b)
        702 while True:
        703     try:
    --> 704         return self._sock.recv_into(b)
        705     except timeout:
        706         self._timeout_occurred = True


key: _node_content
string_value:
{"id_": "adb7d01c-7e47-4234-9a08-225199189c58", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d386bbca-edd3-443a-97c9-b99d8b20657c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "09ecd8f113b9c3da0654820b963de9fe292584089e20c3185e823a212d611da3", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "613a6cfb-9f4c-4478-b3cd-9979af25de9e", "node_type": "1", "metadata": {}, "hash": "d4ca4518f0f25df700197279255ff974056451eb23c8b5bf14c92496b43590ee", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 32267, "end_char_idx": 35058, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:1377, in HTTPConnection.getresponse(self)
       1375 try:
       1376     try:
    -> 1377         response.begin()
       1378     except ConnectionError:
       1379         self.close()


    File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:320, in HTTPResponse.begin(self)
        318 # read until we get a non-100 response
        319 while True:
    --> 320     version, status, reason = self._read_status()
        321     if status != CONTINUE:
        322         break


    File ~/miniconda3/envs/haystack/lib/python3.9/http/client.py:281, in HTTPResponse._read_status(self)
        280 def _read_status(self):
    --> 281     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
        282     if len(line) > _MAXLINE:
        283         raise LineTooLong("status line")


    File ~/miniconda3/envs/haystack/lib/python3.9/socket.py:704, in SocketIO.readinto(self, b)
        702 while True:
        703     try:
    --> 704         return self._sock.recv_into(b)
        705     except timeout:
        706         self._timeout_occurred = True


    File ~/miniconda3/envs/haystack/lib/python3.9/ssl.py:1242, in SSLSocket.recv_into(self, buffer, nbytes, flags)
       1238     if flags != 0:
       1239         raise ValueError(
       1240           "non-zero flags not allowed in calls to recv_into() on %s" %
       1241           self.__class__)
    -> 1242     return self.read(nbytes, buffer)
       1243 else:
       1244     return super().recv_into(buffer, nbytes, flags)


    File ~/miniconda3/envs/haystack/lib/python3.9/ssl.py:1100, in SSLSocket.read(self, len, buffer)
       1098 try:
       1099     if buffer is not None:
    -> 1100         return self._sslobj.read(len, buffer)
       1101     else:
       1102         return self._sslobj.read(len)


    KeyboardInterrupt: 


## Query the Database
With our data safely inserted in Milvus, we can now perform a query. The query takes in a string or a list of strings and searches them. The resuts print out your provided description and the results that include the result score, the result title, and the result book description. 


```python
import textwrap

def query(queries, top_k = 5):
    if type(queries) != list:
        queries = [queries]
    res = collection.search(embed(queries), anns_field='embedding', param=QUERY_PARAM, limit = top_k, output_fields=['title', 'description'])
    for i, hit in enumerate(res):
        print('Description:', queries[i])
        print('Results:')
        for ii, hits in enumerate(hit):
            print('\t' + 'Rank:', ii + 1, 'Score:', hits.score, 'Title:', hits.entity.get('title'))
            print(textwrap.fill(hits.entity.get('description'), 88))
            print()
```


key: _node_content
string_value:
{"id_": "613a6cfb-9f4c-4478-b3cd-9979af25de9e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "adb7d01c-7e47-4234-9a08-225199189c58", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "e159f8b326b97c6c38c1395c0ada681408ed3b49860afb7f9748df16c68a7912", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "0ae86e5d-e49b-4ee5-b29b-312f59b58f48", "node_type": "1", "metadata": {}, "hash": "298e4ad1cca7c341fbdc4d4a9ebbec3851271fe8f097e0be9daa102dd65c4d05", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 33849, "end_char_idx": 36671, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/ssl.py:1100, in SSLSocket.read(self, len, buffer)
       1098 try:
       1099     if buffer is not None:
    -> 1100         return self._sslobj.read(len, buffer)
       1101     else:
       1102         return self._sslobj.read(len)


    KeyboardInterrupt: 


## Query the Database
With our data safely inserted in Milvus, we can now perform a query. The query takes in a string or a list of strings and searches them. The resuts print out your provided description and the results that include the result score, the result title, and the result book description. 


```python
import textwrap

def query(queries, top_k = 5):
    if type(queries) != list:
        queries = [queries]
    res = collection.search(embed(queries), anns_field='embedding', param=QUERY_PARAM, limit = top_k, output_fields=['title', 'description'])
    for i, hit in enumerate(res):
        print('Description:', queries[i])
        print('Results:')
        for ii, hits in enumerate(hit):
            print('\t' + 'Rank:', ii + 1, 'Score:', hits.score, 'Title:', hits.entity.get('title'))
            print(textwrap.fill(hits.entity.get('description'), 88))
            print()
```


```python
query('Book about a k-9 from europe')
```

    RPC error: [search], <MilvusException: (code=1, message=code: UnexpectedError, reason: code: CollectionNotExists, reason: can't find collection: book_search)>, <Time:{'RPC start': '2023-03-17 14:22:18.368461', 'RPC error': '2023-03-17 14:22:18.382086'}>



    ---------------------------------------------------------------------------

    MilvusException                           Traceback (most recent call last)

    Cell In[32], line 1
    ----> 1 query('Book about a k-9 from europe')


    Cell In[31], line 6, in query(queries, top_k)
          4 if type(queries) != list:
          5     queries = [queries]
    ----> 6 res = collection.search(embed(queries), anns_field='embedding', param=QUERY_PARAM, limit = top_k, output_fields=['title', 'description'])
          7 for i, hit in enumerate(res):
          8     print('Description:', queries[i])


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/orm/collection.py:614, in Collection.search(self, data, anns_field, param, limit, expr, partition_names, output_fields, timeout, round_decimal, **kwargs)
        611     raise DataTypeNotMatchException(message=ExceptionsMessage.ExprType % type(expr))
        613 conn = self._get_connection()
    --> 614 res = conn.search(self._name, data, anns_field, param, limit, expr,
        615                   partition_names, output_fields, round_decimal, timeout=timeout,
        616                   schema=self._schema_dict, **kwargs)
        617 if kwargs.get("_async", False):
        618     return SearchFuture(res)


key: _node_content
string_value:
{"id_": "0ae86e5d-e49b-4ee5-b29b-312f59b58f48", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "613a6cfb-9f4c-4478-b3cd-9979af25de9e", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "533b414558c74f563a3d04194b595215b1d930a9269312d4438d9721a4d0cc93", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "44b9b5e3-0c40-45c5-b238-d33a9b59c61a", "node_type": "1", "metadata": {}, "hash": "d45753293a9531e227d0e23d0a52f666557917e516b676cdc371ddc8416ed92b", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 35379, "end_char_idx": 38226, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
---------------------------------------------------------------------------

    MilvusException                           Traceback (most recent call last)

    Cell In[32], line 1
    ----> 1 query('Book about a k-9 from europe')


    Cell In[31], line 6, in query(queries, top_k)
          4 if type(queries) != list:
          5     queries = [queries]
    ----> 6 res = collection.search(embed(queries), anns_field='embedding', param=QUERY_PARAM, limit = top_k, output_fields=['title', 'description'])
          7 for i, hit in enumerate(res):
          8     print('Description:', queries[i])


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/orm/collection.py:614, in Collection.search(self, data, anns_field, param, limit, expr, partition_names, output_fields, timeout, round_decimal, **kwargs)
        611     raise DataTypeNotMatchException(message=ExceptionsMessage.ExprType % type(expr))
        613 conn = self._get_connection()
    --> 614 res = conn.search(self._name, data, anns_field, param, limit, expr,
        615                   partition_names, output_fields, round_decimal, timeout=timeout,
        616                   schema=self._schema_dict, **kwargs)
        617 if kwargs.get("_async", False):
        618     return SearchFuture(res)


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:109, in error_handler.<locals>.wrapper.<locals>.handler(*args, **kwargs)
        107     record_dict["RPC error"] = str(datetime.datetime.now())
        108     LOGGER.error(f"RPC error: [{inner_name}], {e}, <Time:{record_dict}>")
    --> 109     raise e
        110 except grpc.FutureTimeoutError as e:
        111     record_dict["gRPC timeout"] = str(datetime.datetime.now())


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:105, in error_handler.<locals>.wrapper.<locals>.handler(*args, **kwargs)
        103 try:
        104     record_dict["RPC start"] = str(datetime.datetime.now())
    --> 105     return func(*args, **kwargs)
        106 except MilvusException as e:
        107     record_dict["RPC error"] = str(datetime.datetime.now())


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:136, in tracing_request.<locals>.wrapper.<locals>.handler(self, *args, **kwargs)
        134 if req_id:
        135     self.set_onetime_request_id(req_id)
    --> 136 ret = func(self, *args, **kwargs)
        137 return ret


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:85, in retry_on_rpc_failure.<locals>.wrapper.<locals>.handler(self, *args, **kwargs)
         83         back_off = min(back_off * back_off_multiplier, max_back_off)
         84     else:
    ---> 85         raise e
         86 except Exception as e:
         87     raise e


key: _node_content
string_value:
{"id_": "44b9b5e3-0c40-45c5-b238-d33a9b59c61a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0ae86e5d-e49b-4ee5-b29b-312f59b58f48", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "64b28ca6d8851b18c09707ff44bba3d2b96ce446a680cd7e0527565faf121f55", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "066c7299-a04a-4b4f-9a00-e190fdb3d33d", "node_type": "1", "metadata": {}, "hash": "133309850c8fdd5b5181d4e47f2d5e41ee9080f6d2a40e4b79926de739e57461", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 37146, "end_char_idx": 39952, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:105, in error_handler.<locals>.wrapper.<locals>.handler(*args, **kwargs)
        103 try:
        104     record_dict["RPC start"] = str(datetime.datetime.now())
    --> 105     return func(*args, **kwargs)
        106 except MilvusException as e:
        107     record_dict["RPC error"] = str(datetime.datetime.now())


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:136, in tracing_request.<locals>.wrapper.<locals>.handler(self, *args, **kwargs)
        134 if req_id:
        135     self.set_onetime_request_id(req_id)
    --> 136 ret = func(self, *args, **kwargs)
        137 return ret


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:85, in retry_on_rpc_failure.<locals>.wrapper.<locals>.handler(self, *args, **kwargs)
         83         back_off = min(back_off * back_off_multiplier, max_back_off)
         84     else:
    ---> 85         raise e
         86 except Exception as e:
         87     raise e


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/decorators.py:50, in retry_on_rpc_failure.<locals>.wrapper.<locals>.handler(self, *args, **kwargs)
         48 while True:
         49     try:
    ---> 50         return func(self, *args, **kwargs)
         51     except grpc.RpcError as e:
         52         # DEADLINE_EXCEEDED means that the task wat not completed
         53         # UNAVAILABLE means that the service is not reachable currently
         54         # Reference: https://grpc.github.io/grpc/python/grpc.html#grpc-status-code
         55         if e.code() != grpc.StatusCode.DEADLINE_EXCEEDED and e.code() != grpc.StatusCode.UNAVAILABLE:


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/client/grpc_handler.py:472, in GrpcHandler.search(self, collection_name, data, anns_field, param, limit, expression, partition_names, output_fields, round_decimal, timeout, schema, **kwargs)
        467 requests = Prepare.search_requests_with_expr(collection_name, data, anns_field, param, limit, schema,
        468                                              expression, partition_names, output_fields, round_decimal,
        469                                              **kwargs)
        471 auto_id = schema["auto_id"]
    --> 472 return self._execute_search_requests(requests, timeout, round_decimal=round_decimal, auto_id=auto_id, **kwargs)


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/client/grpc_handler.py:441, in GrpcHandler._execute_search_requests(self, requests, timeout, **kwargs)
        439 if kwargs.get("_async", False):
        440     return SearchFuture(None, None, True, pre_err)
    --> 441 raise pre_err


key: _node_content
string_value:
{"id_": "066c7299-a04a-4b4f-9a00-e190fdb3d33d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7f3966ec-be9b-4c81-9cc2-3d3f3d4b532a", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "08c6a77c234553e2b0977ce8ec8638a156b231ccbdf1ea631386a5119a9268d4", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "44b9b5e3-0c40-45c5-b238-d33a9b59c61a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Getting_started_with_Milvus_and_OpenAI.ipynb", "title": "Getting Started with Milvus and OpenAI", "tokens": 11649, "retrieve_doc": false, "source": "openai_cookbooks"}, "hash": "067b55775ce39e9624222edb20fdb3e26584abb1ce5cc5a0fcc56213e6d21ad5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 38921, "end_char_idx": 40606, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/client/grpc_handler.py:472, in GrpcHandler.search(self, collection_name, data, anns_field, param, limit, expression, partition_names, output_fields, round_decimal, timeout, schema, **kwargs)
        467 requests = Prepare.search_requests_with_expr(collection_name, data, anns_field, param, limit, schema,
        468                                              expression, partition_names, output_fields, round_decimal,
        469                                              **kwargs)
        471 auto_id = schema["auto_id"]
    --> 472 return self._execute_search_requests(requests, timeout, round_decimal=round_decimal, auto_id=auto_id, **kwargs)


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/client/grpc_handler.py:441, in GrpcHandler._execute_search_requests(self, requests, timeout, **kwargs)
        439 if kwargs.get("_async", False):
        440     return SearchFuture(None, None, True, pre_err)
    --> 441 raise pre_err


    File ~/miniconda3/envs/haystack/lib/python3.9/site-packages/pymilvus/client/grpc_handler.py:432, in GrpcHandler._execute_search_requests(self, requests, timeout, **kwargs)
        429     response = self._stub.Search(request, timeout=timeout)
        431     if response.status.error_code != 0:
    --> 432         raise MilvusException(response.status.error_code, response.status.reason)
        434     raws.append(response)
        435 round_decimal = kwargs.get("round_decimal", -1)


    MilvusException: <MilvusException: (code=1, message=code: UnexpectedError, reason: code: CollectionNotExists, reason: can't find collection: book_search)>


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb


key: _node_content
string_value:
{"id_": "1782b7fa-8859-4b43-af51-e883ba1c90fb", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "d12aefcd-866b-4fdd-9883-10d2b54661d6", "node_type": "1", "metadata": {}, "hash": "86516a7d5b93f7a6e9d67edcf6290f226b59183d833778e2ded412058337ffba", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2678, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Filtered Search with Milvus and OpenAI
### Finding your next movie

In this notebook we will be going over generating embeddings of movie descriptions with OpenAI and using those embeddings within Milvus to find relevant movies. To narrow our search results and try something new, we are going to be using filtering to do metadata searches. The dataset in this example is sourced from HuggingFace datasets, and contains a little over 8 thousand movie entries.

Lets begin by first downloading the required libraries for this notebook:
- `openai` is used for communicating with the OpenAI embedding service
- `pymilvus` is used for communicating with the Milvus server
- `datasets` is used for downloading the dataset
- `tqdm` is used for the progress bars



```python
! pip install openai pymilvus datasets tqdm
```

With the required packages installed we can get started. Lets begin by launching the Milvus service. The file being run is the `docker-compose.yaml` found in the folder of this file. This command launches a Milvus standalone instance which we will use for this test.  


```python
! docker compose up -d
```

    E0317 14:06:38.344884000 140704629352640 fork_posix.cc:76]             Other threads are currently calling into gRPC, skipping fork() handlers


    [1A[1B[0G[?25l[+] Running 1/0
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠋ Container milvus-etcd   Creating                                        0.0s
    [0m[37m ⠋ Container milvus-minio  Creating                                        0.0s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠙ Container milvus-etcd   Creating                                        0.1s
    [0m[37m ⠙ Container milvus-minio  Creating                                        0.1s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 2/3
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.2s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.2s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.3s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.


key: _node_content
string_value:
{"id_": "d12aefcd-866b-4fdd-9883-10d2b54661d6", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1782b7fa-8859-4b43-af51-e883ba1c90fb", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "88860f81230f09977a640970b35c10ea662969cbcf964f7eeba9bf036c8b396c", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c10690ee-62fe-42b2-bb8f-5f61bfee7875", "node_type": "1", "metadata": {}, "hash": "6ee8822d128cd4f0cc801be5c4b0c23187981e9cc50572db695d408933e7e6ec", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1495, "end_char_idx": 3776, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
0s
    [0m[37m ⠋ Container milvus-minio  Creating                                        0.0s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 1/3
    [34m ⠿ Network milvus          Created                                         0.1s
    [0m[37m ⠙ Container milvus-etcd   Creating                                        0.1s
    [0m[37m ⠙ Container milvus-minio  Creating                                        0.1s
    [0m[?25h[1A[1A[1A[1A[0G[?25l[+] Running 2/3
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.2s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.2s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.3s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.4s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.4s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.5s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.5s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.


key: _node_content
string_value:
{"id_": "c10690ee-62fe-42b2-bb8f-5f61bfee7875", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "d12aefcd-866b-4fdd-9883-10d2b54661d6", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "e55f29224e6ebacba563fe36eaaf9f7631af269ef3c6463a47cda997e7491bef", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "2fd0c561-3396-4786-a06a-d4d1665dfd01", "node_type": "1", "metadata": {}, "hash": "666e14a0e55239bb9929a0cf095e07f7bd8bd9021f598f3f27ea8b745eb8d2e8", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2585, "end_char_idx": 4911, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
3s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.3s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.4s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.4s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.5s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.5s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.6s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.6s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.7s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.8s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.8s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.


key: _node_content
string_value:
{"id_": "2fd0c561-3396-4786-a06a-d4d1665dfd01", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c10690ee-62fe-42b2-bb8f-5f61bfee7875", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0bbca253c1dacc281888ea5ca2358144c33ee6842b9a82c0f6a0b2d4086c2fce", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9cd74544-75b0-4a3f-a35b-7a2765dd5b0c", "node_type": "1", "metadata": {}, "hash": "b8e9ac503c15690ba2dea275b76df8011e547e3f506bf86ee41d022e9af46c11", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3776, "end_char_idx": 6102, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.6s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.6s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.7s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.7s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.8s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.8s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.9s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.9s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.0s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.


key: _node_content
string_value:
{"id_": "9cd74544-75b0-4a3f-a35b-7a2765dd5b0c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "2fd0c561-3396-4786-a06a-d4d1665dfd01", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d8e77e96763068d63d101e90cd29428e2605862b481fa478df634938252540f8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "4180aff4-2ef8-4b4a-be1c-ca00ff5f2206", "node_type": "1", "metadata": {}, "hash": "190267ab6f073e9a34ea5fa89b907f83beddaa4bb4d6f9aca735d37d1c71e6a9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4911, "end_char_idx": 7200, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 2/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[37m ⠿ Container milvus-etcd        Starting                                   0.9s
    [0m[37m ⠿ Container milvus-minio       Starting                                   0.9s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[37m ⠿ Container milvus-minio       Starting                                   1.0s
    [0m[34m ⠿ Container milvus-standalone  Created                                    0.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.0s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.


key: _node_content
string_value:
{"id_": "4180aff4-2ef8-4b4a-be1c-ca00ff5f2206", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "9cd74544-75b0-4a3f-a35b-7a2765dd5b0c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "3dfe5d77b391cc79acf3bc35f90c2bcd27ec4ef0d046b9be4448154b747a636a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b23de2d1-7bb7-4669-82a4-c27ca3b022e6", "node_type": "1", "metadata": {}, "hash": "fdcdd1e85c5a400cad6b40a7ec9dc0d90b1289f13a2bc1894b38681ea73374a3", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6009, "end_char_idx": 8335, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.0s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.1s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.2s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.4s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.


key: _node_content
string_value:
{"id_": "b23de2d1-7bb7-4669-82a4-c27ca3b022e6", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "4180aff4-2ef8-4b4a-be1c-ca00ff5f2206", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "8ab9d5bc3af04eca8fbc3e80066bcb6a3e6b3bda074d91eca8e5b620e952c3bb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "46ece4ac-2535-4481-9454-ac29b19893d2", "node_type": "1", "metadata": {}, "hash": "7ac28e738edfe6461edc29d6ce5a536fd152346765ef84965227172bb262b600", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7200, "end_char_idx": 9650, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.3s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.4s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[+] Running 3/4
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[37m ⠿ Container milvus-standalone  Starting                                   1.5s
    [0m[?25h[1A[1A[1A[1A[1A[0G[?25l[34m[+] Running 4/4[0m
    [34m ⠿ Network milvus               Created                                    0.1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[34m ⠿ Container milvus-standalone  Started                                    1.6s
    [0m[?25h

With Milvus running we can setup our global variables:
- HOST: The Milvus host address
- PORT: The Milvus port number
- COLLECTION_NAME: What to name the collection within Milvus
- DIMENSION: The dimension of the embeddings
- OPENAI_ENGINE: Which embedding model to use
- openai.api_key: Your OpenAI account key
- INDEX_PARAM: The index settings to use for the collection
- QUERY_PARAM: The search parameters to use
- BATCH_SIZE: How many movies to embed and insert at once


```python
import openai

HOST = 'localhost'
PORT = 19530
COLLECTION_NAME = 'movie_search'
DIMENSION = 1536
OPENAI_ENGINE = 'text-embedding-3-small'
openai.api_key = 'sk-your_key'

INDEX_PARAM = {
    'metric_type':'L2',
    'index_type':"HNSW",
    'params':{'M': 8, 'efConstruction': 64}
}

QUERY_PARAM = {
    "metric_type": "L2",
    "params": {"ef": 64},
}

BATCH_SIZE = 1000
```


key: _node_content
string_value:
{"id_": "46ece4ac-2535-4481-9454-ac29b19893d2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b23de2d1-7bb7-4669-82a4-c27ca3b022e6", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "f5e049390835c4a144a26d597a79b0fafe367028f950129e56496f4d85cdd9b7", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "36dc0e90-55c9-42fd-8a95-c2e18974c71d", "node_type": "1", "metadata": {}, "hash": "a02b78b0677e812ae3ac35b8fe93c96739b309b54ff95eb10161b74fb3d74ab2", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 8493, "end_char_idx": 11269, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1s
    [0m[34m ⠿ Container milvus-etcd        Started                                    0.9s
    [0m[34m ⠿ Container milvus-minio       Started                                    1.0s
    [0m[34m ⠿ Container milvus-standalone  Started                                    1.6s
    [0m[?25h

With Milvus running we can setup our global variables:
- HOST: The Milvus host address
- PORT: The Milvus port number
- COLLECTION_NAME: What to name the collection within Milvus
- DIMENSION: The dimension of the embeddings
- OPENAI_ENGINE: Which embedding model to use
- openai.api_key: Your OpenAI account key
- INDEX_PARAM: The index settings to use for the collection
- QUERY_PARAM: The search parameters to use
- BATCH_SIZE: How many movies to embed and insert at once


```python
import openai

HOST = 'localhost'
PORT = 19530
COLLECTION_NAME = 'movie_search'
DIMENSION = 1536
OPENAI_ENGINE = 'text-embedding-3-small'
openai.api_key = 'sk-your_key'

INDEX_PARAM = {
    'metric_type':'L2',
    'index_type':"HNSW",
    'params':{'M': 8, 'efConstruction': 64}
}

QUERY_PARAM = {
    "metric_type": "L2",
    "params": {"ef": 64},
}

BATCH_SIZE = 1000
```


```python
from pymilvus import connections, utility, FieldSchema, Collection, CollectionSchema, DataType

# Connect to Milvus Database
connections.connect(host=HOST, port=PORT)
```


```python
# Remove collection if it already exists
if utility.has_collection(COLLECTION_NAME):
    utility.drop_collection(COLLECTION_NAME)
```


```python
# Create collection which includes the id, title, and embedding.
fields = [
    FieldSchema(name='id', dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name='title', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='type', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='release_year', dtype=DataType.INT64),
    FieldSchema(name='rating', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='description', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, dim=DIMENSION)
]
schema = CollectionSchema(fields=fields)
collection = Collection(name=COLLECTION_NAME, schema=schema)
```


```python
# Create the index on the collection and load it.
collection.create_index(field_name="embedding", index_params=INDEX_PARAM)
collection.load()
```

## Dataset
With Milvus up and running we can begin grabbing our data. Hugging Face Datasets is a hub that holds many different user datasets, and for this example we are using HuggingLearners's netflix-shows dataset. This dataset contains movies and their metadata pairs for over 8 thousand movies. We are going to embed each description and store it within Milvus along with its title, type, release_year and rating.


key: _node_content
string_value:
{"id_": "36dc0e90-55c9-42fd-8a95-c2e18974c71d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "46ece4ac-2535-4481-9454-ac29b19893d2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "779ef2c79d3c521ff9188d3e77245fa62c2bf5d3bf41eec9c8571aa9e98fbdae", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "27449da9-1c18-4682-b39d-9686dff215c2", "node_type": "1", "metadata": {}, "hash": "5fe12854d2246871beed11de7581b95fabcd2c042a86d6c481ebf9d15adb9815", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9653, "end_char_idx": 12290, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
from pymilvus import connections, utility, FieldSchema, Collection, CollectionSchema, DataType

# Connect to Milvus Database
connections.connect(host=HOST, port=PORT)
```


```python
# Remove collection if it already exists
if utility.has_collection(COLLECTION_NAME):
    utility.drop_collection(COLLECTION_NAME)
```


```python
# Create collection which includes the id, title, and embedding.
fields = [
    FieldSchema(name='id', dtype=DataType.INT64, is_primary=True, auto_id=True),
    FieldSchema(name='title', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='type', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='release_year', dtype=DataType.INT64),
    FieldSchema(name='rating', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='description', dtype=DataType.VARCHAR, max_length=64000),
    FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, dim=DIMENSION)
]
schema = CollectionSchema(fields=fields)
collection = Collection(name=COLLECTION_NAME, schema=schema)
```


```python
# Create the index on the collection and load it.
collection.create_index(field_name="embedding", index_params=INDEX_PARAM)
collection.load()
```

## Dataset
With Milvus up and running we can begin grabbing our data. Hugging Face Datasets is a hub that holds many different user datasets, and for this example we are using HuggingLearners's netflix-shows dataset. This dataset contains movies and their metadata pairs for over 8 thousand movies. We are going to embed each description and store it within Milvus along with its title, type, release_year and rating.


```python
import datasets

# Download the dataset 
dataset = datasets.load_dataset('hugginglearners/netflix-shows', split='train')
```

    Found cached dataset csv (/Users/filiphaltmayer/.cache/huggingface/datasets/hugginglearners___csv/hugginglearners--netflix-shows-03475319fc65a05a/0.0.0/6b34fb8fcf56f7c8ba51dc895bfa2bfbe43546f190a60fcf74bb5e8afdcc2317)


## Insert the Data
Now that we have our data on our machine we can begin embedding it and inserting it into Milvus. The embedding function takes in text and returns the embeddings in a list format. 


```python
# Simple function that converts the texts to embeddings
def embed(texts):
    embeddings = openai.Embedding.create(
        input=texts,
        engine=OPENAI_ENGINE
    )
    return [x['embedding'] for x in embeddings['data']]

```

This next step does the actual inserting. We iterate through all the entries and create batches that we insert once we hit our set batch size. After the loop is over we insert the last remaning batch if it exists.


key: _node_content
string_value:
{"id_": "27449da9-1c18-4682-b39d-9686dff215c2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "36dc0e90-55c9-42fd-8a95-c2e18974c71d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "549f3f14f7aa114435e7d69953d58a951c523588f09605aefdcff8bee89533fb", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e9c76286-6c32-415c-98fd-4d01d51b215b", "node_type": "1", "metadata": {}, "hash": "23ef84f74f416ff28903e29cfd7209278c0bfef0ecb7cfd45011253b66cae88f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 10695, "end_char_idx": 13550, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
# Create the index on the collection and load it.
collection.create_index(field_name="embedding", index_params=INDEX_PARAM)
collection.load()
```

## Dataset
With Milvus up and running we can begin grabbing our data. Hugging Face Datasets is a hub that holds many different user datasets, and for this example we are using HuggingLearners's netflix-shows dataset. This dataset contains movies and their metadata pairs for over 8 thousand movies. We are going to embed each description and store it within Milvus along with its title, type, release_year and rating.


```python
import datasets

# Download the dataset 
dataset = datasets.load_dataset('hugginglearners/netflix-shows', split='train')
```

    Found cached dataset csv (/Users/filiphaltmayer/.cache/huggingface/datasets/hugginglearners___csv/hugginglearners--netflix-shows-03475319fc65a05a/0.0.0/6b34fb8fcf56f7c8ba51dc895bfa2bfbe43546f190a60fcf74bb5e8afdcc2317)


## Insert the Data
Now that we have our data on our machine we can begin embedding it and inserting it into Milvus. The embedding function takes in text and returns the embeddings in a list format. 


```python
# Simple function that converts the texts to embeddings
def embed(texts):
    embeddings = openai.Embedding.create(
        input=texts,
        engine=OPENAI_ENGINE
    )
    return [x['embedding'] for x in embeddings['data']]

```

This next step does the actual inserting. We iterate through all the entries and create batches that we insert once we hit our set batch size. After the loop is over we insert the last remaning batch if it exists. 


```python
from tqdm import tqdm

data = [
    [], # title
    [], # type
    [], # release_year
    [], # rating
    [], # description
]

# Embed and insert in batches
for i in tqdm(range(0, len(dataset))):
    data[0].append(dataset[i]['title'] or '')
    data[1].append(dataset[i]['type'] or '')
    data[2].append(dataset[i]['release_year'] or -1)
    data[3].append(dataset[i]['rating'] or '')
    data[4].append(dataset[i]['description'] or '')
    if len(data[0]) % BATCH_SIZE == 0:
        data.append(embed(data[4]))
        collection.insert(data)
        data = [[],[],[],[],[]]

# Embed and insert the remainder 
if len(data[0]) != 0:
    data.append(embed(data[4]))
    collection.insert(data)
    data = [[],[],[],[],[]]

```

    100%|██████████| 8807/8807 [00:31<00:00, 276.82it/s]


## Query the Database
With our data safely inserted in Milvus, we can now perform a query. The query takes in a tuple of the movie description you are searching for an the filter to use. More info about the filter can be found [here](https://milvus.io/docs/boolean.md). The search first prints out your description and filter expression. After that for each result we print the score, title, type, release year, rating, and description of the result movies.


key: _node_content
string_value:
{"id_": "e9c76286-6c32-415c-98fd-4d01d51b215b", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "056f7cdd-2c42-4c80-a90e-2e070d777fea", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "b1fa81ddbfee65f19b82c729d479c7525a55e4d249536b13a4536660d026db49", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "27449da9-1c18-4682-b39d-9686dff215c2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/milvus/Filtered_search_with_Milvus_and_OpenAI.ipynb", "title": "Filtered Search with Milvus and OpenAI", "tokens": 4573, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "588d152d2157aa18f6c28bd6b13dcf59e63852d58ebfc48becdfbdb0a3906e0d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 12294, "end_char_idx": 15877, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
from tqdm import tqdm

data = [
    [], # title
    [], # type
    [], # release_year
    [], # rating
    [], # description
]

# Embed and insert in batches
for i in tqdm(range(0, len(dataset))):
    data[0].append(dataset[i]['title'] or '')
    data[1].append(dataset[i]['type'] or '')
    data[2].append(dataset[i]['release_year'] or -1)
    data[3].append(dataset[i]['rating'] or '')
    data[4].append(dataset[i]['description'] or '')
    if len(data[0]) % BATCH_SIZE == 0:
        data.append(embed(data[4]))
        collection.insert(data)
        data = [[],[],[],[],[]]

# Embed and insert the remainder 
if len(data[0]) != 0:
    data.append(embed(data[4]))
    collection.insert(data)
    data = [[],[],[],[],[]]

```

    100%|██████████| 8807/8807 [00:31<00:00, 276.82it/s]


## Query the Database
With our data safely inserted in Milvus, we can now perform a query. The query takes in a tuple of the movie description you are searching for an the filter to use. More info about the filter can be found [here](https://milvus.io/docs/boolean.md). The search first prints out your description and filter expression. After that for each result we print the score, title, type, release year, rating, and description of the result movies. 


```python
import textwrap

def query(query, top_k = 5):
    text, expr = query
    res = collection.search(embed(text), anns_field='embedding', expr = expr, param=QUERY_PARAM, limit = top_k, output_fields=['title', 'type', 'release_year', 'rating', 'description'])
    for i, hit in enumerate(res):
        print('Description:', text, 'Expression:', expr)
        print('Results:')
        for ii, hits in enumerate(hit):
            print('\t' + 'Rank:', ii + 1, 'Score:', hits.score, 'Title:', hits.entity.get('title'))
            print('\t\t' + 'Type:', hits.entity.get('type'), 'Release Year:', hits.entity.get('release_year'), 'Rating:', hits.entity.get('rating'))
            print(textwrap.fill(hits.entity.get('description'), 88))
            print()

my_query = ('movie about a fluffly animal', 'release_year < 2019 and rating like \"PG%\"')

query(my_query)
```

    Description: movie about a fluffly animal Expression: release_year < 2019 and rating like "PG%"
    Results:
    	Rank: 1 Score: 0.30083978176116943 Title: The Lamb
    		Type: Movie Release Year: 2017 Rating: PG
    A big-dreaming donkey escapes his menial existence and befriends some free-spirited
    animal pals in this imaginative retelling of the Nativity Story.
    
    	Rank: 2 Score: 0.33528298139572144 Title: Puss in Boots
    		Type: Movie Release Year: 2011 Rating: PG
    The fabled feline heads to the Land of Giants with friends Humpty Dumpty and Kitty
    Softpaws on a quest to nab its greatest treasure: the Golden Goose.
    
    	Rank: 3 Score: 0.33528298139572144 Title: Puss in Boots
    		Type: Movie Release Year: 2011 Rating: PG
    The fabled feline heads to the Land of Giants with friends Humpty Dumpty and Kitty
    Softpaws on a quest to nab its greatest treasure: the Golden Goose.
    
    	Rank: 4 Score: 0.3414868116378784 Title: Show Dogs
    		Type: Movie Release Year: 2018 Rating: PG
    A rough and tough police dog must go undercover with an FBI agent as a prim and proper
    pet at a dog show to save a baby panda from an illegal sale.
    
    	Rank: 5 Score: 0.3414868116378784 Title: Show Dogs
    		Type: Movie Release Year: 2018 Rating: PG
    A rough and tough police dog must go undercover with an FBI agent as a prim and proper
    pet at a dog show to save a baby panda from an illegal sale.


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb


key: _node_content
string_value:
{"id_": "0f8bca00-0899-479b-9c69-b0b2df7f4c78", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "403de4ce-58aa-4e7f-9cee-9d74f3ccf644", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "436edb368979905aebf7d00f3f021a82934ad92a7e8acc7b5ffdec37c46878ca", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "13d1d528-d981-4e36-b104-5eb8871373de", "node_type": "1", "metadata": {}, "hash": "e0fc29ee1885346a43a6a917cf0c0b00f9f7df9a4569475e34de187648c9fc37", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3242, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# Using AnalyticDB as a vector database for OpenAI embeddings

This notebook guides you step by step on using AnalyticDB as a vector database for OpenAI embeddings.

This notebook presents an end-to-end process of:
1. Using precomputed embeddings created by OpenAI API.
2. Storing the embeddings in a cloud instance of AnalyticDB.
3. Converting raw text query to an embedding with OpenAI API.
4. Using AnalyticDB to perform the nearest neighbour search in the created collection.

### What is AnalyticDB

[AnalyticDB](https://www.alibabacloud.com/help/en/analyticdb-for-postgresql/latest/product-introduction-overview) is a high-performance distributed vector database. Fully compatible with PostgreSQL syntax, you can effortlessly utilize it. AnalyticDB is Alibaba Cloud managed cloud-native database with strong-performed vector compute engine. Absolute out-of-box experience allow to scale into billions of data vectors processing with rich features including indexing algorithms, structured & non-structured data features, realtime update, distance metrics, scalar filtering, time travel searches etc. Also equipped with full OLAP database functionality and SLA commitment for production usage promise;

### Deployment options

- Using [AnalyticDB Cloud Vector Database](https://www.alibabacloud.com/help/zh/analyticdb-for-postgresql/latest/overview-2). [Click here](https://www.alibabacloud.com/product/hybriddb-postgresql) to fast deploy it.


## Prerequisites

For the purposes of this exercise we need to prepare a couple of things:

1. AnalyticDB cloud server instance.
2. The 'psycopg2' library to interact with the vector database. Any other postgresql client library is ok.
3. An [OpenAI API key](https://beta.openai.com/account/api-keys).



We might validate if the server was launched successfully by running a simple curl command:


### Install requirements

This notebook obviously requires the `openai` and `psycopg2` packages, but there are also some other additional libraries we will use. The following command installs them all:



```python
! pip install openai psycopg2 pandas wget
```

### Prepare your OpenAI API key

The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://beta.openai.com/account/api-keys](https://beta.openai.com/account/api-keys).

Once you get your key, please add it to your environment variables as `OPENAI_API_KEY`.


```python
# Test that your OpenAI API key is correctly set as an environment variable
# Note. if you run this notebook locally, you will need to reload your terminal and the notebook for the env variables to be live.
import os

# Note. alternatively you can set a temporary env variable like this:
# os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

if os.getenv("OPENAI_API_KEY") is not None:
    print("OPENAI_API_KEY is ready")
else:
    print("OPENAI_API_KEY environment variable not found")
```

    OPENAI_API_KEY is ready


## Connect to AnalyticDB
First add it to your environment variables. or you can just change the "psycopg2.connect" parameters below

Connecting to a running instance of AnalyticDB server is easy with the official Python library:


key: _node_content
string_value:
{"id_": "13d1d528-d981-4e36-b104-5eb8871373de", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "403de4ce-58aa-4e7f-9cee-9d74f3ccf644", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "436edb368979905aebf7d00f3f021a82934ad92a7e8acc7b5ffdec37c46878ca", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0f8bca00-0899-479b-9c69-b0b2df7f4c78", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "1c9ea2a49d50eb3868f2b090fc3a9f46c1341732ca7fff60619de469b862b08a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "0305ddf1-8a8c-4fce-a239-e2835db3656c", "node_type": "1", "metadata": {}, "hash": "3570b5a49d48e59a70f6615599e5691f6ce91bf0c05055c5336890b4508759a5", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1755, "end_char_idx": 4986, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
We might validate if the server was launched successfully by running a simple curl command:


### Install requirements

This notebook obviously requires the `openai` and `psycopg2` packages, but there are also some other additional libraries we will use. The following command installs them all:



```python
! pip install openai psycopg2 pandas wget
```

### Prepare your OpenAI API key

The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from [https://beta.openai.com/account/api-keys](https://beta.openai.com/account/api-keys).

Once you get your key, please add it to your environment variables as `OPENAI_API_KEY`.


```python
# Test that your OpenAI API key is correctly set as an environment variable
# Note. if you run this notebook locally, you will need to reload your terminal and the notebook for the env variables to be live.
import os

# Note. alternatively you can set a temporary env variable like this:
# os.environ["OPENAI_API_KEY"] = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

if os.getenv("OPENAI_API_KEY") is not None:
    print("OPENAI_API_KEY is ready")
else:
    print("OPENAI_API_KEY environment variable not found")
```

    OPENAI_API_KEY is ready


## Connect to AnalyticDB
First add it to your environment variables. or you can just change the "psycopg2.connect" parameters below

Connecting to a running instance of AnalyticDB server is easy with the official Python library:


```python
import os
import psycopg2

# Note. alternatively you can set a temporary env variable like this:
# os.environ["PGHOST"] = "your_host"
# os.environ["PGPORT"] "5432"),
# os.environ["PGDATABASE"] "postgres"),
# os.environ["PGUSER"] "user"),
# os.environ["PGPASSWORD"] "password"),

connection = psycopg2.connect(
    host=os.environ.get("PGHOST", "localhost"),
    port=os.environ.get("PGPORT", "5432"),
    database=os.environ.get("PGDATABASE", "postgres"),
    user=os.environ.get("PGUSER", "user"),
    password=os.environ.get("PGPASSWORD", "password")
)

# Create a new cursor object
cursor = connection.cursor()
```

We can test the connection by running any available method:


```python

# Execute a simple query to test the connection
cursor.execute("SELECT 1;")
result = cursor.fetchone()

# Check the query result
if result == (1,):
    print("Connection successful!")
else:
    print("Connection failed.")
```

    Connection successful!



```python
import wget

embeddings_url = "https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip"

# The file is ~700 MB so this will take some time
wget.download(embeddings_url)
```

    100% [......................................................................] 698933052 / 698933052




    'vector_database_wikipedia_articles_embedded.zip'



The downloaded file has to be then extracted:


```python
import zipfile
import os
import re
import tempfile

current_directory = os.getcwd()
zip_file_path = os.path.join(current_directory, "vector_database_wikipedia_articles_embedded.zip")
output_directory = os.path.join(current_directory, "../../data")

with zipfile.ZipFile(zip_file_path, "r") as zip_ref:
    zip_ref.extractall(output_directory)


key: _node_content
string_value:
{"id_": "0305ddf1-8a8c-4fce-a239-e2835db3656c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "403de4ce-58aa-4e7f-9cee-9d74f3ccf644", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "436edb368979905aebf7d00f3f021a82934ad92a7e8acc7b5ffdec37c46878ca", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "13d1d528-d981-4e36-b104-5eb8871373de", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "867eddc618c2249b0c2d5f21d65708ca32b6b662cd8652fe30e651174d0546d3", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a8918869-e54f-4e98-93ef-07cb093ebd32", "node_type": "1", "metadata": {}, "hash": "3e7d0d8e04992c4faaf2983ec85155b21b9c5c22e831931e647af48cfbd79534", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3245, "end_char_idx": 5855, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
import os
import psycopg2

# Note. alternatively you can set a temporary env variable like this:
# os.environ["PGHOST"] = "your_host"
# os.environ["PGPORT"] "5432"),
# os.environ["PGDATABASE"] "postgres"),
# os.environ["PGUSER"] "user"),
# os.environ["PGPASSWORD"] "password"),

connection = psycopg2.connect(
    host=os.environ.get("PGHOST", "localhost"),
    port=os.environ.get("PGPORT", "5432"),
    database=os.environ.get("PGDATABASE", "postgres"),
    user=os.environ.get("PGUSER", "user"),
    password=os.environ.get("PGPASSWORD", "password")
)

# Create a new cursor object
cursor = connection.cursor()
```

We can test the connection by running any available method:


```python

# Execute a simple query to test the connection
cursor.execute("SELECT 1;")
result = cursor.fetchone()

# Check the query result
if result == (1,):
    print("Connection successful!")
else:
    print("Connection failed.")
```

    Connection successful!



```python
import wget

embeddings_url = "https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip"

# The file is ~700 MB so this will take some time
wget.download(embeddings_url)
```

    100% [......................................................................] 698933052 / 698933052




    'vector_database_wikipedia_articles_embedded.zip'



The downloaded file has to be then extracted:


```python
import zipfile
import os
import re
import tempfile

current_directory = os.getcwd()
zip_file_path = os.path.join(current_directory, "vector_database_wikipedia_articles_embedded.zip")
output_directory = os.path.join(current_directory, "../../data")

with zipfile.ZipFile(zip_file_path, "r") as zip_ref:
    zip_ref.extractall(output_directory)


# check the csv file exist
file_name = "vector_database_wikipedia_articles_embedded.csv"
data_directory = os.path.join(current_directory, "../../data")
file_path = os.path.join(data_directory, file_name)


if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Index data

AnalyticDB stores data in __relation__ where each object is described by at least one vector. Our relation will be called **articles** and each object will be described by both **title** and **content** vectors. \

We will start with creating a relation and create a vector index on both **title** and **content**, and then we will fill it with our precomputed embeddings.


key: _node_content
string_value:
{"id_": "a8918869-e54f-4e98-93ef-07cb093ebd32", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "403de4ce-58aa-4e7f-9cee-9d74f3ccf644", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "436edb368979905aebf7d00f3f021a82934ad92a7e8acc7b5ffdec37c46878ca", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0305ddf1-8a8c-4fce-a239-e2835db3656c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "bef73a651edee0cbaa52454ce2d6913aa735fc07ecd30e9883dd591b64e89f81", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "bfdfe09b-8f6b-4696-9c0e-bbfeb001310e", "node_type": "1", "metadata": {}, "hash": "4f7cf2e1128bcc8126b3749e93e7c6695b7934ba520280c8022905333f3dc75a", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4204, "end_char_idx": 6855, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
import wget

embeddings_url = "https://cdn.openai.com/API/examples/data/vector_database_wikipedia_articles_embedded.zip"

# The file is ~700 MB so this will take some time
wget.download(embeddings_url)
```

    100% [......................................................................] 698933052 / 698933052




    'vector_database_wikipedia_articles_embedded.zip'



The downloaded file has to be then extracted:


```python
import zipfile
import os
import re
import tempfile

current_directory = os.getcwd()
zip_file_path = os.path.join(current_directory, "vector_database_wikipedia_articles_embedded.zip")
output_directory = os.path.join(current_directory, "../../data")

with zipfile.ZipFile(zip_file_path, "r") as zip_ref:
    zip_ref.extractall(output_directory)


# check the csv file exist
file_name = "vector_database_wikipedia_articles_embedded.csv"
data_directory = os.path.join(current_directory, "../../data")
file_path = os.path.join(data_directory, file_name)


if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Index data

AnalyticDB stores data in __relation__ where each object is described by at least one vector. Our relation will be called **articles** and each object will be described by both **title** and **content** vectors. \

We will start with creating a relation and create a vector index on both **title** and **content**, and then we will fill it with our precomputed embeddings.


```python
create_table_sql = '''
CREATE TABLE IF NOT EXISTS public.articles (
    id INTEGER NOT NULL,
    url TEXT,
    title TEXT,
    content TEXT,
    title_vector REAL[],
    content_vector REAL[],
    vector_id INTEGER
);

ALTER TABLE public.articles ADD PRIMARY KEY (id);
'''

# SQL statement for creating indexes
create_indexes_sql = '''
CREATE INDEX ON public.articles USING ann (content_vector) WITH (distancemeasure = l2, dim = '1536', pq_segments = '64', hnsw_m = '100', pq_centers = '2048');

CREATE INDEX ON public.articles USING ann (title_vector) WITH (distancemeasure = l2, dim = '1536', pq_segments = '64', hnsw_m = '100', pq_centers = '2048');
'''

# Execute the SQL statements
cursor.execute(create_table_sql)
cursor.execute(create_indexes_sql)

# Commit the changes
connection.commit()
```

## Load data

In this section we are going to load the data prepared previous to this session, so you don't have to recompute the embeddings of Wikipedia articles with your own credits.


key: _node_content
string_value:
{"id_": "bfdfe09b-8f6b-4696-9c0e-bbfeb001310e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "403de4ce-58aa-4e7f-9cee-9d74f3ccf644", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "436edb368979905aebf7d00f3f021a82934ad92a7e8acc7b5ffdec37c46878ca", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a8918869-e54f-4e98-93ef-07cb093ebd32", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/analyticdb/Getting_started_with_AnalyticDB_and_OpenAI.ipynb", "title": "Using AnalyticDB as a vector database for OpenAI embeddings", "tokens": 2780, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "dcdc5b4e8ebc30adf5f76a67448f62a878cb86a0755c49df215b6fd13a68f85f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9bfed376-64d7-4407-908d-6f5f163303c9", "node_type": "1", "metadata": {}, "hash": "380abebc31013c6f9ccb12dbb7852bfee634f461c2e211ca9b03e43e3db116ea", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5195, "end_char_idx": 8384, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
if os.path.exists(file_path):
    print(f"The file {file_name} exists in the data directory.")
else:
    print(f"The file {file_name} does not exist in the data directory.")

```

    The file vector_database_wikipedia_articles_embedded.csv exists in the data directory.


## Index data

AnalyticDB stores data in __relation__ where each object is described by at least one vector. Our relation will be called **articles** and each object will be described by both **title** and **content** vectors. \

We will start with creating a relation and create a vector index on both **title** and **content**, and then we will fill it with our precomputed embeddings.


```python
create_table_sql = '''
CREATE TABLE IF NOT EXISTS public.articles (
    id INTEGER NOT NULL,
    url TEXT,
    title TEXT,
    content TEXT,
    title_vector REAL[],
    content_vector REAL[],
    vector_id INTEGER
);

ALTER TABLE public.articles ADD PRIMARY KEY (id);
'''

# SQL statement for creating indexes
create_indexes_sql = '''
CREATE INDEX ON public.articles USING ann (content_vector) WITH (distancemeasure = l2, dim = '1536', pq_segments = '64', hnsw_m = '100', pq_centers = '2048');

CREATE INDEX ON public.articles USING ann (title_vector) WITH (distancemeasure = l2, dim = '1536', pq_segments = '64', hnsw_m = '100', pq_centers = '2048');
'''

# Execute the SQL statements
cursor.execute(create_table_sql)
cursor.execute(create_indexes_sql)

# Commit the changes
connection.commit()
```

## Load data

In this section we are going to load the data prepared previous to this session, so you don't have to recompute the embeddings of Wikipedia articles with your own credits.


```python
import io

# Path to your local CSV file
csv_file_path = '../../data/vector_database_wikipedia_articles_embedded.csv'

# Define a generator function to process the file line by line
def process_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            # Replace '[' with '{' and ']' with '}'
            modified_line = line.replace('[', '{').replace(']', '}')
            yield modified_line

# Create a StringIO object to store the modified lines
modified_lines = io.StringIO(''.join(list(process_file(csv_file_path))))

# Create the COPY command for the copy_expert method
copy_command = '''
COPY public.articles (id, url, title, content, title_vector, content_vector, vector_id)
FROM STDIN WITH (FORMAT CSV, HEADER true, DELIMITER ',');
'''

# Execute the COPY command using the copy_expert method
cursor.copy_expert(copy_command, modified_lines)

# Commit the changes
connection.commit()
```


```python
# Check the collection size to make sure all the points have been stored
count_sql = """select count(*) from public.articles;"""
cursor.execute(count_sql)
result = cursor.fetchone()
print(f"Count:{result[0]}")

```

    Count:25000


## Search data

Once the data is put into Qdrant we will start querying the collection for the closest vectors. We may provide an additional parameter `vector_name` to switch from title to content based search. Since the precomputed embeddings were created with `text-embedding-3-small` OpenAI model we also have to use it during search.


