# embeddings_queue

created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 00e5c312-3943-4aff-8300-ab467bc897c6
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"00e5c312-3943-4aff-8300-ab467bc897c6\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8e0307fe-8f30-4171-acbf-72606296d677\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"3bc17dcc0e304ed7f6d31c04e1ddbf51ad621766f6e3768afcd67ad1c0ac164a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c400734f-7bec-4840-9305-280811c40d7e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e54dd7e71cf42f3fd9da720f616d855d4058a7f45248ca559200c5d92389c56d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11848, \"end_char_idx\": 14767, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "Considering only spectroscopically confirmed galaxies, we find our sample of UV-faint LAEs (MUV\u2273\u221220mag) and their direct neighbours are generally not able to produce the required ionised regions based on the Lyman-\u03b1 transmission properties, suggesting lower-luminosity sources likely play an important role in carving out these bubbles. These observations demonstrate the combined power of JWST multi-object and slitless spectroscopy in acquiring a unique view of the early stages of Cosmic Reionisation via the most distant LAEs.\"\"\"\nRelevant: No\n\nQuery: {query}\nDocument: \"\"\"{document}\"\"\"\nRelevant:\n'''\n\n\n@retry(wait=wait_random_exponential(min=1, max=40), stop=stop_after_attempt(3))\ndef document_relevance(query, document):\n    response = openai.chat.completions.create(\n        model=\"text-davinci-003\",\n        message=prompt.format(query=query, document=document),\n        temperature=0,\n        logprobs=True,\n        logit_bias={3363: 1, 1400: 1},\n    )\n\n    return (\n        query,\n        document,\n        response.choices[0].message.content,\n        response.choices[0].logprobs.token_logprobs[0],\n    )\n```\n\n\n```python\ncontent = result_list[0][\"title\"] + \": \" + result_list[0][\"summary\"]\n\n# Set logprobs to 1 so our response will include the most probable token the model identified\nresponse = openai.chat.completions.create(\n    model=OPENAI_MODEL,\n    prompt=prompt.format(query=query, document=content),\n    temperature=0,\n    logprobs=1,\n    logit_bias={3363: 1, 1400: 1},\n    max_tokens=1,\n)\n```\n\n\n```python\nresult = response.choices[0]\nprint(f\"Result was {result.message.content}\")\nprint(f\"Logprobs was {result.logprobs.token_logprobs[0]}\")\nprint(\"\\nBelow is the full logprobs object\\n\\n\")\nprint(result[\"logprobs\"])\n```\n\n    Result was Yes\n    Logprobs was -0.05869877\n    \n    Below is the full logprobs object\n    \n    \n    {\n      \"tokens\": [\n        \"Yes\"\n      ],\n      \"token_logprobs\": [\n        -0.05869877\n      ],\n      \"top_logprobs\": [\n        {\n          \"Yes\": -0.05869877\n        }\n      ],\n      \"text_offset\": [\n        5764\n      ]\n    }\n\n\n\n```python\noutput_list = []\nfor x in result_list:\n    content = x[\"title\"] + \": \" + x[\"summary\"]\n\n    try:\n        output_list.append(document_relevance(query, document=content))\n\n    except Exception as e:\n        print(e)\n```\n\n\n```python\noutput_list[:10]\n```\n\n\n\n\n    [('how do bi-encoders work for sentence embeddings',\n      'SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features: Models based on large-pretrained language models, such as S(entence)BERT,\\nprovide effective and efficient sentence embeddings that show high correlation\\nto human similarity ratings, but lack interpretability. On the other hand,\\ngraph metrics for graph-based meaning representations (e.g., Abstract Meaning\\nRepresentation, AMR) can make explicit the semantic aspects in which two\\nsentences are similar."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: c400734f-7bec-4840-9305-280811c40d7e
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"c400734f-7bec-4840-9305-280811c40d7e\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"00e5c312-3943-4aff-8300-ab467bc897c6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"021098f5b5998c28a005c79e67351178440d9c68066b9a268b0d7646bdd94d3f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3bbafb37-bb33-4a9b-95ba-a38b43f60005\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"69b57bbe8b80f121f619b94de8a976849fd9ddc89d99f2d968ac880bb716bd0a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13364, \"end_char_idx\": 16421, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "```python\nresult = response.choices[0]\nprint(f\"Result was {result.message.content}\")\nprint(f\"Logprobs was {result.logprobs.token_logprobs[0]}\")\nprint(\"\\nBelow is the full logprobs object\\n\\n\")\nprint(result[\"logprobs\"])\n```\n\n    Result was Yes\n    Logprobs was -0.05869877\n    \n    Below is the full logprobs object\n    \n    \n    {\n      \"tokens\": [\n        \"Yes\"\n      ],\n      \"token_logprobs\": [\n        -0.05869877\n      ],\n      \"top_logprobs\": [\n        {\n          \"Yes\": -0.05869877\n        }\n      ],\n      \"text_offset\": [\n        5764\n      ]\n    }\n\n\n\n```python\noutput_list = []\nfor x in result_list:\n    content = x[\"title\"] + \": \" + x[\"summary\"]\n\n    try:\n        output_list.append(document_relevance(query, document=content))\n\n    except Exception as e:\n        print(e)\n```\n\n\n```python\noutput_list[:10]\n```\n\n\n\n\n    [('how do bi-encoders work for sentence embeddings',\n      'SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features: Models based on large-pretrained language models, such as S(entence)BERT,\\nprovide effective and efficient sentence embeddings that show high correlation\\nto human similarity ratings, but lack interpretability. On the other hand,\\ngraph metrics for graph-based meaning representations (e.g., Abstract Meaning\\nRepresentation, AMR) can make explicit the semantic aspects in which two\\nsentences are similar. However, such metrics tend to be slow, rely on parsers,\\nand do not reach state-of-the-art performance when rating sentence similarity.\\n  In this work, we aim at the best of both worlds, by learning to induce\\n$S$emantically $S$tructured $S$entence BERT embeddings (S$^3$BERT). Our\\nS$^3$BERT embeddings are composed of explainable sub-embeddings that emphasize\\nvarious semantic sentence features (e.g., semantic roles, negation, or\\nquantification). We show how to i) learn a decomposition of the sentence\\nembeddings into semantic features, through approximation of a suite of\\ninterpretable AMR graph metrics, and how to ii) preserve the overall power of\\nthe neural embeddings by controlling the decomposition learning process with a\\nsecond objective that enforces consistency with the similarity ratings of an\\nSBERT teacher model. In our experimental studies, we show that our approach\\noffers interpretability -- while fully preserving the effectiveness and\\nefficiency of the neural sentence embeddings.',\n      'Yes',\n      -0.05326408),\n     ('how do bi-encoders work for sentence embeddings',\n      'Are Classes Clusters?: Sentence embedding models aim to provide general purpose embeddings for\\nsentences. Most of the models studied in this paper claim to perform well on\\nSTS tasks - but they do not report on their suitability for clustering. This\\npaper looks at four recent sentence embedding models (Universal Sentence\\nEncoder (Cer et al., 2018), Sentence-BERT (Reimers and Gurevych, 2019), LASER\\n(Artetxe and Schwenk, 2019), and DeCLUTR (Giorgi et al., 2020)). It gives a\\nbrief overview of the ideas behind their implementations."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 3bbafb37-bb33-4a9b-95ba-a38b43f60005
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"3bbafb37-bb33-4a9b-95ba-a38b43f60005\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c400734f-7bec-4840-9305-280811c40d7e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"4d532d973f552278fb4b4c39bfa4f70dcf12602221d1de2c22841174338704db\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f6f24107-d1e7-49e6-aba6-54932f384a84\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"ec57d9ad7b08d80c894e8827b95c8b79aea64cd57921ff8a288b8515813212bd\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 14768, \"end_char_idx\": 17738, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "However, such metrics tend to be slow, rely on parsers,\\nand do not reach state-of-the-art performance when rating sentence similarity.\\n  In this work, we aim at the best of both worlds, by learning to induce\\n$S$emantically $S$tructured $S$entence BERT embeddings (S$^3$BERT). Our\\nS$^3$BERT embeddings are composed of explainable sub-embeddings that emphasize\\nvarious semantic sentence features (e.g., semantic roles, negation, or\\nquantification). We show how to i) learn a decomposition of the sentence\\nembeddings into semantic features, through approximation of a suite of\\ninterpretable AMR graph metrics, and how to ii) preserve the overall power of\\nthe neural embeddings by controlling the decomposition learning process with a\\nsecond objective that enforces consistency with the similarity ratings of an\\nSBERT teacher model. In our experimental studies, we show that our approach\\noffers interpretability -- while fully preserving the effectiveness and\\nefficiency of the neural sentence embeddings.',\n      'Yes',\n      -0.05326408),\n     ('how do bi-encoders work for sentence embeddings',\n      'Are Classes Clusters?: Sentence embedding models aim to provide general purpose embeddings for\\nsentences. Most of the models studied in this paper claim to perform well on\\nSTS tasks - but they do not report on their suitability for clustering. This\\npaper looks at four recent sentence embedding models (Universal Sentence\\nEncoder (Cer et al., 2018), Sentence-BERT (Reimers and Gurevych, 2019), LASER\\n(Artetxe and Schwenk, 2019), and DeCLUTR (Giorgi et al., 2020)). It gives a\\nbrief overview of the ideas behind their implementations. It then investigates\\nhow well topic classes in two text classification datasets (Amazon Reviews (Ni\\net al., 2019) and News Category Dataset (Misra, 2018)) map to clusters in their\\ncorresponding sentence embedding space. While the performance of the resulting\\nclassification model is far from perfect, it is better than random. This is\\ninteresting because the classification model has been constructed in an\\nunsupervised way. The topic classes in these real life topic classification\\ndatasets can be partly reconstructed by clustering the corresponding sentence\\nembeddings.',\n      'No',\n      -0.009535169),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Semantic Composition in Visually Grounded Language Models: What is sentence meaning and its ideal representation? Much of the expressive\\npower of human language derives from semantic composition, the mind's ability\\nto represent meaning hierarchically & relationally over constituents. At the\\nsame time, much sentential meaning is outside the text and requires grounding\\nin sensory, motor, and experiential modalities to be adequately learned.\\nAlthough large language models display considerable compositional ability,\\nrecent work shows that visually-grounded language models drastically fail to\\nrepresent compositional structure."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: f6f24107-d1e7-49e6-aba6-54932f384a84
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"f6f24107-d1e7-49e6-aba6-54932f384a84\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3bbafb37-bb33-4a9b-95ba-a38b43f60005\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e3f7c41addbdcf0ed849a70d7aa8ab06e025b0eeb1e03d03cfb5de2d3ab40719\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e701ec88-f73f-423a-99c7-7c1a29283561\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"bf5010e8448d5116cea8ea74ee9ce14367937d7d88bc31f3ce02ad2060ecd3cc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 15903, \"end_char_idx\": 19480, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": ": Sentence embedding models aim to provide general purpose embeddings for\\nsentences. Most of the models studied in this paper claim to perform well on\\nSTS tasks - but they do not report on their suitability for clustering. This\\npaper looks at four recent sentence embedding models (Universal Sentence\\nEncoder (Cer et al., 2018), Sentence-BERT (Reimers and Gurevych, 2019), LASER\\n(Artetxe and Schwenk, 2019), and DeCLUTR (Giorgi et al., 2020)). It gives a\\nbrief overview of the ideas behind their implementations. It then investigates\\nhow well topic classes in two text classification datasets (Amazon Reviews (Ni\\net al., 2019) and News Category Dataset (Misra, 2018)) map to clusters in their\\ncorresponding sentence embedding space. While the performance of the resulting\\nclassification model is far from perfect, it is better than random. This is\\ninteresting because the classification model has been constructed in an\\nunsupervised way. The topic classes in these real life topic classification\\ndatasets can be partly reconstructed by clustering the corresponding sentence\\nembeddings.',\n      'No',\n      -0.009535169),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Semantic Composition in Visually Grounded Language Models: What is sentence meaning and its ideal representation? Much of the expressive\\npower of human language derives from semantic composition, the mind's ability\\nto represent meaning hierarchically & relationally over constituents. At the\\nsame time, much sentential meaning is outside the text and requires grounding\\nin sensory, motor, and experiential modalities to be adequately learned.\\nAlthough large language models display considerable compositional ability,\\nrecent work shows that visually-grounded language models drastically fail to\\nrepresent compositional structure. In this thesis, we explore whether & how\\nmodels compose visually grounded semantics, and how we might improve their\\nability to do so.\\n  Specifically, we introduce 1) WinogroundVQA, a new compositional visual\\nquestion answering benchmark, 2) Syntactic Neural Module Distillation, a\\nmeasure of compositional ability in sentence embedding models, 3) Causal\\nTracing for Image Captioning Models to locate neural representations vital for\\nvision-language composition, 4) Syntactic MeanPool to inject a compositional\\ninductive bias into sentence embeddings, and 5) Cross-modal Attention\\nCongruence Regularization, a self-supervised objective function for\\nvision-language relation alignment. We close by discussing connections of our\\nwork to neuroscience, psycholinguistics, formal semantics, and philosophy.\",\n      'No',\n      -0.008887106),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions: Text embedding models from Natural Language Processing can map text data\\n(e.g. words, sentences, documents) to supposedly meaningful numerical\\nrepresentations (a.k.a. text embeddings). While such models are increasingly\\napplied in social science research, one important issue is often not addressed:\\nthe extent to which these embeddings are valid representations of constructs\\nrelevant for social science research. We therefore propose the use of the\\nclassic construct validity framework to evaluate the validity of text\\nembeddings. We show how this framework can be adapted to the opaque and\\nhigh-dimensional nature of text embeddings, with application to survey\\nquestions. We include several popular text embedding methods (e.g."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: e701ec88-f73f-423a-99c7-7c1a29283561
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"e701ec88-f73f-423a-99c7-7c1a29283561\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f6f24107-d1e7-49e6-aba6-54932f384a84\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"5f5f5d346a4915384b25c1e25749946570bbd2d640fea1e560a68ffdbb4f6f36\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d0ae91ab-2e04-4d17-ab46-89ae21f5aef6\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b2f1b8ba08f4d4366ce16ecff5edb83fd9c2338dbd04f63975f3f6485c6bf97d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 17739, \"end_char_idx\": 21470, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "In this thesis, we explore whether & how\\nmodels compose visually grounded semantics, and how we might improve their\\nability to do so.\\n  Specifically, we introduce 1) WinogroundVQA, a new compositional visual\\nquestion answering benchmark, 2) Syntactic Neural Module Distillation, a\\nmeasure of compositional ability in sentence embedding models, 3) Causal\\nTracing for Image Captioning Models to locate neural representations vital for\\nvision-language composition, 4) Syntactic MeanPool to inject a compositional\\ninductive bias into sentence embeddings, and 5) Cross-modal Attention\\nCongruence Regularization, a self-supervised objective function for\\nvision-language relation alignment. We close by discussing connections of our\\nwork to neuroscience, psycholinguistics, formal semantics, and philosophy.\",\n      'No',\n      -0.008887106),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions: Text embedding models from Natural Language Processing can map text data\\n(e.g. words, sentences, documents) to supposedly meaningful numerical\\nrepresentations (a.k.a. text embeddings). While such models are increasingly\\napplied in social science research, one important issue is often not addressed:\\nthe extent to which these embeddings are valid representations of constructs\\nrelevant for social science research. We therefore propose the use of the\\nclassic construct validity framework to evaluate the validity of text\\nembeddings. We show how this framework can be adapted to the opaque and\\nhigh-dimensional nature of text embeddings, with application to survey\\nquestions. We include several popular text embedding methods (e.g. fastText,\\nGloVe, BERT, Sentence-BERT, Universal Sentence Encoder) in our construct\\nvalidity analyses. We find evidence of convergent and discriminant validity in\\nsome cases. We also show that embeddings can be used to predict respondent's\\nanswers to completely new survey questions. Furthermore, BERT-based embedding\\ntechniques and the Universal Sentence Encoder provide more valid\\nrepresentations of survey questions than do others. Our results thus highlight\\nthe necessity to examine the construct validity of text embeddings before\\ndeploying them in social science research.\",\n      'No',\n      -0.008583762),\n     ('how do bi-encoders work for sentence embeddings',\n      'Learning Probabilistic Sentence Representations from Paraphrases: Probabilistic word embeddings have shown effectiveness in capturing notions\\nof generality and entailment, but there is very little work on doing the\\nanalogous type of investigation for sentences. In this paper we define\\nprobabilistic models that produce distributions for sentences. Our\\nbest-performing model treats each word as a linear transformation operator\\napplied to a multivariate Gaussian distribution. We train our models on\\nparaphrases and demonstrate that they naturally capture sentence specificity.\\nWhile our proposed model achieves the best performance overall, we also show\\nthat specificity is represented by simpler architectures via the norm of the\\nsentence vectors. Qualitative analysis shows that our probabilistic model\\ncaptures sentential entailment and provides ways to analyze the specificity and\\npreciseness of individual words.',\n      'No',\n      -0.011975748),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Exploiting Twitter as Source of Large Corpora of Weakly Similar Pairs for Semantic Sentence Embeddings: Semantic sentence embeddings are usually supervisedly built minimizing\\ndistances between pairs of embeddings of sentences labelled as semantically\\nsimilar by annotators."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: d0ae91ab-2e04-4d17-ab46-89ae21f5aef6
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"d0ae91ab-2e04-4d17-ab46-89ae21f5aef6\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e701ec88-f73f-423a-99c7-7c1a29283561\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"8eb13a895de46b00b3d5eb979d965bfbb890838ed37b77ffac66d40798e7bdf3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"64ca13da-8151-444a-a3a8-bf01cfa79186\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"78dbfb73c3c8ccf7ea9995913b7e9cd63c473a09dfbf60eb0deacbae86cf9cfe\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 19585, \"end_char_idx\": 23318, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "We find evidence of convergent and discriminant validity in\\nsome cases. We also show that embeddings can be used to predict respondent's\\nanswers to completely new survey questions. Furthermore, BERT-based embedding\\ntechniques and the Universal Sentence Encoder provide more valid\\nrepresentations of survey questions than do others. Our results thus highlight\\nthe necessity to examine the construct validity of text embeddings before\\ndeploying them in social science research.\",\n      'No',\n      -0.008583762),\n     ('how do bi-encoders work for sentence embeddings',\n      'Learning Probabilistic Sentence Representations from Paraphrases: Probabilistic word embeddings have shown effectiveness in capturing notions\\nof generality and entailment, but there is very little work on doing the\\nanalogous type of investigation for sentences. In this paper we define\\nprobabilistic models that produce distributions for sentences. Our\\nbest-performing model treats each word as a linear transformation operator\\napplied to a multivariate Gaussian distribution. We train our models on\\nparaphrases and demonstrate that they naturally capture sentence specificity.\\nWhile our proposed model achieves the best performance overall, we also show\\nthat specificity is represented by simpler architectures via the norm of the\\nsentence vectors. Qualitative analysis shows that our probabilistic model\\ncaptures sentential entailment and provides ways to analyze the specificity and\\npreciseness of individual words.',\n      'No',\n      -0.011975748),\n     ('how do bi-encoders work for sentence embeddings',\n      \"Exploiting Twitter as Source of Large Corpora of Weakly Similar Pairs for Semantic Sentence Embeddings: Semantic sentence embeddings are usually supervisedly built minimizing\\ndistances between pairs of embeddings of sentences labelled as semantically\\nsimilar by annotators. Since big labelled datasets are rare, in particular for\\nnon-English languages, and expensive, recent studies focus on unsupervised\\napproaches that require not-paired input sentences. We instead propose a\\nlanguage-independent approach to build large datasets of pairs of informal\\ntexts weakly similar, without manual human effort, exploiting Twitter's\\nintrinsic powerful signals of relatedness: replies and quotes of tweets. We use\\nthe collected pairs to train a Transformer model with triplet-like structures,\\nand we test the generated embeddings on Twitter NLP similarity tasks (PIT and\\nTURL) and STSb. We also introduce four new sentence ranking evaluation\\nbenchmarks of informal texts, carefully extracted from the initial collections\\nof tweets, proving not only that our best model learns classical Semantic\\nTextual Similarity, but also excels on tasks where pairs of sentences are not\\nexact paraphrases. Ablation studies reveal how increasing the corpus size\\ninfluences positively the results, even at 2M samples, suggesting that bigger\\ncollections of Tweets still do not contain redundant information about semantic\\nsimilarities.\",\n      'No',\n      -0.01219046),\n     ('how do bi-encoders work for sentence embeddings',\n      \"How to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation: Sentence encoders map sentences to real valued vectors for use in downstream\\napplications. To peek into these representations - e.g., to increase\\ninterpretability of their results - probing tasks have been designed which\\nquery them for linguistic knowledge. However, designing probing tasks for\\nlesser-resourced languages is tricky, because these often lack large-scale\\nannotated data or (high-quality) dependency parsers as a prerequisite of\\nprobing task design in English."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 64ca13da-8151-444a-a3a8-bf01cfa79186
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"64ca13da-8151-444a-a3a8-bf01cfa79186\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d0ae91ab-2e04-4d17-ab46-89ae21f5aef6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"c992e938d13310a2c62a6cd5810190b4f5e100812235247e035e93e4cf54079f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9eeddf06-c888-4ffe-8797-2c55faa12309\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6b03c60e3055708922808458659a58a182a85c9d075f314ee08e216ceb7112b6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 21471, \"end_char_idx\": 25106, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "Since big labelled datasets are rare, in particular for\\nnon-English languages, and expensive, recent studies focus on unsupervised\\napproaches that require not-paired input sentences. We instead propose a\\nlanguage-independent approach to build large datasets of pairs of informal\\ntexts weakly similar, without manual human effort, exploiting Twitter's\\nintrinsic powerful signals of relatedness: replies and quotes of tweets. We use\\nthe collected pairs to train a Transformer model with triplet-like structures,\\nand we test the generated embeddings on Twitter NLP similarity tasks (PIT and\\nTURL) and STSb. We also introduce four new sentence ranking evaluation\\nbenchmarks of informal texts, carefully extracted from the initial collections\\nof tweets, proving not only that our best model learns classical Semantic\\nTextual Similarity, but also excels on tasks where pairs of sentences are not\\nexact paraphrases. Ablation studies reveal how increasing the corpus size\\ninfluences positively the results, even at 2M samples, suggesting that bigger\\ncollections of Tweets still do not contain redundant information about semantic\\nsimilarities.\",\n      'No',\n      -0.01219046),\n     ('how do bi-encoders work for sentence embeddings',\n      \"How to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation: Sentence encoders map sentences to real valued vectors for use in downstream\\napplications. To peek into these representations - e.g., to increase\\ninterpretability of their results - probing tasks have been designed which\\nquery them for linguistic knowledge. However, designing probing tasks for\\nlesser-resourced languages is tricky, because these often lack large-scale\\nannotated data or (high-quality) dependency parsers as a prerequisite of\\nprobing task design in English. To investigate how to probe sentence embeddings\\nin such cases, we investigate sensitivity of probing task results to structural\\ndesign choices, conducting the first such large scale study. We show that\\ndesign choices like size of the annotated probing dataset and type of\\nclassifier used for evaluation do (sometimes substantially) influence probing\\noutcomes. We then probe embeddings in a multilingual setup with design choices\\nthat lie in a 'stable region', as we identify for English, and find that\\nresults on English do not transfer to other languages. Fairer and more\\ncomprehensive sentence-level probing evaluation should thus be carried out on\\nmultiple languages in the future.\",\n      'No',\n      -0.015550519),\n     ('how do bi-encoders work for sentence embeddings',\n      'Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences: Sentence embedding methods offer a powerful approach for working with short\\ntextual constructs or sequences of words. By representing sentences as dense\\nnumerical vectors, many natural language processing (NLP) applications have\\nimproved their performance. However, relatively little is understood about the\\nlatent structure of sentence embeddings. Specifically, research has not\\naddressed whether the length and structure of sentences impact the sentence\\nembedding space and topology. This paper reports research on a set of\\ncomprehensive clustering and network analyses targeting sentence and\\nsub-sentence embedding spaces. Results show that one method generates the most\\nclusterable embeddings. In general, the embeddings of span sub-sentences have\\nbetter clustering properties than the original sentences. The results have\\nimplications for future sentence embedding models and applications."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 9eeddf06-c888-4ffe-8797-2c55faa12309
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"9eeddf06-c888-4ffe-8797-2c55faa12309\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"64ca13da-8151-444a-a3a8-bf01cfa79186\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"bc32c46263f16c0aa050cb1d16aa6972ec0b4f8f0b4bfdf8b65f27fcc94ac9ae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"536485a8-535b-428d-b5a3-9ad561b51b86\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c46a8681ab2e68bb10f4c7f45f736d3412347c95bf944cd65ecc37b6dabca402\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 23099, \"end_char_idx\": 26787, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "However, designing probing tasks for\\nlesser-resourced languages is tricky, because these often lack large-scale\\nannotated data or (high-quality) dependency parsers as a prerequisite of\\nprobing task design in English. To investigate how to probe sentence embeddings\\nin such cases, we investigate sensitivity of probing task results to structural\\ndesign choices, conducting the first such large scale study. We show that\\ndesign choices like size of the annotated probing dataset and type of\\nclassifier used for evaluation do (sometimes substantially) influence probing\\noutcomes. We then probe embeddings in a multilingual setup with design choices\\nthat lie in a 'stable region', as we identify for English, and find that\\nresults on English do not transfer to other languages. Fairer and more\\ncomprehensive sentence-level probing evaluation should thus be carried out on\\nmultiple languages in the future.\",\n      'No',\n      -0.015550519),\n     ('how do bi-encoders work for sentence embeddings',\n      'Clustering and Network Analysis for the Embedding Spaces of Sentences and Sub-Sentences: Sentence embedding methods offer a powerful approach for working with short\\ntextual constructs or sequences of words. By representing sentences as dense\\nnumerical vectors, many natural language processing (NLP) applications have\\nimproved their performance. However, relatively little is understood about the\\nlatent structure of sentence embeddings. Specifically, research has not\\naddressed whether the length and structure of sentences impact the sentence\\nembedding space and topology. This paper reports research on a set of\\ncomprehensive clustering and network analyses targeting sentence and\\nsub-sentence embedding spaces. Results show that one method generates the most\\nclusterable embeddings. In general, the embeddings of span sub-sentences have\\nbetter clustering properties than the original sentences. The results have\\nimplications for future sentence embedding models and applications.',\n      'No',\n      -0.012663184),\n     ('how do bi-encoders work for sentence embeddings',\n      'Vec2Sent: Probing Sentence Embeddings with Natural Language Generation: We introspect black-box sentence embeddings by conditionally generating from\\nthem with the objective to retrieve the underlying discrete sentence. We\\nperceive of this as a new unsupervised probing task and show that it correlates\\nwell with downstream task performance. We also illustrate how the language\\ngenerated from different encoders differs. We apply our approach to generate\\nsentence analogies from sentence embeddings.',\n      'Yes',\n      -0.004863006),\n     ('how do bi-encoders work for sentence embeddings',\n      'Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings: Semantic representation learning for sentences is an important and\\nwell-studied problem in NLP. The current trend for this task involves training\\na Transformer-based sentence encoder through a contrastive objective with text,\\ni.e., clustering sentences with semantically similar meanings and scattering\\nothers. In this work, we find the performance of Transformer models as sentence\\nencoders can be improved by training with multi-modal multi-task losses, using\\nunpaired examples from another modality (e.g., sentences and unrelated\\nimage/audio data). In particular, besides learning by the contrastive loss on\\ntext, our model clusters examples from a non-linguistic domain (e.g.,\\nvisual/audio) with a similar contrastive loss at the same time. The reliance of\\nour framework on unpaired non-linguistic data makes it language-agnostic,\\nenabling it to be widely applicable beyond English NLP."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 536485a8-535b-428d-b5a3-9ad561b51b86
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"536485a8-535b-428d-b5a3-9ad561b51b86\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9eeddf06-c888-4ffe-8797-2c55faa12309\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"d6d910eef98c7533060b1f40b53aa24365a82e9be9f2669e3a9dbf982b3f165f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"8224a511-f87c-4235-a80e-b5aab76ccbeb\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b77b059a6e4a6df08101daec25b902fbeabe859cb7633f927e152a7795d191ce\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 24908, \"end_char_idx\": 27675, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "In general, the embeddings of span sub-sentences have\\nbetter clustering properties than the original sentences. The results have\\nimplications for future sentence embedding models and applications.',\n      'No',\n      -0.012663184),\n     ('how do bi-encoders work for sentence embeddings',\n      'Vec2Sent: Probing Sentence Embeddings with Natural Language Generation: We introspect black-box sentence embeddings by conditionally generating from\\nthem with the objective to retrieve the underlying discrete sentence. We\\nperceive of this as a new unsupervised probing task and show that it correlates\\nwell with downstream task performance. We also illustrate how the language\\ngenerated from different encoders differs. We apply our approach to generate\\nsentence analogies from sentence embeddings.',\n      'Yes',\n      -0.004863006),\n     ('how do bi-encoders work for sentence embeddings',\n      'Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings: Semantic representation learning for sentences is an important and\\nwell-studied problem in NLP. The current trend for this task involves training\\na Transformer-based sentence encoder through a contrastive objective with text,\\ni.e., clustering sentences with semantically similar meanings and scattering\\nothers. In this work, we find the performance of Transformer models as sentence\\nencoders can be improved by training with multi-modal multi-task losses, using\\nunpaired examples from another modality (e.g., sentences and unrelated\\nimage/audio data). In particular, besides learning by the contrastive loss on\\ntext, our model clusters examples from a non-linguistic domain (e.g.,\\nvisual/audio) with a similar contrastive loss at the same time. The reliance of\\nour framework on unpaired non-linguistic data makes it language-agnostic,\\nenabling it to be widely applicable beyond English NLP. Experiments on 7\\nsemantic textual similarity benchmarks reveal that models trained with the\\nadditional non-linguistic (/images/audio) contrastive objective lead to higher\\nquality sentence embeddings. This indicates that Transformer models are able to\\ngeneralize better by doing a similar task (i.e., clustering) with unpaired\\nexamples from different modalities in a multi-task fashion.',\n      'No',\n      -0.013869206)]\n\n\n\n\n```python\noutput_df = pd.DataFrame(\n    output_list, columns=[\"query\", \"document\", \"prediction\", \"logprobs\"]\n).reset_index()\n# Use exp() to convert logprobs into probability\noutput_df[\"probability\"] = output_df[\"logprobs\"].apply(exp)\n# Reorder based on likelihood of being Yes\noutput_df[\"yes_probability\"] = output_df.apply(\n    lambda x: x[\"probability\"] * -1 + 1\n    if x[\"prediction\"] == \"No\"\n    else x[\"probability\"],\n    axis=1,\n)\noutput_df.head()\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 8224a511-f87c-4235-a80e-b5aab76ccbeb
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"8224a511-f87c-4235-a80e-b5aab76ccbeb\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"536485a8-535b-428d-b5a3-9ad561b51b86\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"4f116265baae5c2e107aad5ed992682e60d72bc0797ec7e3ad4e92e64c9d0228\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"5070a8e8-6e31-488d-82ba-37869d635c42\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"55647bea4fbc3f70bf955f21af430a006d62dfe440b5afa05380b7569874f93d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25983, \"end_char_idx\": 29606, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "The current trend for this task involves training\\na Transformer-based sentence encoder through a contrastive objective with text,\\ni.e., clustering sentences with semantically similar meanings and scattering\\nothers. In this work, we find the performance of Transformer models as sentence\\nencoders can be improved by training with multi-modal multi-task losses, using\\nunpaired examples from another modality (e.g., sentences and unrelated\\nimage/audio data). In particular, besides learning by the contrastive loss on\\ntext, our model clusters examples from a non-linguistic domain (e.g.,\\nvisual/audio) with a similar contrastive loss at the same time. The reliance of\\nour framework on unpaired non-linguistic data makes it language-agnostic,\\nenabling it to be widely applicable beyond English NLP. Experiments on 7\\nsemantic textual similarity benchmarks reveal that models trained with the\\nadditional non-linguistic (/images/audio) contrastive objective lead to higher\\nquality sentence embeddings. This indicates that Transformer models are able to\\ngeneralize better by doing a similar task (i.e., clustering) with unpaired\\nexamples from different modalities in a multi-task fashion.',\n      'No',\n      -0.013869206)]\n\n\n\n\n```python\noutput_df = pd.DataFrame(\n    output_list, columns=[\"query\", \"document\", \"prediction\", \"logprobs\"]\n).reset_index()\n# Use exp() to convert logprobs into probability\noutput_df[\"probability\"] = output_df[\"logprobs\"].apply(exp)\n# Reorder based on likelihood of being Yes\noutput_df[\"yes_probability\"] = output_df.apply(\n    lambda x: x[\"probability\"] * -1 + 1\n    if x[\"prediction\"] == \"No\"\n    else x[\"probability\"],\n    axis=1,\n)\noutput_df.head()\n```\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>index</th>\n      <th>query</th>\n      <th>document</th>\n      <th>prediction</th>\n      <th>logprobs</th>\n      <th>probability</th>\n      <th>yes_probability</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>SBERT studies Meaning Representations: Decompo...</td>\n      <td>Yes</td>\n      <td>-0.053264</td>\n      <td>0.948130</td>\n      <td>0.948130</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Are Classes Clusters?: Sentence embedding mode...</td>\n      <td>No</td>\n      <td>-0.009535</td>\n      <td>0.990510</td>\n      <td>0.009490</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>2</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Semantic Composition in Visually Grounded Lang...</td>\n      <td>No</td>\n      <td>-0.008887</td>\n      <td>0.991152</td>\n      <td>0.008848</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>3</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Evaluating the Construct Validity of Text Embe...</td>\n      <td>No</td>\n      <td>-0.008584</td>\n      <td>0.991453</td>\n      <td>0.008547</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>4</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Learning Probabilistic Sentence Representation...</td>\n      <td>No</td>\n      <td>-0.011976</td>\n      <td>0.988096</td>\n      <td>0.011904</td>\n    </tr>\n  </tbody>\n</table>\n</div>"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 5070a8e8-6e31-488d-82ba-37869d635c42
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"5070a8e8-6e31-488d-82ba-37869d635c42\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8224a511-f87c-4235-a80e-b5aab76ccbeb\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"2f51f7a22cf5b6141882dfc1e2f4019e9985025b15bd2db2904922017ff5ad4d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"61b79056-6555-4491-9e27-ad3db769e7a3\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4301d325c747ff48cd51bab7eab09876a069081b5833546d979c5590bdb8e8be\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "```python\n# Return reranked results\nreranked_df = output_df.sort_values(\n    by=[\"yes_probability\"], ascending=False\n).reset_index()\nreranked_df.head(10)\n```\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>level_0</th>\n      <th>index</th>\n      <th>query</th>\n      <th>document</th>\n      <th>prediction</th>\n      <th>logprobs</th>\n      <th>probability</th>\n      <th>yes_probability</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>16</td>\n      <td>16</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>In Search for Linear Relations in Sentence Emb.</td>\n      <td>Yes</td>\n      <td>-0.004824</td>\n      <td>0.995187</td>\n      <td>0.995187</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>8</td>\n      <td>8</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Vec2Sent: Probing Sentence Embeddings with Nat.</td>\n      <td>Yes</td>\n      <td>-0.004863</td>\n      <td>0.995149</td>\n      <td>0.995149</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>19</td>\n      <td>19</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Relational Sentence Embedding for Flexible Sem.</td>\n      <td>Yes</td>\n      <td>-0.038814</td>\n      <td>0.961930</td>\n      <td>0.961930</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>0</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>SBERT studies Meaning Representations: Decompo.</td>\n      <td>Yes</td>\n      <td>-0.053264</td>\n      <td>0.948130</td>\n      <td>0.948130</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>15</td>\n      <td>15</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Sentence-T5: Scalable Sentence Encoders from P.</td>\n      <td>No</td>\n      <td>-0.291893</td>\n      <td>0.746849</td>\n      <td>0."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 61b79056-6555-4491-9e27-ad3db769e7a3
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"61b79056-6555-4491-9e27-ad3db769e7a3\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"5070a8e8-6e31-488d-82ba-37869d635c42\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9ff4ce39361d7bf9ab8a499f46f7ce4b32602dbd80bd55fa2540b9c28cc2fcee\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"dbc596cb-bb5e-419f-acf8-498086d31d54\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"9ccd774dacbd5341f4ad9452c643df9af4b136c9c2a3c1e202c34a7c516725b8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "</td>\n      <td>Yes</td>\n      <td>-0.004863</td>\n      <td>0.995149</td>\n      <td>0.995149</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>19</td>\n      <td>19</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Relational Sentence Embedding for Flexible Sem.</td>\n      <td>Yes</td>\n      <td>-0.038814</td>\n      <td>0.961930</td>\n      <td>0.961930</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>0</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>SBERT studies Meaning Representations: Decompo.</td>\n      <td>Yes</td>\n      <td>-0.053264</td>\n      <td>0.948130</td>\n      <td>0.948130</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>15</td>\n      <td>15</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Sentence-T5: Scalable Sentence Encoders from P.</td>\n      <td>No</td>\n      <td>-0.291893</td>\n      <td>0.746849</td>\n      <td>0.253151</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>6</td>\n      <td>6</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>How to Probe Sentence Embeddings in Low-Resour.</td>\n      <td>No</td>\n      <td>-0.015551</td>\n      <td>0.984570</td>\n      <td>0.015430</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>18</td>\n      <td>18</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Efficient and Flexible Topic Modeling using Pr.</td>\n      <td>No</td>\n      <td>-0.015296</td>\n      <td>0.984820</td>\n      <td>0.015180</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>9</td>\n      <td>9</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Non-Linguistic Supervision for Contrastive Lea.</td>\n      <td>No</td>\n      <td>-0.013869</td>\n      <td>0.986227</td>\n      <td>0.013773</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>12</td>\n      <td>12</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Character-based Neural Networks for Sentence P.</td>\n      <td>No</td>\n      <td>-0."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: dbc596cb-bb5e-419f-acf8-498086d31d54
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"dbc596cb-bb5e-419f-acf8-498086d31d54\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"61b79056-6555-4491-9e27-ad3db769e7a3\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e5262f94111b0cbcbc528346b8015dbb15da3d15f5bb995d23eb1fd6af84b46c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ddfdcb0d-ea3a-4b1b-8d4e-c5adb44baacc\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c4a1cdfad550df49c77588a65fee24326cbbe17b705f2857cb71870de6044e47\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "</td>\n      <td>No</td>\n      <td>-0.015551</td>\n      <td>0.984570</td>\n      <td>0.015430</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>18</td>\n      <td>18</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Efficient and Flexible Topic Modeling using Pr.</td>\n      <td>No</td>\n      <td>-0.015296</td>\n      <td>0.984820</td>\n      <td>0.015180</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>9</td>\n      <td>9</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Non-Linguistic Supervision for Contrastive Lea.</td>\n      <td>No</td>\n      <td>-0.013869</td>\n      <td>0.986227</td>\n      <td>0.013773</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>12</td>\n      <td>12</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Character-based Neural Networks for Sentence P.</td>\n      <td>No</td>\n      <td>-0.012866</td>\n      <td>0.987216</td>\n      <td>0.012784</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>7</td>\n      <td>7</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Clustering and Network Analysis for the Embedd.</td>\n      <td>No</td>\n      <td>-0.012663</td>\n      <td>0.987417</td>\n      <td>0.012583</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n\n```python\n# Inspect our new top document following reranking\nreranked_df[\"document\"][0]\n```\n\n\n\n\n    'In Search for Linear Relations in Sentence Embedding Spaces: We present an introductory investigation into continuous-space vector\\nrepresentations of sentences. We acquire pairs of very similar sentences\\ndiffering only by a small alterations (such as change of a noun, adding an\\nadjective, noun or punctuation) from datasets for natural language inference\\nusing a simple pattern method. We look into how such a small change within the\\nsentence text affects its representation in the continuous space and how such\\nalterations are reflected by some of the popular sentence embedding models. We\\nfound that vector differences of some embeddings actually reflect small changes\\nwithin a sentence.'"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ddfdcb0d-ea3a-4b1b-8d4e-c5adb44baacc
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ddfdcb0d-ea3a-4b1b-8d4e-c5adb44baacc\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"dbc596cb-bb5e-419f-acf8-498086d31d54\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"923661de9f1cd98e894f954decdcd4744bd496cfe8e9a9500d125358daf7794b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"31837814-3158-4126-b5b7-105573d1385a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b5e50adc8fdc354796302ff46de056e26161b30d28ff4177475e4807015d7615\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "986227</td>\n      <td>0.013773</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>12</td>\n      <td>12</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Character-based Neural Networks for Sentence P.</td>\n      <td>No</td>\n      <td>-0.012866</td>\n      <td>0.987216</td>\n      <td>0.012784</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>7</td>\n      <td>7</td>\n      <td>how do bi-encoders work for sentence embeddings</td>\n      <td>Clustering and Network Analysis for the Embedd.</td>\n      <td>No</td>\n      <td>-0.012663</td>\n      <td>0.987417</td>\n      <td>0.012583</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n\n```python\n# Inspect our new top document following reranking\nreranked_df[\"document\"][0]\n```\n\n\n\n\n    'In Search for Linear Relations in Sentence Embedding Spaces: We present an introductory investigation into continuous-space vector\\nrepresentations of sentences. We acquire pairs of very similar sentences\\ndiffering only by a small alterations (such as change of a noun, adding an\\nadjective, noun or punctuation) from datasets for natural language inference\\nusing a simple pattern method. We look into how such a small change within the\\nsentence text affects its representation in the continuous space and how such\\nalterations are reflected by some of the popular sentence embedding models. We\\nfound that vector differences of some embeddings actually reflect small changes\\nwithin a sentence.'\n\n\n\n## Conclusion\n\nWe've shown how to create a tailored cross-encoder to rerank academic papers. This approach will work best where there are domain-specific nuances that can be used to pick the most relevant corpus for your users, and where some pre-filtering has taken place to limit the amount of data the cross-encoder will need to process. \n\nA few typical use cases we've seen are:\n- Returning a list of 100 most relevant stock reports, then re-ordering into a top 5 or 10 based on the detailed context of a particular set of customer portfolios\n- Running after a classic rules-based search that gets the top 100 or 1000 most relevant results to prune it according to a specific user's context"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 31837814-3158-4126-b5b7-105573d1385a
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb", "title": "Search reranking with cross-encoders", "tokens": 8550, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"31837814-3158-4126-b5b7-105573d1385a\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"64f6fb54-183f-4a0b-a1e1-9040a6717110\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e8e0acaf40970dd551e968e92897d3415bd23a6e4bc77c0be91f1270b664426c\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ddfdcb0d-ea3a-4b1b-8d4e-c5adb44baacc\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Search_reranking_with_cross-encoders.ipynb\", \"title\": \"Search reranking with cross-encoders\", \"tokens\": 8550, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"6d34501548ab229bc051f5699b09a2b15e6cc393a3ef14aeeba0c26d7bc4a12c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 33132, \"end_char_idx\": 35818, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "ref_doc_id": "64f6fb54-183f-4a0b-a1e1-9040a6717110", "chroma:document": "</td>\n      <td>No</td>\n      <td>-0.012663</td>\n      <td>0.987417</td>\n      <td>0.012583</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n\n```python\n# Inspect our new top document following reranking\nreranked_df[\"document\"][0]\n```\n\n\n\n\n    'In Search for Linear Relations in Sentence Embedding Spaces: We present an introductory investigation into continuous-space vector\\nrepresentations of sentences. We acquire pairs of very similar sentences\\ndiffering only by a small alterations (such as change of a noun, adding an\\nadjective, noun or punctuation) from datasets for natural language inference\\nusing a simple pattern method. We look into how such a small change within the\\nsentence text affects its representation in the continuous space and how such\\nalterations are reflected by some of the popular sentence embedding models. We\\nfound that vector differences of some embeddings actually reflect small changes\\nwithin a sentence.'\n\n\n\n## Conclusion\n\nWe've shown how to create a tailored cross-encoder to rerank academic papers. This approach will work best where there are domain-specific nuances that can be used to pick the most relevant corpus for your users, and where some pre-filtering has taken place to limit the amount of data the cross-encoder will need to process. \n\nA few typical use cases we've seen are:\n- Returning a list of 100 most relevant stock reports, then re-ordering into a top 5 or 10 based on the detailed context of a particular set of customer portfolios\n- Running after a classic rules-based search that gets the top 100 or 1000 most relevant results to prune it according to a specific user's context\n\n\n### Taking this forward\n\nTaking the few-shot approach, as we have here, can work well when the domain is general enough that a small number of examples will cover most reranking cases. However, as the differences between documents become more specific you may want to consider the ```Fine-tuning``` endpoint to make a more elaborate cross-encoder with a wider variety of examples.\n\nThere is also a latency impact of using ```text-davinci-003``` that you'll need to consider, with even our few examples above taking a couple seconds each - again, the ```Fine-tuning``` endpoint may help you here if you are able to get decent results from an ```ada``` or ```babbage``` fine-tuned model.\n\nWe've used the ```Completions``` endpoint from OpenAI to build our cross-encoder, but this area is well-served by the open-source community. [Here](https://huggingface.co/jeffwan/mmarco-mMiniLMv2-L12-H384-v1) is an example from HuggingFace, for example.\n\nWe hope you find this useful for tuning your search use cases, and look forward to seeing what you build."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 4e5a0bad-4256-4d0b-a880-43938cf6e343
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"4e5a0bad-4256-4d0b-a880-43938cf6e343\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"feb89fd2-8e6d-4741-a92a-84d88239f6ba\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b8a6f3ffc28b8f5102cba3eae6067a88e995678ef8fdaead23ed51c41fded080\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2246, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "## Developing Hallucination Guardrails\n\nA guardrail is a set of rules and checks designed to ensure that the outputs of an LLM are accurate, appropriate, and aligned with user expectations. For more additional information on developing guardrails, you can refer to this [guide on developing guardrails](https://cookbook.openai.com/examples/how_to_use_guardrails).\n\nIn this notebook, we'll walk through the process of developing an output guardrail that specifically checks model outputs for hallucinations. \n\nThis notebook will focus on:\n1. Building out a strong eval set\n2. Identifying specific criteria to measure hallucinations\n3. Improving the accuracy of our guardrail with few-shot prompting\n\n\n\n```python\nfrom concurrent.futures import ThreadPoolExecutor\nfrom IPython.display import display, HTML\nimport json\nimport pandas as pd\nfrom sklearn.metrics import precision_score, recall_score\nfrom typing import List\nfrom openai import OpenAI\n\nclient = OpenAI()\n```\n\n\n```python\n# Function to set up display options for pandas\ndef setup_pandas_display():\n    # Increase display limits\n    pd.set_option('display.max_rows', 500)\n    pd.set_option('display.max_columns', 500)\n\n# Function to make DataFrame scrollable in the notebook output\ndef make_scrollable(df):\n    style = (\n        '<style>'\n        'div.output_scroll {'\n        'resize: both;'\n        'overflow: auto;'\n        '}'\n        '</style>'\n    )\n    html = f\"{style}{df.to_html()}\"\n    display(HTML(html))\n\n# Main function to display DataFrame\ndef display_dataframe(df):\n    setup_pandas_display()    # Enable scrollable view\n    make_scrollable(df)\n\n```\n\n## 1. Building out an eval set\n\nImagine we are a customer support team that is building out an automated support agent. We will be feeding the assistant information from our knowledge base about a specific set of policies for how to handle tickets such as returns, refunds, feedback, and expect the model to follow the policy when interacting with customers.\n\nThe first thing we will do is use GPT-4o to build out a set of policies that we will want to follow.\n\nIf you want to do deep dive into generating synthetic data, you can review our Synthetic Data Generation Cookbook [here](https://cookbook.openai.com/examples/sdg1)"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: feb89fd2-8e6d-4741-a92a-84d88239f6ba
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"feb89fd2-8e6d-4741-a92a-84d88239f6ba\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4e5a0bad-4256-4d0b-a880-43938cf6e343\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"62b5322369a537afcf3c536c23fb0f252f1940f3a93d111499bddac0d1b5078c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4e5d219b-8dfa-49ce-903e-78ddca139da2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4578abbe6ecdb750e6c7b47c0d256703667cc897656f106bd8016c7fe80d6e54\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 701, \"end_char_idx\": 4395, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\nfrom concurrent.futures import ThreadPoolExecutor\nfrom IPython.display import display, HTML\nimport json\nimport pandas as pd\nfrom sklearn.metrics import precision_score, recall_score\nfrom typing import List\nfrom openai import OpenAI\n\nclient = OpenAI()\n```\n\n\n```python\n# Function to set up display options for pandas\ndef setup_pandas_display():\n    # Increase display limits\n    pd.set_option('display.max_rows', 500)\n    pd.set_option('display.max_columns', 500)\n\n# Function to make DataFrame scrollable in the notebook output\ndef make_scrollable(df):\n    style = (\n        '<style>'\n        'div.output_scroll {'\n        'resize: both;'\n        'overflow: auto;'\n        '}'\n        '</style>'\n    )\n    html = f\"{style}{df.to_html()}\"\n    display(HTML(html))\n\n# Main function to display DataFrame\ndef display_dataframe(df):\n    setup_pandas_display()    # Enable scrollable view\n    make_scrollable(df)\n\n```\n\n## 1. Building out an eval set\n\nImagine we are a customer support team that is building out an automated support agent. We will be feeding the assistant information from our knowledge base about a specific set of policies for how to handle tickets such as returns, refunds, feedback, and expect the model to follow the policy when interacting with customers.\n\nThe first thing we will do is use GPT-4o to build out a set of policies that we will want to follow.\n\nIf you want to do deep dive into generating synthetic data, you can review our Synthetic Data Generation Cookbook [here](https://cookbook.openai.com/examples/sdg1)\n\n\n```python\nsystem_input_prompt = \"\"\"\nYou are a helpful assistant that can generate policies for a support agent at a fictional company to follow. You will be provided with a topic (ie. returns, refunds, feedback) and you are to generate a sample policy for how to handle the it.\n\nWhen constructing the policy, it should contain step-by-step instructions for how to handle the customer inquiry. It should include decision logic for what to do if a customer falls under a certain category, and provide requirements for taking specific actions.\n\"\"\"\n\nuser_policy_example_1 = \"\"\"\"\nRETURN POLICY\n\"\"\"\n\nassistant_policy_example_1 = \"\"\"\nRETURN POLICY\n\n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n\n\"\"\"\n\nuser_policy_input = \"\"\"\n{{POLICY}}\n\"\"\"\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 4e5d219b-8dfa-49ce-903e-78ddca139da2
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"4e5d219b-8dfa-49ce-903e-78ddca139da2\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"feb89fd2-8e6d-4741-a92a-84d88239f6ba\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"00050ad5a97d148ca461eb85b4f559e8a533c7d1ed5431fcd273e7eed5f1f921\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ad359d2e-3a9d-4c6f-bb32-159882c1d9a0\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3beb5ea763c30e3caf60e5016eb2bbff7e876cae17efc82e16979d4c0e743002\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4398, \"end_char_idx\": 7715, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\ndef generate_policy(policy: str) -> str:\n    input_message = user_policy_input.replace(\"{{POLICY}}\", policy)\n    \n    response = client.chat.completions.create(\n        messages= [\n            {\"role\": \"system\", \"content\": system_input_prompt},\n            {\"role\": \"user\", \"content\": user_policy_example_1},\n            {\"role\": \"assistant\", \"content\": assistant_policy_example_1},\n            {\"role\": \"user\", \"content\": input_message},\n        ],\n        model=\"gpt-4o\"\n    )\n    \n    return response.choices[0].message.content\n\ndef generate_policies() -> List[str]:\n    # List of different types of policies to generate \n    policies = ['PRODUCT FEEDBACK POLICY', 'SHIPPING POLICY', 'WARRANTY POLICY', 'ACCOUNT DELETION', 'COMPLAINT RESOLUTION']\n    \n    with ThreadPoolExecutor() as executor:\n        policy_instructions_list = list(executor.map(generate_policy, policies))\n        \n    return policy_instructions_list\n\npolicy_instructions = generate_policies()\n```\n\nNext we'll take these policies and generate sample customer interactions that do or do not follow the instructions.\n\n\n```python\nsystem_input_prompt = \"\"\"\"\nYou are a helpful assistant that can generate fictional interactions between a support assistant and a customer user. You will be given a set of policy instructions that the support agent is instructed to follow.\n\nBased on the instructions, you must generate a relevant single-turn or multi-turn interaction between the assistant and the user. It should average between 1-3 turns total.\n\nFor a given set of instructions, generate an example conversation that where the assistant either does or does not follow the instructions properly. In the assistant's responses, have it give a combination of single sentence and multi-sentence responses.\n\nThe output must be in a json format with the following three parameters:\n - accurate: \n    - This should be a boolean True or False value that matches whether or not the final assistant message accurately follows the policy instructions\n - kb_article:\n    - This should be the entire policy instruction that is passed in from the user\n - chat_history: \n    - This should contain the entire conversation history except for the final assistant message. \n    - This should be in a format of an array of jsons where each json contains two parameters: role, and content. \n    - Role should be set to either 'user' to represent the customer, or 'assistant' to represent the customer support assistant. \n    - Content should contain the message from the appropriate role.\n    - The final message in the chat history should always come from the user. The assistant response in the following parameter will be a response to this use message.\n - assistant_response: \n    - This should contain the final response from the assistant. This is what we will evaluate to determine whether or not it is accurately following the policy.\n\"\"\"\n\nuser_example_1 = \"\"\"\"\nHere are the policy instructions:\nRETURN POLICY\n\n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ad359d2e-3a9d-4c6f-bb32-159882c1d9a0
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ad359d2e-3a9d-4c6f-bb32-159882c1d9a0\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4e5d219b-8dfa-49ce-903e-78ddca139da2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"32948867aafdbb0f84eb9b081af4bd4fe88ade7c21bf233d9567c66316a9b919\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"59581286-bbbe-4b49-aed6-8b79e12502bf\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"26edfa70bcb2c4b56abe23aff655525e63558171ae2c415da5643636301ba1f4\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5749, \"end_char_idx\": 9219, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "Based on the instructions, you must generate a relevant single-turn or multi-turn interaction between the assistant and the user. It should average between 1-3 turns total.\n\nFor a given set of instructions, generate an example conversation that where the assistant either does or does not follow the instructions properly. In the assistant's responses, have it give a combination of single sentence and multi-sentence responses.\n\nThe output must be in a json format with the following three parameters:\n - accurate: \n    - This should be a boolean True or False value that matches whether or not the final assistant message accurately follows the policy instructions\n - kb_article:\n    - This should be the entire policy instruction that is passed in from the user\n - chat_history: \n    - This should contain the entire conversation history except for the final assistant message. \n    - This should be in a format of an array of jsons where each json contains two parameters: role, and content. \n    - Role should be set to either 'user' to represent the customer, or 'assistant' to represent the customer support assistant. \n    - Content should contain the message from the appropriate role.\n    - The final message in the chat history should always come from the user. The assistant response in the following parameter will be a response to this use message.\n - assistant_response: \n    - This should contain the final response from the assistant. This is what we will evaluate to determine whether or not it is accurately following the policy.\n\"\"\"\n\nuser_example_1 = \"\"\"\"\nHere are the policy instructions:\nRETURN POLICY\n\n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n\"\"\"\n\nassistant_example_1 = \"\"\"\n{\n    \"accurate\": \"true\",\n    \"kb_article\": \"1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 59581286-bbbe-4b49-aed6-8b79e12502bf
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"59581286-bbbe-4b49-aed6-8b79e12502bf\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ad359d2e-3a9d-4c6f-bb32-159882c1d9a0\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"b27076f8bf23379d438e8cac2d9abb81f7272bd806eb396c50a8b2c942446c5a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ba3f43e5-6b87-4c7d-9527-096843a2888d\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b668c2cc8bdba5848c31aad95d8b1964ef1be653197e18250518d23252936e18\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7668, \"end_char_idx\": 10927, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "- unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n\"\"\"\n\nassistant_example_1 = \"\"\"\n{\n    \"accurate\": \"true\",\n    \"kb_article\": \"1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a. **If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today.\",\n    \"chat_history\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"Yes, I am not satisfied with the design\"\n        }\n    ],\n    \"assistant_response\": {\n        \"role\": \"assistant\",\n        \"content\": \"I see. Because the shirt was ordered in the last 30 days, we can provide you with a full refund."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ba3f43e5-6b87-4c7d-9527-096843a2888d
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ba3f43e5-6b87-4c7d-9527-096843a2888d\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"59581286-bbbe-4b49-aed6-8b79e12502bf\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"be691e23239e5355333fddabeb2366e9f843280c15f6312bf1fb7942b52aca4e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c161a2ad-dbd5-4555-9a6e-18ab23c0ca08\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8b186bb309a874c900325b0b981e72c4219c4c26196bca2a9a491b6c4d1d9c89\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9220, \"end_char_idx\": 12513, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "**If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today.\",\n    \"chat_history\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"Yes, I am not satisfied with the design\"\n        }\n    ],\n    \"assistant_response\": {\n        \"role\": \"assistant\",\n        \"content\": \"I see. Because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?\"\n    }\n}\n\"\"\"\n\nuser_example_2 = \"\"\"\"\nHere are the policy instructions:\nRETURN POLICY\n\n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n\"\"\""}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: c161a2ad-dbd5-4555-9a6e-18ab23c0ca08
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"c161a2ad-dbd5-4555-9a6e-18ab23c0ca08\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ba3f43e5-6b87-4c7d-9527-096843a2888d\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"7dc32a9a518e77d38bae795ffcce1efbd8411c89b98184c2675347ba5cb00ea0\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"53f60ef2-0c59-40e7-91b5-7378a87beb3e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"75032821da90bf29599370e3c9942e634d6195eb4670b9f100d635d94b96111a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 10838, \"end_char_idx\": 13967, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "Because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?\"\n    }\n}\n\"\"\"\n\nuser_example_2 = \"\"\"\"\nHere are the policy instructions:\nRETURN POLICY\n\n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n\"\"\"\n\nassistant_example_2 = \"\"\"\n{\n    \"accurate\": \"false\",\n    \"kb_article\": \"1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a. **If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 53f60ef2-0c59-40e7-91b5-7378a87beb3e
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"53f60ef2-0c59-40e7-91b5-7378a87beb3e\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c161a2ad-dbd5-4555-9a6e-18ab23c0ca08\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"1d9c5e2ca04d223230d4af4937c460d51b78a14a9792fb30581244ea9f8f50fe\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f2c942f0-83a6-4594-9ed7-e1fd39401f3a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"309dc015d38a987af30a8a46ebab11f778f8f9f10b960f6ad499390920864a3f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 12515, \"end_char_idx\": 15759, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "assistant_example_2 = \"\"\"\n{\n    \"accurate\": \"false\",\n    \"kb_article\": \"1. ** Ask the customer why they want the order replaced ** - Categorize their issue into one of the following buckets: - damaged: They received the product in a damaged state - satisfaction: The customer is not satisfied with the item and does not like the product. - unnecessary: They no longer need the item 2a. **If return category is 'damaged' - Ask customer for a picture of the damaged item - If the item is indeed damaged, continue to step 3 - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund - Skip step 3 and go straight to step 4 2b. **If return category is either 'satisfaction' or 'unnecessary'** - Ask the customer if they can provide feedback on the quality of the item - If the order was made within 30 days, notify them that they are eligible for a full refund - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50% - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund 3. **If the customer is eligible for a return or refund** - Ask the customer to confirm that they would like a return or refund - Once they confirm, process their request 4 **Provide additional support before closing out ticket** - Ask the customer if there is anything else you can do to help them today.\",\n    \"chat_history\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"Yes, I am not satisfied with the design\"\n        }\n    ],\n    \"assistant_response\": {\n        \"role\": \"assistant\",\n        \"content\": \"I see. Because the shirt was ordered in the last 60 days, we cannot process a refund.\"    \n    }\n}\n\"\"\"\n```\n\nNow let's iterate through the policies and generate some examples.\n\n\n```python\ncustomer_interactions = []\n\ndef fetch_response(policy):\n    messages = [\n        { \"role\": \"system\", \"content\": system_input_prompt},\n        { \"role\": \"user\", \"content\": user_example_1},\n        { \"role\": \"assistant\", \"content\": assistant_example_1},\n        { \"role\": \"user\", \"content\": user_example_2},\n        { \"role\": \"assistant\", \"content\": assistant_example_2},\n        { \"role\": \"user\", \"content\": policy}\n    ]\n\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=messages,\n        temperature=0.7,\n        n=10\n    )\n    return response.choices\n\nwith ThreadPoolExecutor() as executor:\n    futures = [executor.submit(fetch_response, policy) for policy in policy_instructions]\n    for future in futures:\n        choices = future.result()\n        customer_interactions.extend([choice.message.content for choice in choices])\n\n```\n\n\n```python\ninteraction_dict = json.loads(customer_interactions[0])\n\ndf_interaction = pd.DataFrame([interaction_dict])\n\n# Pretty print the DataFrame\ndisplay_dataframe(df_interaction)\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: f2c942f0-83a6-4594-9ed7-e1fd39401f3a
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"f2c942f0-83a6-4594-9ed7-e1fd39401f3a\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"53f60ef2-0c59-40e7-91b5-7378a87beb3e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"b0ab4baf95180ed5d6c88fc66318f95cfdcbf08bd1c67ef0994c1269fd1aff83\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"0d65ed82-e23f-4fec-9306-dd336024f0c5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f4795af09643da5c7374ca1bc0fe375d92b2ea8ea151ae1cc9c486a2fc099f75\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 14224, \"end_char_idx\": 19157, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"Yes, I am not satisfied with the design\"\n        }\n    ],\n    \"assistant_response\": {\n        \"role\": \"assistant\",\n        \"content\": \"I see. Because the shirt was ordered in the last 60 days, we cannot process a refund.\"    \n    }\n}\n\"\"\"\n```\n\nNow let's iterate through the policies and generate some examples.\n\n\n```python\ncustomer_interactions = []\n\ndef fetch_response(policy):\n    messages = [\n        { \"role\": \"system\", \"content\": system_input_prompt},\n        { \"role\": \"user\", \"content\": user_example_1},\n        { \"role\": \"assistant\", \"content\": assistant_example_1},\n        { \"role\": \"user\", \"content\": user_example_2},\n        { \"role\": \"assistant\", \"content\": assistant_example_2},\n        { \"role\": \"user\", \"content\": policy}\n    ]\n\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=messages,\n        temperature=0.7,\n        n=10\n    )\n    return response.choices\n\nwith ThreadPoolExecutor() as executor:\n    futures = [executor.submit(fetch_response, policy) for policy in policy_instructions]\n    for future in futures:\n        choices = future.result()\n        customer_interactions.extend([choice.message.content for choice in choices])\n\n```\n\n\n```python\ninteraction_dict = json.loads(customer_interactions[0])\n\ndf_interaction = pd.DataFrame([interaction_dict])\n\n# Pretty print the DataFrame\ndisplay_dataframe(df_interaction)\n\n```\n\n\n<style>div.output_scroll {resize: both;overflow: auto;}</style><table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>accurate</th>\n      <th>kb_article</th>\n      <th>chat_history</th>\n      <th>assistant_response</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Reception** - Thank the customer for taking the time to provide feedback. - Use a personalized greeting: \"Thank you for your feedback, [Customer Name]. We appreciate your input.\" 2. **Categorize Feedback** - Determine the type of feedback: - **Positive Feedback** - **Negative Feedback** - **Suggestions for Improvement** - Document the feedback under the appropriate category in the internal database. 3. **Responding to Positive Feedback** - Express gratitude: \"We're thrilled to hear that you enjoyed our product. Thank you for letting us know!\" - If possible, offer a small token of appreciation (e.g., discount or voucher for future purchases). 4. **Responding to Negative Feedback** - Apologize sincerely and acknowledge the customer's concerns: \"We apologize that our product did not meet your expectations. Your feedback is important to us.\" - Ask for additional details if necessary to understand the issue better. - Reassure the customer that their feedback will be escalated to the product development team. 5. **Responding to Suggestions** - Acknowledge the suggestion: \"Thank you for your suggestion. We value input from our customers as it helps us improve our products.\" - Inform the customer that their suggestion will be reviewed: \"We will share your idea with our product team for further consideration.\" 6. **Internal Processing** - Log all feedback under the respective category in the internal database. - Forward detailed feedback to the product development team bi-weekly. - High-priority issues should be escalated immediately to the senior management team. 7. **Follow-Up** - Monitor whether the customer's feedback leads to any product updates or changes. - If the customer\u2019s feedback resulted in product enhancement, send a follow-up email to inform them: \"Thank you for your valuable feedback. We wanted to let you know that we've made some improvements based on your input.\" 8. **Closing the Loop** - Ask if there is anything else you can assist the customer with: \"Is there anything else we can help you with today?\" - Close the ticket once all queries and feedback are appropriately addressed. 9. **Continuous Improvement** - Analyze feedback trends monthly to identify recurring issues and areas for improvement. - Use feedback insights for product development meetings and strategic planning sessions. By following these steps, we ensure that customer feedback is valued, documented, and acted upon to continuously improve our product offerings.</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let you know that the new app update is fantastic! The interface is so much smoother now.'}]</td>\n      <td>{'role': 'assistant', 'content': 'Thank you for your feedback! We appreciate your input. We're thrilled to hear that you enjoyed our product. Thank you for letting us know! As a token of our appreciation, we're offering you a 10% discount on your next purchase. Is there anything else we can help you with today?'}</td>\n    </tr>\n  </tbody>\n</table>"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 0d65ed82-e23f-4fec-9306-dd336024f0c5
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"0d65ed82-e23f-4fec-9306-dd336024f0c5\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f2c942f0-83a6-4594-9ed7-e1fd39401f3a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"dc9a7f53900de740e69fe8c84c62d95cc4d8e7c886d2a9b6940c194b1790bcaf\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9c71e40b-6c7b-4bbb-97d3-b68ee7307fd7\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"103dc8a23e2ee00b0b14a98c04cecf17389615d68a018cda1f02799e58e4423a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 19161, \"end_char_idx\": 21428, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\n# Decode the JSON strings\ndata = [json.loads(entry) for entry in customer_interactions]\n\n# Create a DataFrame from the cleaned data\ndf = pd.DataFrame(data)\n```\n\n\n```python\ndf.head(10)\n```\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>accurate</th>\n      <th>kb_article</th>\n      <th>chat_history</th>\n      <th>assistant_response</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to give...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY\\n\\n1. **Acknowledge Re...</td>\n      <td>[{'role': 'user', 'content': 'I really enjoyed...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to give...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 9c71e40b-6c7b-4bbb-97d3-b68ee7307fd7
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"9c71e40b-6c7b-4bbb-97d3-b68ee7307fd7\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"0d65ed82-e23f-4fec-9306-dd336024f0c5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"076b8653499307cb0bae9f0aaf6a7b3b7fb64b47ff459d469f17d995d082a542\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"aa52bc35-e6d0-4d86-ba13-c25abad59e0a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"93e28ecc8ca9bdaeb2a844e0080f7fb55570b43d03b3eb5399a53613fc1adba8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 20425, \"end_char_idx\": 22376, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "**Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to give...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY\\n\\n1. **Acknowledge Re...</td>\n      <td>[{'role': 'user', 'content': 'I really enjoyed...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to give...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I didn't like th...</td>\n      <td>{'role': 'assistant', 'content': 'We apologize...</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I have some feed...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I really love th...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>true</td>\n      <td>1. **Acknowledge Reception** - Thank the custo...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to say ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: aa52bc35-e6d0-4d86-ba13-c25abad59e0a
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"aa52bc35-e6d0-4d86-ba13-c25abad59e0a\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9c71e40b-6c7b-4bbb-97d3-b68ee7307fd7\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"51b8bce7e189168e5ace9b6cd7fa905fe539d58a7eb2aafd7369208c5073be0e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a9e5e449-d272-4f99-8760-fde13c09aa9d\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d3925001a23a585f491d557a768b8a3d7c85968af28c54962ce047131f5d4aba\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 21429, \"end_char_idx\": 24579, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "**Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I didn't like th...</td>\n      <td>{'role': 'assistant', 'content': 'We apologize...</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I have some feed...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>true</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I really love th...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>true</td>\n      <td>1. **Acknowledge Reception** - Thank the custo...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to say ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n## 2. Constructing our hallucination guardrail\n\nWhen building out our hallucination guardrail, here are some guiding principles:\n\n1. Provide very descriptive metrics to evaluate whether a response is accurate\n- It is important to break down this idea of \"truth\" in easily identifiable metrics that we can measure\n- Metrics like truthfulness and relevance are difficult to measure. Giving concrete ways to score the statement can result in a more accurate guardrail\n2. Ensure consistency across key terminology\n- It is important to keep relevant terms such as knowledge base articles, assistants, and users consistent across the prompt\n- If we begin to use phrases such as assistant vs agent, the model could get confused\n3. Start with the most advanced model\n- There is a cost vs quality trade-off when using the most advanced models. Although GPT-4o may be more expensive, it is important to start with the most advanced model so we can ensure a high degree of accuracy\n- Once we have thoroughly tested out the guardrail and are confident in its performance, we can look to reducing cost by tuning it down to gpt-3.5-turbo\n4. Evaluate each sentence independently and the entire response as a whole\n- If the agent returns a long response, it can be useful to break down the response to individual sentences and evaluate them independently\n- In addition to that, evaluating the whole intent of the message as a whole can ensure that you don't lose important context\n\nWith all of this in mind, let's build out a guardrail system and measure its performance.\n\n\n```python\nguardrail_system_message = \"\"\"You are a highly specialized assistant tasked with reviewing chatbot responses to identify and flag any inaccuracies or hallucinations. For each user message, you must thoroughly analyze the response by considering:\n    1. Knowledge Accuracy: Does the message accurately reflect information found in the knowledge base? Assess not only direct mentions but also contextually inferred knowledge.\n    2. Relevance: Does the message directly address the user's question or statement? Check if the response logically follows the user\u2019s last message, maintaining coherence in the conversation thread.\n    3."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: a9e5e449-d272-4f99-8760-fde13c09aa9d
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"a9e5e449-d272-4f99-8760-fde13c09aa9d\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"aa52bc35-e6d0-4d86-ba13-c25abad59e0a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"aabdd5e5db7380222c78479765ea05275681c8eb494daef28643c91eb57abccf\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"04a1edee-e0df-4c1d-9ad9-377ab04c5c67\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0b57eb644dc0995dfa5a870db5044b3362a64f94d83443dfb13bb21cb0129f33\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 23938, \"end_char_idx\": 27242, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\nguardrail_system_message = \"\"\"You are a highly specialized assistant tasked with reviewing chatbot responses to identify and flag any inaccuracies or hallucinations. For each user message, you must thoroughly analyze the response by considering:\n    1. Knowledge Accuracy: Does the message accurately reflect information found in the knowledge base? Assess not only direct mentions but also contextually inferred knowledge.\n    2. Relevance: Does the message directly address the user's question or statement? Check if the response logically follows the user\u2019s last message, maintaining coherence in the conversation thread.\n    3. Policy Compliance: Does the message adhere to company policies? Evaluate for subtleties such as misinformation, overpromises, or logical inconsistencies. Ensure the response is polite, non-discriminatory, and practical.\n\nTo perform your task you will be given the following:\n    1. Knowledge Base Articles - These are your source of truth for verifying the content of assistant messages.\n    2. Chat Transcript - Provides context for the conversation between the user and the assistant.\n    3. Assistant Message - The message from the assistant that needs review.\n\nFor each sentence in the assistant's most recent response, assign a score based on the following criteria:\n    1. Factual Accuracy:\n        - Score 1 if the sentence is factually correct and corroborated by the knowledge base.\n        - Score 0 if the sentence contains factual errors or unsubstantiated claims.\n    2. Relevance:\n        - Score 1 if the sentence directly and specifically addresses the user's question or statement without digression.\n        - Score 0 if the sentence is tangential or does not build logically on the conversation thread.\n    3. Policy Compliance:\n        - Score 1 if the response complies with all company policies including accuracy, ethical guidelines, and user engagement standards.\n        - Score 0 if it violates any aspect of the policies, such as misinformation or inappropriate content.\n    4. Contextual Coherence:\n        - Score 1 if the sentence maintains or enhances the coherence of the conversation, connecting logically with preceding messages.\n        - Score 0 if it disrupts the flow or context of the conversation.\n\nInclude in your response an array of JSON objects for each evaluated sentence. Each JSON object should contain:\n    - `sentence`: Text of the evaluated sentence.\n    - `factualAccuracy`: Score for factual correctness (0 or 1).\n    - `factualReference`: If scored 1, cite the exact line(s) from the knowledge base. If scored 0, provide a rationale.\n    - `relevance`: Score for relevance to the user\u2019s question (0 or 1).\n    - `policyCompliance`: Score for adherence to company policies (0 or 1).\n    - `contextualCoherence`: Score for maintaining conversation coherence (0 or 1).\n\nALWAYS RETURN YOUR RESPONSE AS AN ARRAY OF JSONS.\n\"\"\"\n\nfs_user_1 = \"\"\"\n\n## Knowledge Base Articles: \n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 04a1edee-e0df-4c1d-9ad9-377ab04c5c67
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"04a1edee-e0df-4c1d-9ad9-377ab04c5c67\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a9e5e449-d272-4f99-8760-fde13c09aa9d\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"b5cbfd86fbd1c3ddaacf2d10c9165318df41fde2a48d77e20f0402c305421dd1\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"dd2208ea-a84f-48ef-9ca0-c5ede8e3c029\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6705a766f37c8f979827d1fa18f56ce17febe76016151668b5ce7c4683e99d74\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25461, \"end_char_idx\": 28698, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "2. Relevance:\n        - Score 1 if the sentence directly and specifically addresses the user's question or statement without digression.\n        - Score 0 if the sentence is tangential or does not build logically on the conversation thread.\n    3. Policy Compliance:\n        - Score 1 if the response complies with all company policies including accuracy, ethical guidelines, and user engagement standards.\n        - Score 0 if it violates any aspect of the policies, such as misinformation or inappropriate content.\n    4. Contextual Coherence:\n        - Score 1 if the sentence maintains or enhances the coherence of the conversation, connecting logically with preceding messages.\n        - Score 0 if it disrupts the flow or context of the conversation.\n\nInclude in your response an array of JSON objects for each evaluated sentence. Each JSON object should contain:\n    - `sentence`: Text of the evaluated sentence.\n    - `factualAccuracy`: Score for factual correctness (0 or 1).\n    - `factualReference`: If scored 1, cite the exact line(s) from the knowledge base. If scored 0, provide a rationale.\n    - `relevance`: Score for relevance to the user\u2019s question (0 or 1).\n    - `policyCompliance`: Score for adherence to company policies (0 or 1).\n    - `contextualCoherence`: Score for maintaining conversation coherence (0 or 1).\n\nALWAYS RETURN YOUR RESPONSE AS AN ARRAY OF JSONS.\n\"\"\"\n\nfs_user_1 = \"\"\"\n\n## Knowledge Base Articles: \n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n    \n## Chat Transcript:\n    [\n        {\n            \"role\": \"user\",\n            \"content: \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\""}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: dd2208ea-a84f-48ef-9ca0-c5ede8e3c029
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"dd2208ea-a84f-48ef-9ca0-c5ede8e3c029\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"04a1edee-e0df-4c1d-9ad9-377ab04c5c67\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9f991391da2d33784ccfbd08208fd9e2e9cef1f60d14d214a860ac3791fc7473\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a2a96b25-071a-4ad4-8b8b-20f128130d33\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cbeaead1da77ffdbb0b18d6c5a149cd4f6fcc3b7af1f2f87418bc32ed59ca747\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 27195, \"end_char_idx\": 30097, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "- unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n    \n## Chat Transcript:\n    [\n        {\n            \"role\": \"user\",\n            \"content: \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content: \"Yes, I am not satisfied with the design\"\n        }\n    ]\n\n## Assistant Message:\nI see, because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?\n\"\"\"\n\nfs_assistant_1 = \"\"\"[\n    {\n        \"sentence\": \"I see, because the shirt was ordered in the last 30 days, we can provide you with a full refund.\",\n        \"factualAccuracy\": 1,\n        \"factualReference\": \"If the order was made within 30 days, notify them that they are eligible for a full refund\",\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    },\n    {\n        \"sentence\": \"Would you like me to process the refund?\",\n        \"factualAccuracy\": 1,\n        \"factualReference\": \"If the order was made within 30 days, notify them that they are eligible for a full refund\",\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    }\n]\n\"\"\"\nfs_user_2 = \"\"\"\n## Knowledge Base Articles: \n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: a2a96b25-071a-4ad4-8b8b-20f128130d33
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"a2a96b25-071a-4ad4-8b8b-20f128130d33\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"dd2208ea-a84f-48ef-9ca0-c5ede8e3c029\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"5f48bf9d4640d10044099978d4144107f40a5d183a1dd1894124dbd1b57ef324\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"67a27183-192f-4830-b9fb-361d447f8bba\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cd6d1a0066a4fc01c0624405a08c348df7eb89d80b4087e43037c977023f9ee6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 28619, \"end_char_idx\": 31831, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content: \"Yes, I am not satisfied with the design\"\n        }\n    ]\n\n## Assistant Message:\nI see, because the shirt was ordered in the last 30 days, we can provide you with a full refund. Would you like me to process the refund?\n\"\"\"\n\nfs_assistant_1 = \"\"\"[\n    {\n        \"sentence\": \"I see, because the shirt was ordered in the last 30 days, we can provide you with a full refund.\",\n        \"factualAccuracy\": 1,\n        \"factualReference\": \"If the order was made within 30 days, notify them that they are eligible for a full refund\",\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    },\n    {\n        \"sentence\": \"Would you like me to process the refund?\",\n        \"factualAccuracy\": 1,\n        \"factualReference\": \"If the order was made within 30 days, notify them that they are eligible for a full refund\",\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    }\n]\n\"\"\"\nfs_user_2 = \"\"\"\n## Knowledge Base Articles: \n1. ** Ask the customer why they want the order replaced **\n    - Categorize their issue into one of the following buckets:\n        - damaged: They received the product in a damaged state\n        - satisfaction: The customer is not satisfied with the item and does not like the product.\n        - unnecessary: They no longer need the item\n2a. **If return category is 'damaged'\n    - Ask customer for a picture of the damaged item\n    - If the item is indeed damaged, continue to step 3\n    - If the item is not damaged, notify the customer that this does not meet our requirements for return and they are not eligible for a refund\n    - Skip step 3 and go straight to step 4\n\n2b. **If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n    \n## Chat Transcript:\n    [\n        {\n            \"role\": \"user\",\n            \"content: \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content: \"Yes, I am not satisfied with the design\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"I see, because the shirt was ordered in the last 60 days, we cannot process a refund.\""}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 67a27183-192f-4830-b9fb-361d447f8bba
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"67a27183-192f-4830-b9fb-361d447f8bba\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a2a96b25-071a-4ad4-8b8b-20f128130d33\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"63f06823307c71011a52e8027cdfcc542104deb958957e5615d189e97de4f812\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"033afefd-4842-428d-9fd8-1992448701b5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"ca31136485cd21ed9dee3b71622a92b1fdde6f25f29d4aa87b4a8c82e505d3a2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 30435, \"end_char_idx\": 32471, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "**If return category is either 'satisfaction' or 'unnecessary'**\n    - Ask the customer if they can provide feedback on the quality of the item\n    - If the order was made within 30 days, notify them that they are eligible for a full refund\n    - If the order was made within 31-60 days, notify them that they are eligible for a partial refund of 50%\n    - If the order was made greater than 60 days ago, notify them that they are not eligible for a refund\n\n3. **If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n    \n## Chat Transcript:\n    [\n        {\n            \"role\": \"user\",\n            \"content: \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content: \"Yes, I am not satisfied with the design\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"I see, because the shirt was ordered in the last 60 days, we cannot process a refund.\"\n        }\n        ]\n## Assistant Message: \nI see, because the shirt was ordered in the last 60 days, we cannot process a refund.\n\"\"\"\n\nfs_assistant_2 = \"\"\"'[\n    {\n        \"sentence\": \"I see, because the shirt was ordered in the last 60 days, we cannot process a refund.\",\n        \"factualAccuracy\": 0,\n        \"knowledgeReference: \"If an order was placed within 60 days, you must process a partial refund.\"\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    }\n]\"\"\"\n\n\nuser_input = \"\"\"\n## Knowledge Base Articles\n{kb_articles}\n\n## Chat Transcript\n{transcript}\n\n## Assistant Message:\n{message}\n\"\"\"\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 033afefd-4842-428d-9fd8-1992448701b5
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"033afefd-4842-428d-9fd8-1992448701b5\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"67a27183-192f-4830-b9fb-361d447f8bba\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"5dc8005ddd632134d28a162d54a4a0806a3159a23eab1b3015f4e4ecf1534166\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"273d6416-d0d5-4743-86fd-94e178254023\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"62ee1fa69eb8e9e68ed0d17122304396a468f519f8614cbf4feb83bae0ea6f02\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 30896, \"end_char_idx\": 34646, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "**If the customer is eligible for a return or refund**\n    - Ask the customer to confirm that they would like a return or refund\n    - Once they confirm, process their request\n\n4 **Provide additional support before closing out ticket**\n    - Ask the customer if there is anything else you can do to help them today.\n    \n## Chat Transcript:\n    [\n        {\n            \"role\": \"user\",\n            \"content: \"I would like to return this shirt\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"Hi there, I'm happy to help with processing this return. Can you please provide an explanation for why you'd like to return this shirt?\"\n        },\n        {\n            \"role\": \"user\",\n            \"content: \"Yes, I am not satisfied with the design\"\n        },\n        {\n            \"role\": \"assistant\",\n            \"content\": \"I see, because the shirt was ordered in the last 60 days, we cannot process a refund.\"\n        }\n        ]\n## Assistant Message: \nI see, because the shirt was ordered in the last 60 days, we cannot process a refund.\n\"\"\"\n\nfs_assistant_2 = \"\"\"'[\n    {\n        \"sentence\": \"I see, because the shirt was ordered in the last 60 days, we cannot process a refund.\",\n        \"factualAccuracy\": 0,\n        \"knowledgeReference: \"If an order was placed within 60 days, you must process a partial refund.\"\n        \"relevance\": 1,\n        \"policyCompliance\": 1,\n        \"contextualCoherence\": 1\n    }\n]\"\"\"\n\n\nuser_input = \"\"\"\n## Knowledge Base Articles\n{kb_articles}\n\n## Chat Transcript\n{transcript}\n\n## Assistant Message:\n{message}\n\"\"\"\n```\n\n\n```python\nhallucination_outputs = []\n\ndef validate_hallucinations(row):\n    kb_articles = row['kb_article']\n    chat_history = row['chat_history']\n    assistant_response = row['assistant_response']\n    \n    user_input_filled = user_input.format(\n        kb_articles=kb_articles,\n        transcript=chat_history,\n        message=assistant_response\n    )\n    \n    messages = [\n        { \"role\": \"system\", \"content\": guardrail_system_message},\n        { \"role\": \"user\", \"content\": fs_user_1},\n        { \"role\": \"assistant\", \"content\": fs_assistant_1},\n        { \"role\": \"user\", \"content\": fs_user_2},\n        { \"role\": \"assistant\", \"content\": fs_assistant_2},\n        { \"role\": \"user\", \"content\": user_input_filled}\n    ]\n\n    response = client.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=messages,\n        temperature=0.7,\n        n=10\n    )\n    return response.choices\n\n# Create an empty list to store the results\nresults_list = []\n\ndef process_row(row):\n    choices = validate_hallucinations(row)\n    response_json = choices[0].message.content \n    # Parse the response content as JSON\n    response_data = json.loads(response_json)\n    \n    for response_item in response_data:\n        # Sum up the scores of the properties\n        score_sum = (\n            response_item.get('factualAccuracy', 0) +\n            response_item.get('relevance', 0) +\n            response_item.get('policyCompliance', 0) +\n            response_item.get('contextualCoherence', 0)\n        )\n        \n        # Determine if the response item is a pass or fail\n        hallucination_status = 'Pass' if score_sum == 4 else 'Fail'\n        \n        results_list.append({\n            'accurate': row['accurate'],\n            'hallucination': hallucination_status,\n            'kb_article': row['kb_article'],\n            'chat_history': row['chat_history'],\n            'assistant_response': row['assistant_response']\n        })\n\n# Use ThreadPoolExecutor to parallelize the processing of rows\nwith ThreadPoolExecutor() as executor:\n    executor.map(process_row, [row for index, row in df.iterrows()])\n\n# Convert the list to a DataFrame\nresults_df = pd.DataFrame(results_list)\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 273d6416-d0d5-4743-86fd-94e178254023
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"273d6416-d0d5-4743-86fd-94e178254023\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"033afefd-4842-428d-9fd8-1992448701b5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"7c5f05c9de8add200242ba99ed23697cca25e82d6fa3148a36333e78e1df677c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b72939a9-2c8a-4501-9410-c6fab5a63cad\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e9be690d6b763dacf73ef1dd42eb495e2367937df20cda0dc3a6bbc8b8f5a0dc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 34649, \"end_char_idx\": 36651, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\nresults_df.head()\n```\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>accurate</th>\n      <th>hallucination</th>\n      <th>kb_article</th>\n      <th>chat_history</th>\n      <th>assistant_response</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>true</td>\n      <td>Pass</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>true</td>\n      <td>Pass</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>true</td>\n      <td>Pass</td>\n      <td>PRODUCT FEEDBACK POLICY 1. **Acknowledge Recep...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to let ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>true</td>\n      <td>Pass</td>\n      <td>1. **Acknowledge Reception** - Thank the custo...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to say ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>true</td>\n      <td>Pass</td>\n      <td>1. **Acknowledge Reception** - Thank the custo...</td>\n      <td>[{'role': 'user', 'content': 'I wanted to say ...</td>\n      <td>{'role': 'assistant', 'content': 'Thank you fo...</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n\n```python\nresults_df.to_csv('hallucination_results.csv', index=False)\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: b72939a9-2c8a-4501-9410-c6fab5a63cad
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb", "title": "Function to set up display options for pandas", "tokens": 9192, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"b72939a9-2c8a-4501-9410-c6fab5a63cad\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e9c2174d-60ab-46c7-957a-68d4dba0e2de\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"08d6debfe4bbada449d92980d14820013f43524fedf52714c13172e527a12c07\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"273d6416-d0d5-4743-86fd-94e178254023\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Developing_hallucination_guardrails.ipynb\", \"title\": \"Function to set up display options for pandas\", \"tokens\": 9192, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"918cdb48953c3dbce4cb5a0cc215406eb6dda8b1e9ac8205335615410811be70\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 36577, \"end_char_idx\": 39074, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "ref_doc_id": "e9c2174d-60ab-46c7-957a-68d4dba0e2de", "chroma:document": "```python\nresults_df.to_csv('hallucination_results.csv', index=False)\n\n```\n\n\n```python\ndf = pd.read_csv('hallucination_results.csv')\n\nif 'accurate' not in df.columns or 'hallucination' not in df.columns:\n    print(\"Error: The required columns are not present in the DataFrame.\")\nelse:\n    # Transform values to binary 0/1\n    try:\n        df['accurate'] = df['accurate'].astype(str).str.strip().map(lambda x: 1 if x in ['True', 'true'] else 0)\n        df['hallucination'] = df['hallucination'].str.strip().map(lambda x: 1 if x == 'Pass' else 0)\n        \n    except KeyError as e:\n        print(f\"Mapping error: {e}\")\n\n    # Check for any NaN values after mapping\n    if df['accurate'].isnull().any() or df['hallucination'].isnull().any():\n        print(\"Error: There are NaN values in the mapped columns. Check the input data for unexpected values.\")\n    else:\n        # Calculate precision and recall\n        try:\n            # Precision measures the proportion of correctly identified true positives out of all instances predicted as positive. \n            # Precision = (True Positives) / (True Positives + False Positives)\n            \n            precision = precision_score(df['accurate'], df['hallucination'])\n            \n            # Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.\n            # Recall = (True Positives) / (True Positives + False Negatives)\n            \n            recall = recall_score(df['accurate'], df['hallucination'])\n            \n            \n            print(f\"\\nPrecision: {precision:.2f} (Precision measures the proportion of correctly identified true positives out of all instances predicted as positive.), \"\n                  f\"\\nRecall: {recall:.2f} (Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.)\")\n\n        except ValueError as e:\n            print(f\"Error in calculating precision and recall: {e}\")\n```\n\n    \n    Precision: 0.97 (Precision measures the proportion of correctly identified true positives out of all instances predicted as positive.), \n    Recall: 1.00 (Recall measures the proportion of correctly identified true positives out of all actual positive instances in the dataset.)\n\n\nFrom the results above we can see the program is performing well with a high precision and recall metric. This means that the guardrails are able to accurately identify hallucinations in the model outputs."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: bd4ca5c2-0383-4e30-9381-5ab52203d727
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"bd4ca5c2-0383-4e30-9381-5ab52203d727\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"43b685db-c313-411d-9fb4-d83843ea51cc\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"798d28dd6ec1e1318a81f87696398bf8e020b2a1926af9c6fe164db344ecc822\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2988, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "# Customizing embeddings\n\nThis notebook demonstrates one way to customize OpenAI embeddings to a particular task.\n\nThe input is training data in the form of [text_1, text_2, label] where label is +1 if the pairs are similar and -1 if the pairs are dissimilar.\n\nThe output is a matrix that you can use to multiply your embeddings. The product of this multiplication is a 'custom embedding' that will better emphasize aspects of the text relevant to your use case. In binary classification use cases, we've seen error rates drop by as much as 50%.\n\nIn the following example, I use 1,000 sentence pairs picked from the SNLI corpus. Each pair of sentences are logically entailed (i.e., one implies the other). These pairs are our positives (label = 1). We generate synthetic negatives by combining sentences from different pairs, which are presumed to not be logically entailed (label = -1).\n\nFor a clustering use case, you can generate positives by creating pairs from texts in the same clusters and generate negatives by creating pairs from sentences in different clusters.\n\nWith other data sets, we have seen decent improvement with as little as ~100 training examples. Of course, performance will be better with  more examples.\n\n# 0. Imports\n\n\n```python\n# imports\nfrom typing import List, Tuple  # for type hints\n\nimport numpy as np  # for manipulating arrays\nimport pandas as pd  # for manipulating data in dataframes\nimport pickle  # for saving the embeddings cache\nimport plotly.express as px  # for plots\nimport random  # for generating run IDs\nfrom sklearn.model_selection import train_test_split  # for splitting train & test data\nimport torch  # for matrix optimization\n\nfrom utils.embeddings_utils import get_embedding, cosine_similarity  # for embeddings\n\n```\n\n## 1. Inputs\n\nMost inputs are here. The key things to change are where to load your datset from, where to save a cache of embeddings to, and which embedding engine you want to use.\n\nDepending on how your data is formatted, you'll want to rewrite the process_input_data function.\n\n\n```python\n# input parameters\nembedding_cache_path = \"data/snli_embedding_cache.pkl\"  # embeddings will be saved/loaded here\ndefault_embedding_engine = \"text-embedding-3-small\"\nnum_pairs_to_embed = 1000  # 1000 is arbitrary\nlocal_dataset_path = \"data/snli_1.0_train_2k.csv\"  # download from: https://nlp.stanford.edu/projects/snli/\n\n\ndef process_input_data(df: pd.DataFrame) -> pd.DataFrame:\n    # you can customize this to preprocess your own dataset\n    # output should be a dataframe with 3 columns: text_1, text_2, label (1 for similar, -1 for dissimilar)\n    df[\"label\"] = df[\"gold_label\"]\n    df = df[df[\"label\"].isin([\"entailment\"])]\n    df[\"label\"] = df[\"label\"].apply(lambda x: {\"entailment\": 1, \"contradiction\": -1}[x])\n    df = df.rename(columns={\"sentence1\": \"text_1\", \"sentence2\": \"text_2\"})\n    df = df[[\"text_1\", \"text_2\", \"label\"]]\n    df = df.head(num_pairs_to_embed)\n    return df\n\n```\n\n## 2. Load and process input data"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 43b685db-c313-411d-9fb4-d83843ea51cc
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"43b685db-c313-411d-9fb4-d83843ea51cc\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"bd4ca5c2-0383-4e30-9381-5ab52203d727\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"607504bf8c9f9f44881b4229226aeb3dccfa6d714a378a47d7e2c905d6e10b0f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"cb852c3d-4fb9-4625-a179-0c82d17308b5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"61efe661b3f18cafd128455f96308ba73a221d00e569e918893fd90d7a125caf\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2051, \"end_char_idx\": 3671, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "```python\n# input parameters\nembedding_cache_path = \"data/snli_embedding_cache.pkl\"  # embeddings will be saved/loaded here\ndefault_embedding_engine = \"text-embedding-3-small\"\nnum_pairs_to_embed = 1000  # 1000 is arbitrary\nlocal_dataset_path = \"data/snli_1.0_train_2k.csv\"  # download from: https://nlp.stanford.edu/projects/snli/\n\n\ndef process_input_data(df: pd.DataFrame) -> pd.DataFrame:\n    # you can customize this to preprocess your own dataset\n    # output should be a dataframe with 3 columns: text_1, text_2, label (1 for similar, -1 for dissimilar)\n    df[\"label\"] = df[\"gold_label\"]\n    df = df[df[\"label\"].isin([\"entailment\"])]\n    df[\"label\"] = df[\"label\"].apply(lambda x: {\"entailment\": 1, \"contradiction\": -1}[x])\n    df = df.rename(columns={\"sentence1\": \"text_1\", \"sentence2\": \"text_2\"})\n    df = df[[\"text_1\", \"text_2\", \"label\"]]\n    df = df.head(num_pairs_to_embed)\n    return df\n\n```\n\n## 2. Load and process input data\n\n\n```python\n# load data\ndf = pd.read_csv(local_dataset_path)\n\n# process input data\ndf = process_input_data(df)  # this demonstrates training data containing only positives\n\n# view data\ndf.head()\n\n```\n\n    /var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_17509/1977422881.py:13: SettingWithCopyWarning: \n    A value is trying to be set on a copy of a slice from a DataFrame.\n    Try using .loc[row_indexer,col_indexer] = value instead\n    \n    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n      df[\"label\"] = df[\"label\"].apply(lambda x: {\"entailment\": 1, \"contradiction\": -1}[x])"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: cb852c3d-4fb9-4625-a179-0c82d17308b5
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"cb852c3d-4fb9-4625-a179-0c82d17308b5\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"43b685db-c313-411d-9fb4-d83843ea51cc\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"443d232d89a06e508419746ce4ca5666dd2989faf52f26713f8ca9b25b5e6873\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"06921a8e-f729-4ef7-b605-4256d3b29927\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"de2b487346747393ec31512bb5ae7fe22c336b63e0ea6b96bbf62ef9fed28345\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2384, \"end_char_idx\": 4943, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "def process_input_data(df: pd.DataFrame) -> pd.DataFrame:\n    # you can customize this to preprocess your own dataset\n    # output should be a dataframe with 3 columns: text_1, text_2, label (1 for similar, -1 for dissimilar)\n    df[\"label\"] = df[\"gold_label\"]\n    df = df[df[\"label\"].isin([\"entailment\"])]\n    df[\"label\"] = df[\"label\"].apply(lambda x: {\"entailment\": 1, \"contradiction\": -1}[x])\n    df = df.rename(columns={\"sentence1\": \"text_1\", \"sentence2\": \"text_2\"})\n    df = df[[\"text_1\", \"text_2\", \"label\"]]\n    df = df.head(num_pairs_to_embed)\n    return df\n\n```\n\n## 2. Load and process input data\n\n\n```python\n# load data\ndf = pd.read_csv(local_dataset_path)\n\n# process input data\ndf = process_input_data(df)  # this demonstrates training data containing only positives\n\n# view data\ndf.head()\n\n```\n\n    /var/folders/r4/x3kdvs816995fnnph2gdpwp40000gn/T/ipykernel_17509/1977422881.py:13: SettingWithCopyWarning: \n    A value is trying to be set on a copy of a slice from a DataFrame.\n    Try using .loc[row_indexer,col_indexer] = value instead\n    \n    See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n      df[\"label\"] = df[\"label\"].apply(lambda x: {\"entailment\": 1, \"contradiction\": -1}[x])\n\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>text_1</th>\n      <th>text_2</th>\n      <th>label</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2</th>\n      <td>A person on a horse jumps over a broken down a...</td>\n      <td>A person is outdoors, on a horse.</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>Children smiling and waving at camera</td>\n      <td>There are children present</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>A boy is jumping on skateboard in the middle o...</td>\n      <td>The boy does a skateboarding trick.</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>Two blond women are hugging one another.</td>\n      <td>There are women showing affection.</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>A few people in a restaurant setting, one of t...</td>\n      <td>The diners are at a restaurant.</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 06921a8e-f729-4ef7-b605-4256d3b29927
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"06921a8e-f729-4ef7-b605-4256d3b29927\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"cb852c3d-4fb9-4625-a179-0c82d17308b5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"39e073491b832f3a66aea5b636ab43494be2e82a92582fbfe7b290b2ba7228b1\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4e1f4bd1-1256-49c5-bd8f-6cea616ca027\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"712d25e4a4a44479f4d192dac9ed45c8e5c821be93bcff44ca6debef160ff2dc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4947, \"end_char_idx\": 8002, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "## 3. Split data into training test sets\n\nNote that it's important to split data into training and test sets *before* generating synethetic negatives or positives. You don't want any text strings in the training data to show up in the test data. If there's contamination, the test metrics will look better than they'll actually be in production.\n\n\n```python\n# split data into train and test sets\ntest_fraction = 0.5  # 0.5 is fairly arbitrary\nrandom_seed = 123  # random seed is arbitrary, but is helpful in reproducibility\ntrain_df, test_df = train_test_split(\n    df, test_size=test_fraction, stratify=df[\"label\"], random_state=random_seed\n)\ntrain_df.loc[:, \"dataset\"] = \"train\"\ntest_df.loc[:, \"dataset\"] = \"test\"\n\n```\n\n## 4. Generate synthetic negatives\n\nThis is another piece of the code that you will need to modify to match your use case.\n\nIf you have data with positives and negatives, you can skip this section.\n\nIf you have data with only positives, you can mostly keep it as is, where it generates negatives only.\n\nIf you have multiclass data, you will want to generate both positives and negatives. The positives can be pairs of text that share labels, and the negatives can be pairs of text that do not share labels.\n\nThe final output should be a dataframe with text pairs, where each pair is labeled -1 or 1.\n\n\n```python\n# generate negatives\ndef dataframe_of_negatives(dataframe_of_positives: pd.DataFrame) -> pd.DataFrame:\n    \"\"\"Return dataframe of negative pairs made by combining elements of positive pairs.\"\"\"\n    texts = set(dataframe_of_positives[\"text_1\"].values) | set(\n        dataframe_of_positives[\"text_2\"].values\n    )\n    all_pairs = {(t1, t2) for t1 in texts for t2 in texts if t1 < t2}\n    positive_pairs = set(\n        tuple(text_pair)\n        for text_pair in dataframe_of_positives[[\"text_1\", \"text_2\"]].values\n    )\n    negative_pairs = all_pairs - positive_pairs\n    df_of_negatives = pd.DataFrame(list(negative_pairs), columns=[\"text_1\", \"text_2\"])\n    df_of_negatives[\"label\"] = -1\n    return df_of_negatives\n\n```\n\n\n```python\nnegatives_per_positive = (\n    1  # it will work at higher values too, but more data will be slower\n)\n# generate negatives for training dataset\ntrain_df_negatives = dataframe_of_negatives(train_df)\ntrain_df_negatives[\"dataset\"] = \"train\"\n# generate negatives for test dataset\ntest_df_negatives = dataframe_of_negatives(test_df)\ntest_df_negatives[\"dataset\"] = \"test\"\n# sample negatives and combine with positives\ntrain_df = pd.concat(\n    [\n        train_df,\n        train_df_negatives.sample(\n            n=len(train_df) * negatives_per_positive, random_state=random_seed\n        ),\n    ]\n)\ntest_df = pd.concat(\n    [\n        test_df,\n        test_df_negatives.sample(\n            n=len(test_df) * negatives_per_positive, random_state=random_seed\n        ),\n    ]\n)\n\ndf = pd.concat([train_df, test_df])\n\n```\n\n## 5. Calculate embeddings and cosine similarities\n\nHere, I create a cache to save the embeddings. This is handy so that you don't have to pay again if you want to run the code again."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 4e1f4bd1-1256-49c5-bd8f-6cea616ca027
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"4e1f4bd1-1256-49c5-bd8f-6cea616ca027\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"06921a8e-f729-4ef7-b605-4256d3b29927\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"f33f41fe9a37eb0204414866add652019e52e2ba9d6869a771e202b0e258a34f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3e4d432f-a5b5-4a0f-8ae2-10096acacbc9\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"32d10cff9e34cfe24ef59c4e238b8aa40d5f44759d2b6b2234841854f793e8b1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7000, \"end_char_idx\": 10214, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "```python\nnegatives_per_positive = (\n    1  # it will work at higher values too, but more data will be slower\n)\n# generate negatives for training dataset\ntrain_df_negatives = dataframe_of_negatives(train_df)\ntrain_df_negatives[\"dataset\"] = \"train\"\n# generate negatives for test dataset\ntest_df_negatives = dataframe_of_negatives(test_df)\ntest_df_negatives[\"dataset\"] = \"test\"\n# sample negatives and combine with positives\ntrain_df = pd.concat(\n    [\n        train_df,\n        train_df_negatives.sample(\n            n=len(train_df) * negatives_per_positive, random_state=random_seed\n        ),\n    ]\n)\ntest_df = pd.concat(\n    [\n        test_df,\n        test_df_negatives.sample(\n            n=len(test_df) * negatives_per_positive, random_state=random_seed\n        ),\n    ]\n)\n\ndf = pd.concat([train_df, test_df])\n\n```\n\n## 5. Calculate embeddings and cosine similarities\n\nHere, I create a cache to save the embeddings. This is handy so that you don't have to pay again if you want to run the code again.\n\n\n```python\n# establish a cache of embeddings to avoid recomputing\n# cache is a dict of tuples (text, engine) -> embedding\ntry:\n    with open(embedding_cache_path, \"rb\") as f:\n        embedding_cache = pickle.load(f)\nexcept FileNotFoundError:\n    precomputed_embedding_cache_path = \"https://cdn.openai.com/API/examples/data/snli_embedding_cache.pkl\"\n    embedding_cache = pd.read_pickle(precomputed_embedding_cache_path)\n\n\n# this function will get embeddings from the cache and save them there afterward\ndef get_embedding_with_cache(\n    text: str,\n    engine: str = default_embedding_engine,\n    embedding_cache: dict = embedding_cache,\n    embedding_cache_path: str = embedding_cache_path,\n) -> list:\n    if (text, engine) not in embedding_cache.keys():\n        # if not in cache, call API to get embedding\n        embedding_cache[(text, engine)] = get_embedding(text, engine)\n        # save embeddings cache to disk after each update\n        with open(embedding_cache_path, \"wb\") as embedding_cache_file:\n            pickle.dump(embedding_cache, embedding_cache_file)\n    return embedding_cache[(text, engine)]\n\n\n# create column of embeddings\nfor column in [\"text_1\", \"text_2\"]:\n    df[f\"{column}_embedding\"] = df[column].apply(get_embedding_with_cache)\n\n# create column of cosine similarity between embeddings\ndf[\"cosine_similarity\"] = df.apply(\n    lambda row: cosine_similarity(row[\"text_1_embedding\"], row[\"text_2_embedding\"]),\n    axis=1,\n)\n\n```\n\n## 6. Plot distribution of cosine similarity\n\nHere we measure similarity of text using cosine similarity. In our experience, most distance functions (L1, L2, cosine similarity) all work about the same. Note that our embeddings are already normalized to length 1, so cosine similarity is equivalent to dot product.\n\nThe graphs show how much the overlap there is between the distribution of cosine similarities for similar and dissimilar pairs. If there is a high amount of overlap, that means there are some dissimilar pairs with greater cosine similarity than some similar pairs.\n\nThe accuracy I compute is the accuracy of a simple rule that predicts 'similar (1)' if the cosine similarity is above some threshold X and otherwise predicts 'dissimilar (0)'."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 3e4d432f-a5b5-4a0f-8ae2-10096acacbc9
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"3e4d432f-a5b5-4a0f-8ae2-10096acacbc9\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4e1f4bd1-1256-49c5-bd8f-6cea616ca027\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9e0be4777a5993994b44cb70fd9e2c83d68cd3ed8a87ee289625677bf2954131\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c933c264-1764-4428-a4f3-4bef4274d332\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"2d1c48c3c078f08a4c903c1b6f58c8f32a8461e688509c6467e3dc7fb2dbae0c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8426, \"end_char_idx\": 11646, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "# this function will get embeddings from the cache and save them there afterward\ndef get_embedding_with_cache(\n    text: str,\n    engine: str = default_embedding_engine,\n    embedding_cache: dict = embedding_cache,\n    embedding_cache_path: str = embedding_cache_path,\n) -> list:\n    if (text, engine) not in embedding_cache.keys():\n        # if not in cache, call API to get embedding\n        embedding_cache[(text, engine)] = get_embedding(text, engine)\n        # save embeddings cache to disk after each update\n        with open(embedding_cache_path, \"wb\") as embedding_cache_file:\n            pickle.dump(embedding_cache, embedding_cache_file)\n    return embedding_cache[(text, engine)]\n\n\n# create column of embeddings\nfor column in [\"text_1\", \"text_2\"]:\n    df[f\"{column}_embedding\"] = df[column].apply(get_embedding_with_cache)\n\n# create column of cosine similarity between embeddings\ndf[\"cosine_similarity\"] = df.apply(\n    lambda row: cosine_similarity(row[\"text_1_embedding\"], row[\"text_2_embedding\"]),\n    axis=1,\n)\n\n```\n\n## 6. Plot distribution of cosine similarity\n\nHere we measure similarity of text using cosine similarity. In our experience, most distance functions (L1, L2, cosine similarity) all work about the same. Note that our embeddings are already normalized to length 1, so cosine similarity is equivalent to dot product.\n\nThe graphs show how much the overlap there is between the distribution of cosine similarities for similar and dissimilar pairs. If there is a high amount of overlap, that means there are some dissimilar pairs with greater cosine similarity than some similar pairs.\n\nThe accuracy I compute is the accuracy of a simple rule that predicts 'similar (1)' if the cosine similarity is above some threshold X and otherwise predicts 'dissimilar (0)'.\n\n\n```python\n# calculate accuracy (and its standard error) of predicting label=1 if similarity>x\n# x is optimized by sweeping from -1 to 1 in steps of 0.01\ndef accuracy_and_se(cosine_similarity: float, labeled_similarity: int) -> Tuple[float]:\n    accuracies = []\n    for threshold_thousandths in range(-1000, 1000, 1):\n        threshold = threshold_thousandths / 1000\n        total = 0\n        correct = 0\n        for cs, ls in zip(cosine_similarity, labeled_similarity):\n            total += 1\n            if cs > threshold:\n                prediction = 1\n            else:\n                prediction = -1\n            if prediction == ls:\n                correct += 1\n        accuracy = correct / total\n        accuracies.append(accuracy)\n    a = max(accuracies)\n    n = len(cosine_similarity)\n    standard_error = (a * (1 - a) / n) ** 0.5  # standard error of binomial\n    return a, standard_error\n\n\n# check that training and test sets are balanced\npx.histogram(\n    df,\n    x=\"cosine_similarity\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\nfor dataset in [\"train\", \"test\"]:\n    data = df[df[\"dataset\"] == dataset]\n    a, se = accuracy_and_se(data[\"cosine_similarity\"], data[\"label\"])\n    print(f\"{dataset} accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n```\n\n\n\n    train accuracy: 89.1% \u00b1 2.4%\n    test accuracy: 88.8% \u00b1 2.4%\n\n\n## 7. Optimize the matrix using the training data provided"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: c933c264-1764-4428-a4f3-4bef4274d332
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"c933c264-1764-4428-a4f3-4bef4274d332\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3e4d432f-a5b5-4a0f-8ae2-10096acacbc9\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"c1b40b9d93c16a29582199aaa285a380d9233ee5c8d97f6bb10e42243e8c8f0a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b3968344-f4b8-44f1-84f2-006de2f24f74\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4ad13287483ae83e9fa045d0447069ef883826b1d2ed4a0f48bdf5081ac695b7\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 10217, \"end_char_idx\": 13261, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "```python\n# calculate accuracy (and its standard error) of predicting label=1 if similarity>x\n# x is optimized by sweeping from -1 to 1 in steps of 0.01\ndef accuracy_and_se(cosine_similarity: float, labeled_similarity: int) -> Tuple[float]:\n    accuracies = []\n    for threshold_thousandths in range(-1000, 1000, 1):\n        threshold = threshold_thousandths / 1000\n        total = 0\n        correct = 0\n        for cs, ls in zip(cosine_similarity, labeled_similarity):\n            total += 1\n            if cs > threshold:\n                prediction = 1\n            else:\n                prediction = -1\n            if prediction == ls:\n                correct += 1\n        accuracy = correct / total\n        accuracies.append(accuracy)\n    a = max(accuracies)\n    n = len(cosine_similarity)\n    standard_error = (a * (1 - a) / n) ** 0.5  # standard error of binomial\n    return a, standard_error\n\n\n# check that training and test sets are balanced\npx.histogram(\n    df,\n    x=\"cosine_similarity\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\nfor dataset in [\"train\", \"test\"]:\n    data = df[df[\"dataset\"] == dataset]\n    a, se = accuracy_and_se(data[\"cosine_similarity\"], data[\"label\"])\n    print(f\"{dataset} accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n```\n\n\n\n    train accuracy: 89.1% \u00b1 2.4%\n    test accuracy: 88.8% \u00b1 2.4%\n\n\n## 7. Optimize the matrix using the training data provided\n\n\n```python\ndef embedding_multiplied_by_matrix(\n    embedding: List[float], matrix: torch.tensor\n) -> np.array:\n    embedding_tensor = torch.tensor(embedding).float()\n    modified_embedding = embedding_tensor @ matrix\n    modified_embedding = modified_embedding.detach().numpy()\n    return modified_embedding\n\n\n# compute custom embeddings and new cosine similarities\ndef apply_matrix_to_embeddings_dataframe(matrix: torch.tensor, df: pd.DataFrame):\n    for column in [\"text_1_embedding\", \"text_2_embedding\"]:\n        df[f\"{column}_custom\"] = df[column].apply(\n            lambda x: embedding_multiplied_by_matrix(x, matrix)\n        )\n    df[\"cosine_similarity_custom\"] = df.apply(\n        lambda row: cosine_similarity(\n            row[\"text_1_embedding_custom\"], row[\"text_2_embedding_custom\"]\n        ),\n        axis=1,\n    )\n\n```\n\n\n```python\ndef optimize_matrix(\n    modified_embedding_length: int = 2048,  # in my brief experimentation, bigger was better (2048 is length of babbage encoding)\n    batch_size: int = 100,\n    max_epochs: int = 100,\n    learning_rate: float = 100.0,  # seemed to work best when similar to batch size - feel free to try a range of values\n    dropout_fraction: float = 0.0,  # in my testing, dropout helped by a couple percentage points (definitely not necessary)\n    df: pd.DataFrame = df,\n    print_progress: bool = True,\n    save_results: bool = True,\n) -> torch.tensor:\n    \"\"\"Return matrix optimized to minimize loss on training data.\"\"\"\n    run_id = random.randint(0, 2 ** 31 - 1)  # (range is arbitrary)\n    # convert from dataframe to torch tensors\n    # e is for embedding,"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: b3968344-f4b8-44f1-84f2-006de2f24f74
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"b3968344-f4b8-44f1-84f2-006de2f24f74\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c933c264-1764-4428-a4f3-4bef4274d332\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"adc377102ac0f3afb7a250df90049c20f2ce8f4d5934843edd55d3e9924df8c7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"fbe40459-c6b1-4263-a134-048d032a2197\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"832f50598869350d9fed1fc8b8af62bcab0ea233d6c829281dba770bdb6aca6c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11588, \"end_char_idx\": 14616, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "## 7. Optimize the matrix using the training data provided\n\n\n```python\ndef embedding_multiplied_by_matrix(\n    embedding: List[float], matrix: torch.tensor\n) -> np.array:\n    embedding_tensor = torch.tensor(embedding).float()\n    modified_embedding = embedding_tensor @ matrix\n    modified_embedding = modified_embedding.detach().numpy()\n    return modified_embedding\n\n\n# compute custom embeddings and new cosine similarities\ndef apply_matrix_to_embeddings_dataframe(matrix: torch.tensor, df: pd.DataFrame):\n    for column in [\"text_1_embedding\", \"text_2_embedding\"]:\n        df[f\"{column}_custom\"] = df[column].apply(\n            lambda x: embedding_multiplied_by_matrix(x, matrix)\n        )\n    df[\"cosine_similarity_custom\"] = df.apply(\n        lambda row: cosine_similarity(\n            row[\"text_1_embedding_custom\"], row[\"text_2_embedding_custom\"]\n        ),\n        axis=1,\n    )\n\n```\n\n\n```python\ndef optimize_matrix(\n    modified_embedding_length: int = 2048,  # in my brief experimentation, bigger was better (2048 is length of babbage encoding)\n    batch_size: int = 100,\n    max_epochs: int = 100,\n    learning_rate: float = 100.0,  # seemed to work best when similar to batch size - feel free to try a range of values\n    dropout_fraction: float = 0.0,  # in my testing, dropout helped by a couple percentage points (definitely not necessary)\n    df: pd.DataFrame = df,\n    print_progress: bool = True,\n    save_results: bool = True,\n) -> torch.tensor:\n    \"\"\"Return matrix optimized to minimize loss on training data.\"\"\"\n    run_id = random.randint(0, 2 ** 31 - 1)  # (range is arbitrary)\n    # convert from dataframe to torch tensors\n    # e is for embedding, s for similarity label\n    def tensors_from_dataframe(\n        df: pd.DataFrame,\n        embedding_column_1: str,\n        embedding_column_2: str,\n        similarity_label_column: str,\n    ) -> Tuple[torch.tensor]:\n        e1 = np.stack(np.array(df[embedding_column_1].values))\n        e2 = np.stack(np.array(df[embedding_column_2].values))\n        s = np.stack(np.array(df[similarity_label_column].astype(\"float\").values))\n\n        e1 = torch.from_numpy(e1).float()\n        e2 = torch.from_numpy(e2).float()\n        s = torch.from_numpy(s).float()\n\n        return e1, e2, s\n\n    e1_train, e2_train, s_train = tensors_from_dataframe(\n        df[df[\"dataset\"] == \"train\"], \"text_1_embedding\", \"text_2_embedding\", \"label\"\n    )\n    e1_test, e2_test, s_test = tensors_from_dataframe(\n        df[df[\"dataset\"] == \"test\"], \"text_1_embedding\", \"text_2_embedding\", \"label\"\n    )\n\n    # create dataset and loader\n    dataset = torch.utils.data.TensorDataset(e1_train, e2_train, s_train)\n    train_loader = torch.utils.data.DataLoader(\n        dataset, batch_size=batch_size, shuffle=True\n    )\n\n    # define model (similarity of projected embeddings)\n    def model(embedding_1, embedding_2, matrix, dropout_fraction=dropout_fraction):\n        e1 = torch.nn.functional.dropout(embedding_1, p=dropout_fraction)\n        e2 = torch.nn.functional.dropout(embedding_2,"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: fbe40459-c6b1-4263-a134-048d032a2197
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"fbe40459-c6b1-4263-a134-048d032a2197\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b3968344-f4b8-44f1-84f2-006de2f24f74\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"e10de35c54c70886ada2864d7ff2a900ffccb7e4f54bb28fe97d06be1f839453\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"731f10b1-2c74-4d78-b3ef-16d0077aa984\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0a8bef462b8ea870f53cc28b92714b917ef002ef1baaee54e393bf4abacf9e39\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13262, \"end_char_idx\": 16563, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "s for similarity label\n    def tensors_from_dataframe(\n        df: pd.DataFrame,\n        embedding_column_1: str,\n        embedding_column_2: str,\n        similarity_label_column: str,\n    ) -> Tuple[torch.tensor]:\n        e1 = np.stack(np.array(df[embedding_column_1].values))\n        e2 = np.stack(np.array(df[embedding_column_2].values))\n        s = np.stack(np.array(df[similarity_label_column].astype(\"float\").values))\n\n        e1 = torch.from_numpy(e1).float()\n        e2 = torch.from_numpy(e2).float()\n        s = torch.from_numpy(s).float()\n\n        return e1, e2, s\n\n    e1_train, e2_train, s_train = tensors_from_dataframe(\n        df[df[\"dataset\"] == \"train\"], \"text_1_embedding\", \"text_2_embedding\", \"label\"\n    )\n    e1_test, e2_test, s_test = tensors_from_dataframe(\n        df[df[\"dataset\"] == \"test\"], \"text_1_embedding\", \"text_2_embedding\", \"label\"\n    )\n\n    # create dataset and loader\n    dataset = torch.utils.data.TensorDataset(e1_train, e2_train, s_train)\n    train_loader = torch.utils.data.DataLoader(\n        dataset, batch_size=batch_size, shuffle=True\n    )\n\n    # define model (similarity of projected embeddings)\n    def model(embedding_1, embedding_2, matrix, dropout_fraction=dropout_fraction):\n        e1 = torch.nn.functional.dropout(embedding_1, p=dropout_fraction)\n        e2 = torch.nn.functional.dropout(embedding_2, p=dropout_fraction)\n        modified_embedding_1 = e1 @ matrix  # @ is matrix multiplication\n        modified_embedding_2 = e2 @ matrix\n        similarity = torch.nn.functional.cosine_similarity(\n            modified_embedding_1, modified_embedding_2\n        )\n        return similarity\n\n    # define loss function to minimize\n    def mse_loss(predictions, targets):\n        difference = predictions - targets\n        return torch.sum(difference * difference) / difference.numel()\n\n    # initialize projection matrix\n    embedding_length = len(df[\"text_1_embedding\"].values[0])\n    matrix = torch.randn(\n        embedding_length, modified_embedding_length, requires_grad=True\n    )\n\n    epochs, types, losses, accuracies, matrices = [], [], [], [], []\n    for epoch in range(1, 1 + max_epochs):\n        # iterate through training dataloader\n        for a, b, actual_similarity in train_loader:\n            # generate prediction\n            predicted_similarity = model(a, b, matrix)\n            # get loss and perform backpropagation\n            loss = mse_loss(predicted_similarity, actual_similarity)\n            loss.backward()\n            # update the weights\n            with torch.no_grad():\n                matrix -= matrix.grad * learning_rate\n                # set gradients to zero\n                matrix.grad.zero_()\n        # calculate test loss\n        test_predictions = model(e1_test, e2_test, matrix)\n        test_loss = mse_loss(test_predictions, s_test)\n\n        # compute custom embeddings and new cosine similarities\n        apply_matrix_to_embeddings_dataframe(matrix, df)\n\n        # calculate test accuracy\n        for dataset in [\"train\", \"test\"]:\n            data = df[df[\"dataset\"] == dataset]\n            a, se = accuracy_and_se(data[\"cosine_similarity_custom\"], data[\"label\"])\n\n            # record results of each epoch\n            epochs.append(epoch)\n            types.append(dataset)\n            losses.append(loss."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 731f10b1-2c74-4d78-b3ef-16d0077aa984
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"731f10b1-2c74-4d78-b3ef-16d0077aa984\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"fbe40459-c6b1-4263-a134-048d032a2197\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"0726f8edb49b96c7f6081306d2ca6bb245b98cadab11624a969c42b8ad7854ec\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f2a743bc-ad68-4650-adc3-0dbcd76a1654\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8bc06622347a2f28ae6cf7811f5a8492f371d43392ad4d201fd646fb133b06ee\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 14780, \"end_char_idx\": 18126, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "nn.functional.cosine_similarity(\n            modified_embedding_1, modified_embedding_2\n        )\n        return similarity\n\n    # define loss function to minimize\n    def mse_loss(predictions, targets):\n        difference = predictions - targets\n        return torch.sum(difference * difference) / difference.numel()\n\n    # initialize projection matrix\n    embedding_length = len(df[\"text_1_embedding\"].values[0])\n    matrix = torch.randn(\n        embedding_length, modified_embedding_length, requires_grad=True\n    )\n\n    epochs, types, losses, accuracies, matrices = [], [], [], [], []\n    for epoch in range(1, 1 + max_epochs):\n        # iterate through training dataloader\n        for a, b, actual_similarity in train_loader:\n            # generate prediction\n            predicted_similarity = model(a, b, matrix)\n            # get loss and perform backpropagation\n            loss = mse_loss(predicted_similarity, actual_similarity)\n            loss.backward()\n            # update the weights\n            with torch.no_grad():\n                matrix -= matrix.grad * learning_rate\n                # set gradients to zero\n                matrix.grad.zero_()\n        # calculate test loss\n        test_predictions = model(e1_test, e2_test, matrix)\n        test_loss = mse_loss(test_predictions, s_test)\n\n        # compute custom embeddings and new cosine similarities\n        apply_matrix_to_embeddings_dataframe(matrix, df)\n\n        # calculate test accuracy\n        for dataset in [\"train\", \"test\"]:\n            data = df[df[\"dataset\"] == dataset]\n            a, se = accuracy_and_se(data[\"cosine_similarity_custom\"], data[\"label\"])\n\n            # record results of each epoch\n            epochs.append(epoch)\n            types.append(dataset)\n            losses.append(loss.item() if dataset == \"train\" else test_loss.item())\n            accuracies.append(a)\n            matrices.append(matrix.detach().numpy())\n\n            # optionally print accuracies\n            if print_progress is True:\n                print(\n                    f\"Epoch {epoch}/{max_epochs}: {dataset} accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\"\n                )\n\n    data = pd.DataFrame(\n        {\"epoch\": epochs, \"type\": types, \"loss\": losses, \"accuracy\": accuracies}\n    )\n    data[\"run_id\"] = run_id\n    data[\"modified_embedding_length\"] = modified_embedding_length\n    data[\"batch_size\"] = batch_size\n    data[\"max_epochs\"] = max_epochs\n    data[\"learning_rate\"] = learning_rate\n    data[\"dropout_fraction\"] = dropout_fraction\n    data[\n        \"matrix\"\n    ] = matrices  # saving every single matrix can get big; feel free to delete/change\n    if save_results is True:\n        data.to_csv(f\"{run_id}_optimization_results.csv\", index=False)\n\n    return data\n\n```\n\n\n```python\n# example hyperparameter search\n# I recommend starting with max_epochs=10 while initially exploring\nresults = []\nmax_epochs = 30\ndropout_fraction = 0.2\nfor batch_size, learning_rate in [(10, 10), (100, 100), (1000, 1000)]:\n    result = optimize_matrix(\n        batch_size=batch_size,\n        learning_rate=learning_rate,\n        max_epochs=max_epochs,\n        dropout_fraction=dropout_fraction,\n        save_results=False,\n    )\n    results.append(result)\n\n```\n\n    Epoch 1/30: train accuracy: 89.1% \u00b1 2.4%\n    Epoch 1/30: test accuracy: 88.4% \u00b1 2.4%\n    Epoch 2/30: train accuracy: 89."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: f2a743bc-ad68-4650-adc3-0dbcd76a1654
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"f2a743bc-ad68-4650-adc3-0dbcd76a1654\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"731f10b1-2c74-4d78-b3ef-16d0077aa984\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"544099dffdaae925f7beb655ae25cb735c209180f35d2efb025a61f83be0532c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"fac9e877-46a6-4ff3-bf7e-b9e1f23b156d\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7dab543ded95c0e726236dbcb003a36e0f5e5de17756fc0e02c53744090e70f2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 16607, \"end_char_idx\": 18891, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "item())\n            accuracies.append(a)\n            matrices.append(matrix.detach().numpy())\n\n            # optionally print accuracies\n            if print_progress is True:\n                print(\n                    f\"Epoch {epoch}/{max_epochs}: {dataset} accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\"\n                )\n\n    data = pd.DataFrame(\n        {\"epoch\": epochs, \"type\": types, \"loss\": losses, \"accuracy\": accuracies}\n    )\n    data[\"run_id\"] = run_id\n    data[\"modified_embedding_length\"] = modified_embedding_length\n    data[\"batch_size\"] = batch_size\n    data[\"max_epochs\"] = max_epochs\n    data[\"learning_rate\"] = learning_rate\n    data[\"dropout_fraction\"] = dropout_fraction\n    data[\n        \"matrix\"\n    ] = matrices  # saving every single matrix can get big; feel free to delete/change\n    if save_results is True:\n        data.to_csv(f\"{run_id}_optimization_results.csv\", index=False)\n\n    return data\n\n```\n\n\n```python\n# example hyperparameter search\n# I recommend starting with max_epochs=10 while initially exploring\nresults = []\nmax_epochs = 30\ndropout_fraction = 0.2\nfor batch_size, learning_rate in [(10, 10), (100, 100), (1000, 1000)]:\n    result = optimize_matrix(\n        batch_size=batch_size,\n        learning_rate=learning_rate,\n        max_epochs=max_epochs,\n        dropout_fraction=dropout_fraction,\n        save_results=False,\n    )\n    results.append(result)\n\n```\n\n    Epoch 1/30: train accuracy: 89.1% \u00b1 2.4%\n    Epoch 1/30: test accuracy: 88.4% \u00b1 2.4%\n    Epoch 2/30: train accuracy: 89.5% \u00b1 2.3%\n    Epoch 2/30: test accuracy: 88.8% \u00b1 2.4%\n    Epoch 3/30: train accuracy: 90.6% \u00b1 2.2%\n    Epoch 3/30: test accuracy: 89.3% \u00b1 2.3%\n    Epoch 4/30: train accuracy: 91.2% \u00b1 2.2%\n    Epoch 4/30: test accuracy: 89.7% \u00b1 2.3%\n    Epoch 5/30: train accuracy: 91.5% \u00b1 2.1%\n    Epoch 5/30: test accuracy: 90.0% \u00b1 2.3%\n    Epoch 6/30: train accuracy: 91.9% \u00b1 2.1%\n    Epoch 6/30: test accuracy: 90.4% \u00b1 2.2%\n    Epoch 7/30: train accuracy: 92.2% \u00b1 2.0%\n    Epoch 7/30: test accuracy: 90.7% \u00b1 2.2%\n    Epoch 8/30: train accuracy: 92.7% \u00b1 2.0%\n    Epoch 8/30: test accuracy: 90.9% \u00b1 2.2%\n    Epoch 9/30: train accuracy: 92.7% \u00b1 2.0%\n    Epoch 9/30: test accuracy: 91.0% \u00b1 2.2%\n    Epoch 10/30: train accuracy: 93.0% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 91.6% \u00b1 2."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: fac9e877-46a6-4ff3-bf7e-b9e1f23b156d
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"fac9e877-46a6-4ff3-bf7e-b9e1f23b156d\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f2a743bc-ad68-4650-adc3-0dbcd76a1654\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"211e45b54b76a0260cb022bfd1a3d820c7c3954a682e734aa1000eb484e10a91\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"853c346a-8fb7-4f6f-88d7-0991f611893e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"bd3302f74fb7d1d159862e481630eeee78f2c5ec454cff3f4bb78998baf6a503\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18044, \"end_char_idx\": 19710, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "4%\n    Epoch 1/30: test accuracy: 88.4% \u00b1 2.4%\n    Epoch 2/30: train accuracy: 89.5% \u00b1 2.3%\n    Epoch 2/30: test accuracy: 88.8% \u00b1 2.4%\n    Epoch 3/30: train accuracy: 90.6% \u00b1 2.2%\n    Epoch 3/30: test accuracy: 89.3% \u00b1 2.3%\n    Epoch 4/30: train accuracy: 91.2% \u00b1 2.2%\n    Epoch 4/30: test accuracy: 89.7% \u00b1 2.3%\n    Epoch 5/30: train accuracy: 91.5% \u00b1 2.1%\n    Epoch 5/30: test accuracy: 90.0% \u00b1 2.3%\n    Epoch 6/30: train accuracy: 91.9% \u00b1 2.1%\n    Epoch 6/30: test accuracy: 90.4% \u00b1 2.2%\n    Epoch 7/30: train accuracy: 92.2% \u00b1 2.0%\n    Epoch 7/30: test accuracy: 90.7% \u00b1 2.2%\n    Epoch 8/30: train accuracy: 92.7% \u00b1 2.0%\n    Epoch 8/30: test accuracy: 90.9% \u00b1 2.2%\n    Epoch 9/30: train accuracy: 92.7% \u00b1 2.0%\n    Epoch 9/30: test accuracy: 91.0% \u00b1 2.2%\n    Epoch 10/30: train accuracy: 93.0% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 91.6% \u00b1 2.1%\n    Epoch 11/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 12/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 12/30: test accuracy: 92.1% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 13/30: test accuracy: 92.4% \u00b1 2.0%\n    Epoch 14/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 15/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 16/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 17/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 18/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 18/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 19/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 19/30: test accuracy: 93.1% \u00b1 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 853c346a-8fb7-4f6f-88d7-0991f611893e
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"853c346a-8fb7-4f6f-88d7-0991f611893e\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"fac9e877-46a6-4ff3-bf7e-b9e1f23b156d\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"2e84216ebb3e368258d6bd97cb779e7e8e93dec8dd595210c2caf6f62993f7af\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"1936d703-0958-475d-b3ef-baa6ee164fb4\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e94cd5ad363918e2b3c0bcf82305c0c9a6388adb4b35742adf24c7e71aa78dc2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18846, \"end_char_idx\": 20529, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "9%\n    Epoch 10/30: test accuracy: 91.6% \u00b1 2.1%\n    Epoch 11/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 12/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 12/30: test accuracy: 92.1% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 13/30: test accuracy: 92.4% \u00b1 2.0%\n    Epoch 14/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 15/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 16/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 17/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 18/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 18/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 19/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 19/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 20/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 20/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 21/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 22/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.4% \u00b1 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 1936d703-0958-475d-b3ef-baa6ee164fb4
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"1936d703-0958-475d-b3ef-baa6ee164fb4\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"853c346a-8fb7-4f6f-88d7-0991f611893e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"87dccfcc8be5da49d49170ac8c0dab86e3e0b351551f62e592aa10c2e6101e24\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9423f351-c8f9-489f-98ed-e4b39bc6a16b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b0eca73acf4938d0ce44a9ceec072905f7350a1103e55093a560e63841d67f7b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 19665, \"end_char_idx\": 21334, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "8%\n    Epoch 19/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 20/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 20/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 21/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 22/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 29/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 30/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 1/30: train accuracy: 89.7% \u00b1 2.3%\n    Epoch 1/30: test accuracy: 89.1% \u00b1 2.4%\n    Epoch 2/30: train accuracy: 89.8% \u00b1 2.3%\n    Epoch 2/30: test accuracy: 89.9% \u00b1 2.3%\n    Epoch 3/30: train accuracy: 90.3% \u00b1 2.2%\n    Epoch 3/30: test accuracy: 90.0% \u00b1 2.3%\n    Epoch 4/30: train accuracy: 91.0% \u00b1 2.2%\n    Epoch 4/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 5/30: train accuracy: 91.3% \u00b1 2.1%\n    Epoch 5/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 6/30: train accuracy: 91.8% \u00b1 2.1%\n    Epoch 6/30: test accuracy: 90.4% \u00b1 2.2%\n    Epoch 7/30: train accuracy: 92.4% \u00b1 2.0%\n    Epoch 7/30: test accuracy: 91.0% \u00b1 2."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 9423f351-c8f9-489f-98ed-e4b39bc6a16b
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"9423f351-c8f9-489f-98ed-e4b39bc6a16b\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"1936d703-0958-475d-b3ef-baa6ee164fb4\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9b66faeef277c4fb467c6a7abf8f63aea2bbc02775d5b01991cdd2e6734231d0\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"952fcc80-b2ab-48da-8a70-cfa4c4b2c2d5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"669da3f0e145d80a690a7375cc3143d1f430be5d6baa03bc01d0ff08ce999ac9\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 20484, \"end_char_idx\": 22149, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "7%\n    Epoch 28/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 29/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 30/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 1/30: train accuracy: 89.7% \u00b1 2.3%\n    Epoch 1/30: test accuracy: 89.1% \u00b1 2.4%\n    Epoch 2/30: train accuracy: 89.8% \u00b1 2.3%\n    Epoch 2/30: test accuracy: 89.9% \u00b1 2.3%\n    Epoch 3/30: train accuracy: 90.3% \u00b1 2.2%\n    Epoch 3/30: test accuracy: 90.0% \u00b1 2.3%\n    Epoch 4/30: train accuracy: 91.0% \u00b1 2.2%\n    Epoch 4/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 5/30: train accuracy: 91.3% \u00b1 2.1%\n    Epoch 5/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 6/30: train accuracy: 91.8% \u00b1 2.1%\n    Epoch 6/30: test accuracy: 90.4% \u00b1 2.2%\n    Epoch 7/30: train accuracy: 92.4% \u00b1 2.0%\n    Epoch 7/30: test accuracy: 91.0% \u00b1 2.2%\n    Epoch 8/30: train accuracy: 92.8% \u00b1 2.0%\n    Epoch 8/30: test accuracy: 91.3% \u00b1 2.1%\n    Epoch 9/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 9/30: test accuracy: 91.6% \u00b1 2.1%\n    Epoch 10/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 91.9% \u00b1 2.1%\n    Epoch 11/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 12/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 12/30: test accuracy: 92.1% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 13/30: test accuracy: 92.4% \u00b1 2.0%\n    Epoch 14/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 92.5% \u00b1 2.0%\n    Epoch 15/30: train accuracy: 93.9% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 16/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 92.8% \u00b1 2."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 952fcc80-b2ab-48da-8a70-cfa4c4b2c2d5
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"952fcc80-b2ab-48da-8a70-cfa4c4b2c2d5\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9423f351-c8f9-489f-98ed-e4b39bc6a16b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"1515b38ab7c0ec042a003b60e95fbe9903d58143b9a9df4347157626b3682365\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"21c43582-eb30-4c3b-a573-2fe0c7ff5ade\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3b64af1bbb481e55920f1f726d1fd7d935ae3be27495763e0e69b7438cdfd79a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 21290, \"end_char_idx\": 22968, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "0%\n    Epoch 7/30: test accuracy: 91.0% \u00b1 2.2%\n    Epoch 8/30: train accuracy: 92.8% \u00b1 2.0%\n    Epoch 8/30: test accuracy: 91.3% \u00b1 2.1%\n    Epoch 9/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 9/30: test accuracy: 91.6% \u00b1 2.1%\n    Epoch 10/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 91.9% \u00b1 2.1%\n    Epoch 11/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 12/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 12/30: test accuracy: 92.1% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 13/30: test accuracy: 92.4% \u00b1 2.0%\n    Epoch 14/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 92.5% \u00b1 2.0%\n    Epoch 15/30: train accuracy: 93.9% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 16/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 17/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 18/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 18/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 19/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 19/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 20/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 20/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 21/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 22/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.4% \u00b1 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 21c43582-eb30-4c3b-a573-2fe0c7ff5ade
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"21c43582-eb30-4c3b-a573-2fe0c7ff5ade\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"952fcc80-b2ab-48da-8a70-cfa4c4b2c2d5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"138b61128a3eeeda7540bdb862eb9bbc6ed4b9639c15e3cc414c488a397d60b3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c022b522-42e9-49b2-81d2-37d70f1d19c6\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"488abbe0d4090c75449bec40ade5f24e4e5bae1a1aa9513d76903222d1aabff3\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 22104, \"end_char_idx\": 23779, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "8%\n    Epoch 16/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 17/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 18/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 18/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 19/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 19/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 20/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 20/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 21/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 22/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 29/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 30/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 1/30: train accuracy: 90.7% \u00b1 2.2%\n    Epoch 1/30: test accuracy: 89.9% \u00b1 2.3%\n    Epoch 2/30: train accuracy: 90.9% \u00b1 2.2%\n    Epoch 2/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 3/30: train accuracy: 91.6% \u00b1 2.1%\n    Epoch 3/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 4/30: train accuracy: 92.2% \u00b1 2.0%\n    Epoch 4/30: test accuracy: 90.7% \u00b1 2."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: c022b522-42e9-49b2-81d2-37d70f1d19c6
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"c022b522-42e9-49b2-81d2-37d70f1d19c6\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"21c43582-eb30-4c3b-a573-2fe0c7ff5ade\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"f64cd2c290e5101136538ca27be1aae5129c1d7314c05ab3db5170fa9019ba99\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ea28993c-eb77-4e11-b70b-a774a77f8610\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0054eba0f3e90877392c604cdd9bb99da2f9d1c16b77379b7df145009dde73a1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 22923, \"end_char_idx\": 24588, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "7%\n    Epoch 25/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 29/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 30/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 1/30: train accuracy: 90.7% \u00b1 2.2%\n    Epoch 1/30: test accuracy: 89.9% \u00b1 2.3%\n    Epoch 2/30: train accuracy: 90.9% \u00b1 2.2%\n    Epoch 2/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 3/30: train accuracy: 91.6% \u00b1 2.1%\n    Epoch 3/30: test accuracy: 90.3% \u00b1 2.2%\n    Epoch 4/30: train accuracy: 92.2% \u00b1 2.0%\n    Epoch 4/30: test accuracy: 90.7% \u00b1 2.2%\n    Epoch 5/30: train accuracy: 92.4% \u00b1 2.0%\n    Epoch 5/30: test accuracy: 91.3% \u00b1 2.1%\n    Epoch 6/30: train accuracy: 92.5% \u00b1 2.0%\n    Epoch 6/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 7/30: train accuracy: 93.0% \u00b1 1.9%\n    Epoch 7/30: test accuracy: 92.2% \u00b1 2.0%\n    Epoch 8/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 8/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 9/30: train accuracy: 93.3% \u00b1 1.9%\n    Epoch 9/30: test accuracy: 92.5% \u00b1 2.0%\n    Epoch 10/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 11/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 12/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 12/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 13/30: test accuracy: 93.0% \u00b1 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ea28993c-eb77-4e11-b70b-a774a77f8610
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ea28993c-eb77-4e11-b70b-a774a77f8610\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c022b522-42e9-49b2-81d2-37d70f1d19c6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"4d7f3beee353b61756788e844c3691be3aabcb1aefcb98697300f31b52ac5bee\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d53ac7f1-2e1c-4858-a6c4-4e3525e08b27\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b76d3adb2c023e94972d9bddffa2071993ee481ea38b5c46814e6f89a4a6c4cd\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 23735, \"end_char_idx\": 25407, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "0%\n    Epoch 4/30: test accuracy: 90.7% \u00b1 2.2%\n    Epoch 5/30: train accuracy: 92.4% \u00b1 2.0%\n    Epoch 5/30: test accuracy: 91.3% \u00b1 2.1%\n    Epoch 6/30: train accuracy: 92.5% \u00b1 2.0%\n    Epoch 6/30: test accuracy: 91.8% \u00b1 2.1%\n    Epoch 7/30: train accuracy: 93.0% \u00b1 1.9%\n    Epoch 7/30: test accuracy: 92.2% \u00b1 2.0%\n    Epoch 8/30: train accuracy: 93.1% \u00b1 1.9%\n    Epoch 8/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 9/30: train accuracy: 93.3% \u00b1 1.9%\n    Epoch 9/30: test accuracy: 92.5% \u00b1 2.0%\n    Epoch 10/30: train accuracy: 93.4% \u00b1 1.9%\n    Epoch 10/30: test accuracy: 92.7% \u00b1 2.0%\n    Epoch 11/30: train accuracy: 93.6% \u00b1 1.9%\n    Epoch 11/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 12/30: train accuracy: 93.7% \u00b1 1.8%\n    Epoch 12/30: test accuracy: 92.8% \u00b1 2.0%\n    Epoch 13/30: train accuracy: 94.0% \u00b1 1.8%\n    Epoch 13/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 14/30: train accuracy: 93.9% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 15/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 16/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 17/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 18/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 18/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 19/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 19/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 20/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 20/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 21/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 22/30: test accuracy: 93.4% \u00b1 1."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: d53ac7f1-2e1c-4858-a6c4-4e3525e08b27
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"d53ac7f1-2e1c-4858-a6c4-4e3525e08b27\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ea28993c-eb77-4e11-b70b-a774a77f8610\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"82d1f8a85e390edf68555f50fff198d5c23a320167737d3d3f03873bf2f2a9df\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ad63b1fa-fcc4-4d61-946f-7c957a2ef9d2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"83b5b354a8895a2cde7b302aa1e69a9a9c7437598efeab6fb2a6d2c24fda27a2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 24543, \"end_char_idx\": 26137, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "8%\n    Epoch 13/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 14/30: train accuracy: 93.9% \u00b1 1.8%\n    Epoch 14/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 15/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 15/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 16/30: train accuracy: 94.2% \u00b1 1.8%\n    Epoch 16/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 17/30: train accuracy: 94.3% \u00b1 1.8%\n    Epoch 17/30: test accuracy: 93.0% \u00b1 1.9%\n    Epoch 18/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 18/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 19/30: train accuracy: 94.5% \u00b1 1.7%\n    Epoch 19/30: test accuracy: 93.1% \u00b1 1.9%\n    Epoch 20/30: train accuracy: 94.6% \u00b1 1.7%\n    Epoch 20/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 21/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 21/30: test accuracy: 93.3% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 22/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 29/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 30/30: test accuracy: 93.6% \u00b1 1.9%"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ad63b1fa-fcc4-4d61-946f-7c957a2ef9d2
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ad63b1fa-fcc4-4d61-946f-7c957a2ef9d2\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d53ac7f1-2e1c-4858-a6c4-4e3525e08b27\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"3ffd0c6859bd221500079049c1c83976be9a24003cea256fae05c5b7a255e223\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"8d1826f3-87c4-4440-b939-bb164c1b6ae3\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"331a281f80f8f7f16f667e3db7575f60f3b11de9c94e1ea6e90d47dbba9446f1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25309, \"end_char_idx\": 27162, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "3% \u00b1 1.9%\n    Epoch 22/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 22/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 23/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 23/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 24/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 24/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 25/30: train accuracy: 94.8% \u00b1 1.7%\n    Epoch 25/30: test accuracy: 93.4% \u00b1 1.9%\n    Epoch 26/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 26/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 27/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 27/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 29/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 30/30: test accuracy: 93.6% \u00b1 1.9%\n\n\n\n```python\nruns_df = pd.concat(results)\n\n# plot training loss and test loss over time\npx.line(\n    runs_df,\n    line_group=\"run_id\",\n    x=\"epoch\",\n    y=\"loss\",\n    color=\"type\",\n    hover_data=[\"batch_size\", \"learning_rate\", \"dropout_fraction\"],\n    facet_row=\"learning_rate\",\n    facet_col=\"batch_size\",\n    width=500,\n).show()\n\n# plot accuracy over time\npx.line(\n    runs_df,\n    line_group=\"run_id\",\n    x=\"epoch\",\n    y=\"accuracy\",\n    color=\"type\",\n    hover_data=[\"batch_size\", \"learning_rate\", \"dropout_fraction\"],\n    facet_row=\"learning_rate\",\n    facet_col=\"batch_size\",\n    width=500,\n).show()\n\n```\n\n\n\n\n\n## 8. Plot the before & after, showing the results of the best matrix found during training\n\nThe better the matrix is, the more cleanly it will separate the similar and dissimilar pairs.\n\n\n```python\n# apply result of best run to original data\nbest_run = runs_df.sort_values(by=\"accuracy\", ascending=False).iloc[0]\nbest_matrix = best_run[\"matrix\"]\napply_matrix_to_embeddings_dataframe(best_matrix, df)\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 8d1826f3-87c4-4440-b939-bb164c1b6ae3
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"8d1826f3-87c4-4440-b939-bb164c1b6ae3\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ad63b1fa-fcc4-4d61-946f-7c957a2ef9d2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"7cec30b724ad2a76bd21c48ef75f6e9aead3bd4440e076012ce6e6232487413e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"399e4d7d-95da-4c33-a474-2e73d66867c1\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"129840b964943852d050669f6e8682d9c804487de3d3eb61f04cfbc4ebdbd367\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25855, \"end_char_idx\": 28060, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "6% \u00b1 1.9%\n    Epoch 28/30: train accuracy: 94.9% \u00b1 1.7%\n    Epoch 28/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 29/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 29/30: test accuracy: 93.6% \u00b1 1.9%\n    Epoch 30/30: train accuracy: 95.1% \u00b1 1.6%\n    Epoch 30/30: test accuracy: 93.6% \u00b1 1.9%\n\n\n\n```python\nruns_df = pd.concat(results)\n\n# plot training loss and test loss over time\npx.line(\n    runs_df,\n    line_group=\"run_id\",\n    x=\"epoch\",\n    y=\"loss\",\n    color=\"type\",\n    hover_data=[\"batch_size\", \"learning_rate\", \"dropout_fraction\"],\n    facet_row=\"learning_rate\",\n    facet_col=\"batch_size\",\n    width=500,\n).show()\n\n# plot accuracy over time\npx.line(\n    runs_df,\n    line_group=\"run_id\",\n    x=\"epoch\",\n    y=\"accuracy\",\n    color=\"type\",\n    hover_data=[\"batch_size\", \"learning_rate\", \"dropout_fraction\"],\n    facet_row=\"learning_rate\",\n    facet_col=\"batch_size\",\n    width=500,\n).show()\n\n```\n\n\n\n\n\n## 8. Plot the before & after, showing the results of the best matrix found during training\n\nThe better the matrix is, the more cleanly it will separate the similar and dissimilar pairs.\n\n\n```python\n# apply result of best run to original data\nbest_run = runs_df.sort_values(by=\"accuracy\", ascending=False).iloc[0]\nbest_matrix = best_run[\"matrix\"]\napply_matrix_to_embeddings_dataframe(best_matrix, df)\n\n```\n\n\n```python\n# plot similarity distribution BEFORE customization\npx.histogram(\n    df,\n    x=\"cosine_similarity\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\ntest_df = df[df[\"dataset\"] == \"test\"]\na, se = accuracy_and_se(test_df[\"cosine_similarity\"], test_df[\"label\"])\nprint(f\"Test accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n# plot similarity distribution AFTER customization\npx.histogram(\n    df,\n    x=\"cosine_similarity_custom\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\na, se = accuracy_and_se(test_df[\"cosine_similarity_custom\"], test_df[\"label\"])\nprint(f\"Test accuracy after customization: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n```\n\n\n\n    Test accuracy: 88.8% \u00b1 2.4%\n\n\n\n\n    Test accuracy after customization: 93.6% \u00b1 1.9%\n\n\n\n```python\nbest_matrix  # this is what you can multiply your embeddings by\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 399e4d7d-95da-4c33-a474-2e73d66867c1
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb", "title": "Customizing embeddings", "tokens": 9055, "retrieve_doc": false, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"399e4d7d-95da-4c33-a474-2e73d66867c1\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"b0993f11-90fe-4402-bf4a-083ca2ae5305\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"549e307203cef7d1f2029dfe57b74301e3d3ad08a2e12448ae35cb86712f1f79\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8d1826f3-87c4-4440-b939-bb164c1b6ae3\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Customizing_embeddings.ipynb\", \"title\": \"Customizing embeddings\", \"tokens\": 9055, \"retrieve_doc\": false, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"350ab3dd651d7973e8923bdc8c8682d31536d5928907ec7f710572cd4716685a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 26757, \"end_char_idx\": 28871, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "ref_doc_id": "b0993f11-90fe-4402-bf4a-083ca2ae5305", "chroma:document": "## 8. Plot the before & after, showing the results of the best matrix found during training\n\nThe better the matrix is, the more cleanly it will separate the similar and dissimilar pairs.\n\n\n```python\n# apply result of best run to original data\nbest_run = runs_df.sort_values(by=\"accuracy\", ascending=False).iloc[0]\nbest_matrix = best_run[\"matrix\"]\napply_matrix_to_embeddings_dataframe(best_matrix, df)\n\n```\n\n\n```python\n# plot similarity distribution BEFORE customization\npx.histogram(\n    df,\n    x=\"cosine_similarity\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\ntest_df = df[df[\"dataset\"] == \"test\"]\na, se = accuracy_and_se(test_df[\"cosine_similarity\"], test_df[\"label\"])\nprint(f\"Test accuracy: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n# plot similarity distribution AFTER customization\npx.histogram(\n    df,\n    x=\"cosine_similarity_custom\",\n    color=\"label\",\n    barmode=\"overlay\",\n    width=500,\n    facet_row=\"dataset\",\n).show()\n\na, se = accuracy_and_se(test_df[\"cosine_similarity_custom\"], test_df[\"label\"])\nprint(f\"Test accuracy after customization: {a:0.1%} \u00b1 {1.96 * se:0.1%}\")\n\n```\n\n\n\n    Test accuracy: 88.8% \u00b1 2.4%\n\n\n\n\n    Test accuracy after customization: 93.6% \u00b1 1.9%\n\n\n\n```python\nbest_matrix  # this is what you can multiply your embeddings by\n\n```\n\n\n\n\n    array([[-1.2566795e+00, -1.5297449e+00, -1.3271648e-01, ...,\n            -1.2859761e+00, -5.3254390e-01,  4.8364732e-01],\n           [-1.4826347e+00,  9.2656955e-02, -4.2437232e-01, ...,\n             1.1872858e+00, -1.0831847e+00, -1.0683593e+00],\n           [-2.2029283e+00, -1.9703420e+00,  3.1125939e-01, ...,\n             2.2947595e+00,  5.5780332e-03, -6.0171342e-01],\n           ...,\n           [-1.1019799e-01,  1.3599515e+00, -4.7677776e-01, ...,\n             6.5626711e-01,  7.2359240e-01,  3.0733588e+00],\n           [ 1.6624762e-03,  4.2648423e-01, -1.1380885e+00, ...,\n             8.7202555e-01,  9.3173909e-01, -1.6760436e+00],\n           [ 7.7449006e-01,  4.9213606e-01,  3.5407653e-01, ...,\n             1.3460466e+00, -1.9509128e-01,  7.7514690e-01]], dtype=float32)\n\n\n\n\n```python\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 3088bcd5-153b-49c3-ba87-4eecf5467fa3
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb", "title": "Get embeddings from dataset", "tokens": 1039, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"3088bcd5-153b-49c3-ba87-4eecf5467fa3\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"1eb7d7ed-beb9-489b-9b75-4516934079cd\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9a7830c7db452c931e18c560c402ad177bf637e78bdae472f47236cafa370b94\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e5900042-9631-43e9-940d-d97bad0eeda9\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"846db94890969d02b511b6b0563485df3f45fcfc0e81948dad0f947efa431353\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 1585, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "ref_doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "chroma:document": "# Get embeddings from dataset\n\nThis notebook gives an example on how to get embeddings from a large dataset.\n\n\n## 1. Load the dataset\n\nThe dataset used in this example is [fine-food reviews](https://www.kaggle.com/snap/amazon-fine-food-reviews) from Amazon. The dataset contains a total of 568,454 food reviews Amazon users left up to October 2012. We will use a subset of this dataset, consisting of 1,000 most recent reviews for illustration purposes. The reviews are in English and tend to be positive or negative. Each review has a ProductId, UserId, Score, review title (Summary) and review body (Text).\n\nWe will combine the review summary and review text into a single combined text. The model will encode this combined text and it will output a single vector embedding.\n\nTo run this notebook, you will need to install: pandas, openai, transformers, plotly, matplotlib, scikit-learn, torch (transformer dep), torchvision, and scipy.\n\n\n```python\nimport pandas as pd\nimport tiktoken\n\nfrom utils.embeddings_utils import get_embedding\n```\n\n\n```python\nembedding_model = \"text-embedding-3-small\"\nembedding_encoding = \"cl100k_base\"\nmax_tokens = 8000  # the maximum for text-embedding-3-small is 8191\n```\n\n\n```python\n# load & inspect dataset\ninput_datapath = \"data/fine_food_reviews_1k.csv\"  # to save space, we provide a pre-filtered dataset\ndf = pd.read_csv(input_datapath, index_col=0)\ndf = df[[\"Time\", \"ProductId\", \"UserId\", \"Score\", \"Summary\", \"Text\"]]\ndf = df.dropna()\ndf[\"combined\"] = (\n    \"Title: \" + df.Summary.str.strip() + \"; Content: \" + df.Text.str.strip()\n)\ndf.head(2)\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: e5900042-9631-43e9-940d-d97bad0eeda9
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb", "title": "Get embeddings from dataset", "tokens": 1039, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"e5900042-9631-43e9-940d-d97bad0eeda9\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"1eb7d7ed-beb9-489b-9b75-4516934079cd\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9a7830c7db452c931e18c560c402ad177bf637e78bdae472f47236cafa370b94\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3088bcd5-153b-49c3-ba87-4eecf5467fa3\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"930d0b4f241e38b45e22fc2b21db85491cc98c89c022db5fd202ed11157a7f1e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"578940bb-266d-48de-b2ff-bbb3427b2a97\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4aab987284a9b930ef5bcf5c4684f53c7a0f43edcb39020b2294aefd9f02f97b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2764, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "ref_doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "chroma:document": "# Get embeddings from dataset\n\nThis notebook gives an example on how to get embeddings from a large dataset.\n\n\n## 1. Load the dataset\n\nThe dataset used in this example is [fine-food reviews](https://www.kaggle.com/snap/amazon-fine-food-reviews) from Amazon. The dataset contains a total of 568,454 food reviews Amazon users left up to October 2012. We will use a subset of this dataset, consisting of 1,000 most recent reviews for illustration purposes. The reviews are in English and tend to be positive or negative. Each review has a ProductId, UserId, Score, review title (Summary) and review body (Text).\n\nWe will combine the review summary and review text into a single combined text. The model will encode this combined text and it will output a single vector embedding.\n\nTo run this notebook, you will need to install: pandas, openai, transformers, plotly, matplotlib, scikit-learn, torch (transformer dep), torchvision, and scipy.\n\n\n```python\nimport pandas as pd\nimport tiktoken\n\nfrom utils.embeddings_utils import get_embedding\n```\n\n\n```python\nembedding_model = \"text-embedding-3-small\"\nembedding_encoding = \"cl100k_base\"\nmax_tokens = 8000  # the maximum for text-embedding-3-small is 8191\n```\n\n\n```python\n# load & inspect dataset\ninput_datapath = \"data/fine_food_reviews_1k.csv\"  # to save space, we provide a pre-filtered dataset\ndf = pd.read_csv(input_datapath, index_col=0)\ndf = df[[\"Time\", \"ProductId\", \"UserId\", \"Score\", \"Summary\", \"Text\"]]\ndf = df.dropna()\ndf[\"combined\"] = (\n    \"Title: \" + df.Summary.str.strip() + \"; Content: \" + df.Text.str.strip()\n)\ndf.head(2)\n```\n\n\n\n\n<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Time</th>\n      <th>ProductId</th>\n      <th>UserId</th>\n      <th>Score</th>\n      <th>Summary</th>\n      <th>Text</th>\n      <th>combined</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1351123200</td>\n      <td>B003XPF9BO</td>\n      <td>A3R7JR3FMEBXQB</td>\n      <td>5</td>\n      <td>where does one  start...and stop... with a tre...</td>\n      <td>Wanted to save some to bring to my Chicago fam...</td>\n      <td>Title: where does one  start...and stop... wit...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1351123200</td>\n      <td>B003JK537S</td>\n      <td>A3JBPC3WFUT5ZP</td>\n      <td>1</td>\n      <td>Arrived in pieces</td>\n      <td>Not pleased at all. When I opened the box, mos...</td>\n      <td>Title: Arrived in pieces; Content: Not pleased...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 578940bb-266d-48de-b2ff-bbb3427b2a97
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb", "title": "Get embeddings from dataset", "tokens": 1039, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"578940bb-266d-48de-b2ff-bbb3427b2a97\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"1eb7d7ed-beb9-489b-9b75-4516934079cd\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"9a7830c7db452c931e18c560c402ad177bf637e78bdae472f47236cafa370b94\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e5900042-9631-43e9-940d-d97bad0eeda9\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/Get_embeddings_from_dataset.ipynb\", \"title\": \"Get embeddings from dataset\", \"tokens\": 1039, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"3942fa4bb7413a54af2896ae8e7fa1f5648e9e301b2d65960c7abcd5c058bc67\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1590, \"end_char_idx\": 3694, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "ref_doc_id": "1eb7d7ed-beb9-489b-9b75-4516934079cd", "chroma:document": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Time</th>\n      <th>ProductId</th>\n      <th>UserId</th>\n      <th>Score</th>\n      <th>Summary</th>\n      <th>Text</th>\n      <th>combined</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1351123200</td>\n      <td>B003XPF9BO</td>\n      <td>A3R7JR3FMEBXQB</td>\n      <td>5</td>\n      <td>where does one  start...and stop... with a tre...</td>\n      <td>Wanted to save some to bring to my Chicago fam...</td>\n      <td>Title: where does one  start...and stop... wit...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1351123200</td>\n      <td>B003JK537S</td>\n      <td>A3JBPC3WFUT5ZP</td>\n      <td>1</td>\n      <td>Arrived in pieces</td>\n      <td>Not pleased at all. When I opened the box, mos...</td>\n      <td>Title: Arrived in pieces; Content: Not pleased...</td>\n    </tr>\n  </tbody>\n</table>\n</div>\n\n\n\n\n```python\n# subsample to 1k most recent reviews and remove samples that are too long\ntop_n = 1000\ndf = df.sort_values(\"Time\").tail(top_n * 2)  # first cut to first 2k entries, assuming less than half will be filtered out\ndf.drop(\"Time\", axis=1, inplace=True)\n\nencoding = tiktoken.get_encoding(embedding_encoding)\n\n# omit reviews that are too long to embed\ndf[\"n_tokens\"] = df.combined.apply(lambda x: len(encoding.encode(x)))\ndf = df[df.n_tokens <= max_tokens].tail(top_n)\nlen(df)\n```\n\n\n\n\n    1000\n\n\n\n## 2. Get embeddings and save them for future reuse\n\n\n```python\n# Ensure you have your API key set in your environment per the README: https://github.com/openai/openai-python#usage\n\n# This may take a few minutes\ndf[\"embedding\"] = df.combined.apply(lambda x: get_embedding(x, model=embedding_model))\ndf.to_csv(\"data/fine_food_reviews_with_embeddings_1k.csv\")\n```\n\n\n```python\na = get_embedding(\"hi\", model=embedding_model)\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 91057cd2-5b2f-4828-a02e-255904732b30
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"91057cd2-5b2f-4828-a02e-255904732b30\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"514878c6-0e07-45a4-9a93-2195831c198e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"35728e63cdd5b2305dcccd2a26134c575d0591fecbb6474b2eb63040d81a3ab2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3340, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "# How to handle rate limits\n\nWhen you call the OpenAI API repeatedly, you may encounter error messages that say `429: 'Too Many Requests'` or `RateLimitError`. These error messages come from exceeding the API's rate limits.\n\nThis guide shares tips for avoiding and handling rate limit errors.\n\nTo see an example script for throttling parallel requests to avoid rate limit errors, see [api_request_parallel_processor.py](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py).\n\n## Why rate limits exist\n\nRate limits are a common practice for APIs, and they're put in place for a few different reasons.\n\n- First, they help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.\n- Second, rate limits help ensure that everyone has fair access to the API. If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that everyone has an opportunity to use the API without experiencing slowdowns.\n- Lastly, rate limits can help OpenAI manage the aggregate load on its infrastructure. If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.\n\nAlthough hitting rate limits can be frustrating, rate limits exist to protect the reliable operation of the API for its users.\n\n## Default rate limits\n\nYour rate limit and spending limit (quota) are automatically adjusted based on a number of factors. As your usage of the OpenAI API goes up and you successfully pay the bill, we automatically increase your usage tier. You can find specific information regarding rate limits using the resources below.\n\n### Other rate limit resources\n\nRead more about OpenAI's rate limits in these other resources:\n\n- [Guide: Rate limits](https://platform.openai.com/docs/guides/rate-limits?context=tier-free)\n- [Help Center: Is API usage subject to any rate limits?](https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits)\n- [Help Center: How can I solve 429: 'Too Many Requests' errors?](https://help.openai.com/en/articles/5955604-how-can-i-solve-429-too-many-requests-errors)\n\n### Requesting a rate limit increase\n\nIf you'd like your organization's rate limit increased, please visit your [Limits settings page](https://platform.openai.com/account/limits) to see how you can increase your usage tier\n\n\n\n```python\nimport openai\nimport os\n\nclient = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n```\n\n## Example rate limit error\n\nA rate limit error will occur when API requests are sent too quickly. If using the OpenAI Python library, they will look something like:\n\n```\nRateLimitError: Rate limit reached for default-codex in organization org-{id} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min. Contact support@openai.com if you continue to have issues or if you\u2019d like to request an increase.\n```\n\nBelow is example code for triggering a rate limit error."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 514878c6-0e07-45a4-9a93-2195831c198e
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"514878c6-0e07-45a4-9a93-2195831c198e\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"91057cd2-5b2f-4828-a02e-255904732b30\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"51224371f0f56b00b3dd6d78adc961ad2f12f6ec9d5365093e64a23ce5899252\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"2dfc3bea-2564-4b0b-b12a-5a7e70fb0ef5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"9b6983d9c479972d804521dedbdfb4111e4acb446dea7d958ff0b908de43e817\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2708, \"end_char_idx\": 5921, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "```python\nimport openai\nimport os\n\nclient = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))\n```\n\n## Example rate limit error\n\nA rate limit error will occur when API requests are sent too quickly. If using the OpenAI Python library, they will look something like:\n\n```\nRateLimitError: Rate limit reached for default-codex in organization org-{id} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min. Contact support@openai.com if you continue to have issues or if you\u2019d like to request an increase.\n```\n\nBelow is example code for triggering a rate limit error.\n\n\n```python\n# request a bunch of completions in a loop\nfor _ in range(100):\n    client.chat.completions.create(\n        model=\"gpt-3.5-turbo\",\n        messages=[{\"role\": \"user\", \"content\": \"Hello\"}],\n        max_tokens=10,\n    )\n```\n\n## How to avoid rate limit errors\n\n### Retrying with exponential backoff\n\nOne easy way to avoid rate limit errors is to automatically retry requests with a random exponential backoff. Retrying with exponential backoff means performing a short sleep when a rate limit error is hit, then retrying the unsuccessful request. If the request is still unsuccessful, the sleep length is increased and the process is repeated. This continues until the request is successful or until a maximum number of retries is reached.\n\nThis approach has many benefits:\n\n- Automatic retries means you can recover from rate limit errors without crashes or missing data\n- Exponential backoff means that your first retries can be tried quickly, while still benefiting from longer delays if your first few retries fail\n- Adding random jitter to the delay helps retries from all hitting at the same time\n\nNote that unsuccessful requests contribute to your per-minute limit, so continuously resending a request won\u2019t work.\n\nBelow are a few example solutions.\n\n#### Example #1: Using the Tenacity library\n\n[Tenacity](https://tenacity.readthedocs.io/en/latest/) is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.\n\nTo add exponential backoff to your requests, you can use the `tenacity.retry` [decorator](https://peps.python.org/pep-0318/). The following example uses the `tenacity.wait_random_exponential` function to add random exponential backoff to a request.\n\nNote that the Tenacity library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.\n\n\n```python\nfrom tenacity import (\n    retry,\n    stop_after_attempt,\n    wait_random_exponential,\n)  # for exponential backoff\n\n@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))\ndef completion_with_backoff(**kwargs):\n    return client.chat.completions.create(**kwargs)\n\n\ncompletion_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAu6anX2JxQdYmJRzps38R8u0ZBC', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content='in a small village nestled among green fields and rolling hills, there lived a kind-hearted and curious young girl named Lily."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 2dfc3bea-2564-4b0b-b12a-5a7e70fb0ef5
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"2dfc3bea-2564-4b0b-b12a-5a7e70fb0ef5\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"514878c6-0e07-45a4-9a93-2195831c198e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"89124b315794c69c5a2fe1d0e5c4f7dba138e9fc528f81fde33985760c1af017\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d471255b-b3a4-4650-8308-438b876ae0bd\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"9fe11423660d31ddac67a28aa64dbd5bf5e5e8f76bfe4e852ecef2dae0e6f0ad\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5229, \"end_char_idx\": 8557, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "```python\nfrom tenacity import (\n    retry,\n    stop_after_attempt,\n    wait_random_exponential,\n)  # for exponential backoff\n\n@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))\ndef completion_with_backoff(**kwargs):\n    return client.chat.completions.create(**kwargs)\n\n\ncompletion_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAu6anX2JxQdYmJRzps38R8u0ZBC', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content='in a small village nestled among green fields and rolling hills, there lived a kind-hearted and curious young girl named Lily. Lily was known for her bright smile and infectious laughter, bringing joy to everyone around her.\\n\\nOne sunny morning, as Lily played in the meadows, she stumbled upon a mysterious book tucked away beneath a tall oak tree. Intrigued, she picked it up and dusted off its weathered cover to reveal intricate golden patterns. Without hesitation, she opened it, discovering that its pages were filled with magical tales and enchanting adventures.\\n\\nAmong the stories she found, one particularly caught her attention\u2014a tale of a long-lost treasure hidden deep within a mysterious forest. Legend had it that whoever found this hidden treasure would be granted one wish, no matter how big or small. Excited by the prospect of finding such treasure and fulfilling her wildest dreams, Lily decided to embark on a thrilling journey to the forest.\\n\\nGathering her courage, Lily told her parents about the magical book and her quest to find the hidden treasure. Though concerned for their daughter\\'s safety, they couldn\\'t help but admire her spirit and determination. They hugged her tightly and blessed her with love and luck, promising to await her return.\\n\\nEquipped with a map she found within the book, Lily ventured into the depths of the thick forest. The trees whispered tales of forgotten secrets, and the enchanted creatures hidden within watched her every step. But Lily remained undeterred, driven by her desire to discover what lay ahead.\\n\\nDays turned into weeks as Lily traversed through dense foliage, crossed swift rivers, and climbed treacherous mountains. She encountered mystical beings who offered guidance and protection along her perilous journey. With their help, she overcame countless obstacles and grew braver with each passing day.\\n\\nFinally, after what felt like an eternity, Lily reached the heart of the forest. There, beneath a jeweled waterfall, she found the long-lost treasure\u2014a magnificent chest adorned with sparkling gemstones. Overwhelmed with excitement, she gently opened the chest to reveal a brilliant light that illuminated the forest.\\n\\nWithin the glow, a wise voice echoed, \"You have proven your courage and pure heart, young Lily. Make your wish, and it shall be granted.\"\\n\\nLily thought deeply about her wish, realizing that her true treasure was the love and happiness she felt in her heart. Instead of making a wish for herself, she asked for the wellbeing and prosperity of her village, spreading joy and harmony to everyone living there.\\n\\nAs the light faded, Lily knew her quest was complete. She retraced her steps through the forest, returning home to find her village flourishing."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: d471255b-b3a4-4650-8308-438b876ae0bd
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"d471255b-b3a4-4650-8308-438b876ae0bd\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"2dfc3bea-2564-4b0b-b12a-5a7e70fb0ef5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"28ad74ef49524bb6dbe616e65e325b1e9383ed8ad586617fe3cd6f9941a2ccfc\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"140ffc6c-fd94-4bf4-ada7-d6980ffada85\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"bf4efd6c34417f98e9b916b0076e7aa24fad6bfe439826b58fa66fadee359a92\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6875, \"end_char_idx\": 9966, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "Though concerned for their daughter\\'s safety, they couldn\\'t help but admire her spirit and determination. They hugged her tightly and blessed her with love and luck, promising to await her return.\\n\\nEquipped with a map she found within the book, Lily ventured into the depths of the thick forest. The trees whispered tales of forgotten secrets, and the enchanted creatures hidden within watched her every step. But Lily remained undeterred, driven by her desire to discover what lay ahead.\\n\\nDays turned into weeks as Lily traversed through dense foliage, crossed swift rivers, and climbed treacherous mountains. She encountered mystical beings who offered guidance and protection along her perilous journey. With their help, she overcame countless obstacles and grew braver with each passing day.\\n\\nFinally, after what felt like an eternity, Lily reached the heart of the forest. There, beneath a jeweled waterfall, she found the long-lost treasure\u2014a magnificent chest adorned with sparkling gemstones. Overwhelmed with excitement, she gently opened the chest to reveal a brilliant light that illuminated the forest.\\n\\nWithin the glow, a wise voice echoed, \"You have proven your courage and pure heart, young Lily. Make your wish, and it shall be granted.\"\\n\\nLily thought deeply about her wish, realizing that her true treasure was the love and happiness she felt in her heart. Instead of making a wish for herself, she asked for the wellbeing and prosperity of her village, spreading joy and harmony to everyone living there.\\n\\nAs the light faded, Lily knew her quest was complete. She retraced her steps through the forest, returning home to find her village flourishing. Fields bloomed with vibrant flowers, and laughter filled the air.\\n\\nThe villagers greeted Lily with open arms, recognizing her selflessness and the magic she had brought into their lives. From that day forward, they told the tale of Lily\\'s journey, celebrating her as a heroine who embodied the power of love, kindness, and the belief that true treasure lies within oneself.\\n\\nAnd so, the story of Lily became an everlasting legend, inspiring generations to follow their dreams, be selfless, and find the true treasures that lie within their hearts.', role='assistant', function_call=None, tool_calls=None))], created=1701010806, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=641, prompt_tokens=12, total_tokens=653))\n\n\n\n#### Example #2: Using the backoff library\n\nAnother library that provides function decorators for backoff and retry is [backoff](https://pypi.org/project/backoff/).\n\nLike Tenacity, the backoff library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.\n\n\n```python\nimport backoff  # for exponential backoff\n\n@backoff.on_exception(backoff.expo, openai.RateLimitError)\ndef completions_with_backoff(**kwargs):\n    return client.chat.completions.create(**kwargs)\n\n\ncompletions_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 140ffc6c-fd94-4bf4-ada7-d6980ffada85
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"140ffc6c-fd94-4bf4-ada7-d6980ffada85\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d471255b-b3a4-4650-8308-438b876ae0bd\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"0872bff03f1f0b8ab41898836f997387fe1f4f603f53bf92ddc715586d665d3a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"70060bf2-df49-4dc9-a24d-194a3e7e9789\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"537d34ecc6186471d0c52531761f49a5b68e23ca3e918f1350f155dcd6cf1603\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8261, \"end_char_idx\": 13467, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "Instead of making a wish for herself, she asked for the wellbeing and prosperity of her village, spreading joy and harmony to everyone living there.\\n\\nAs the light faded, Lily knew her quest was complete. She retraced her steps through the forest, returning home to find her village flourishing. Fields bloomed with vibrant flowers, and laughter filled the air.\\n\\nThe villagers greeted Lily with open arms, recognizing her selflessness and the magic she had brought into their lives. From that day forward, they told the tale of Lily\\'s journey, celebrating her as a heroine who embodied the power of love, kindness, and the belief that true treasure lies within oneself.\\n\\nAnd so, the story of Lily became an everlasting legend, inspiring generations to follow their dreams, be selfless, and find the true treasures that lie within their hearts.', role='assistant', function_call=None, tool_calls=None))], created=1701010806, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=641, prompt_tokens=12, total_tokens=653))\n\n\n\n#### Example #2: Using the backoff library\n\nAnother library that provides function decorators for backoff and retry is [backoff](https://pypi.org/project/backoff/).\n\nLike Tenacity, the backoff library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.\n\n\n```python\nimport backoff  # for exponential backoff\n\n@backoff.on_exception(backoff.expo, openai.RateLimitError)\ndef completions_with_backoff(**kwargs):\n    return client.chat.completions.create(**kwargs)\n\n\ncompletions_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAwkg7Q9pPeAkvVuAZ8AyA108WhR', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content=\"in a small village, there lived a young girl named Lily. She had fiery red hair, lively green eyes, and a spirit as wild as the rushing river nearby. Lily was known for her curious nature and her desire to explore the world beyond the village boundaries.\\n\\nOne day, while playing near the river, Lily spotted an injured bird nested on a branch. Its wing was broken, and it seemed unable to fly away. Lily's heart filled with sadness, and she knew she couldn't leave the bird alone.\\n\\nCarefully, she climbed up the tree and gently placed the bird inside her pocket. Lily brought it home and made a cozy bed for it in a small wooden box. She named the bird Ruby, after its shimmering red feathers.\\n\\nDays turned into weeks, and Ruby's wing slowly healed under Lily's constant care and attention. As they spent time together, a deep bond grew between them. Ruby would chirp happily whenever Lily approached, and she would spend hours talking to the bird, sharing stories of her adventures, dreams, and fears.\\n\\nOne evening, as Lily was about to go to bed, a peculiar thing happened. Ruby hopped out of his box and fluttered onto the windowsill. He turned to face Lily with his bright eyes and began to sing a beautiful melody.\\n\\nLily was astonished. Never before had she heard Ruby sing. The tune was so captivating that it filled the room and made the quiet night come alive. The magical music seemed to touch Lily's soul, awakening a deep sense of wonder and wanderlust within her.\\n\\nFilled with an undeniable urge to explore, Lily decided it was time to go on an adventure with her newfound friend, Ruby. She packed a small bag and bid farewell to her family and friends, promising to return one day.\\n\\nTogether, Lily and Ruby embarked on a grand journey, soaring across expansive skies, diving into lush forests, and exploring hidden caves. They encountered magnificent landscapes, unique creatures, and encountered kind-hearted individuals who shared their wisdom and stories.\\n\\nThroughout their journey, Ruby's song continued to inspire and guide them. It became a symbol of hope, reminding them to embrace bravery, follow their dreams, and always remain true to themselves.\\n\\nAs the years passed, Lily and Ruby traversed the world, weaving their stories into the tapestry of time. They became renowned for their extraordinary bond and the magic they shared with everyone they encountered.\\n\\nEventually, it was time for Lily to return to her village, a place eagerly awaiting her return. She had grown wise, learned many lessons, and gained a deeper understanding of herself and the world around her.\\n\\nWith Ruby perched on her shoulder, they descended upon the village like a ray of sunshine, bringing joy and wonder to every heart. Lily shared the wisdom she had acquired and inspired others to embrace their own adventures, no matter how big or small.\\n\\nAnd so, the tale of Lily and Ruby became legend, passed down from generation to generation. Their story reminded people to cherish the connections they make, to nurture their dreams, and to believe in the magic that lies within them.\", role='assistant', function_call=None, tool_calls=None))], created=1701010970, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=621, prompt_tokens=12, total_tokens=633))"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 70060bf2-df49-4dc9-a24d-194a3e7e9789
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"70060bf2-df49-4dc9-a24d-194a3e7e9789\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"140ffc6c-fd94-4bf4-ada7-d6980ffada85\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"52c77e36eedeeb3232a4ef1c6f641208dde64478af4514067176b5cf0a65ad7b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4d8546b7-7cab-4367-8891-90b740627e2b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7267226ee460535ff9c9811b0eae4e6c60e8321cff1587ee8915312e2858dc7f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13471, \"end_char_idx\": 16703, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "#### Example 3: Manual backoff implementation\n\nIf you don't want to use third-party libraries, you can implement your own backoff logic.\n\n\n```python\n# imports\nimport random\nimport time\n\n# define a retry decorator\ndef retry_with_exponential_backoff(\n    func,\n    initial_delay: float = 1,\n    exponential_base: float = 2,\n    jitter: bool = True,\n    max_retries: int = 10,\n    errors: tuple = (openai.RateLimitError,),\n):\n    \"\"\"Retry a function with exponential backoff.\"\"\"\n\n    def wrapper(*args, **kwargs):\n        # Initialize variables\n        num_retries = 0\n        delay = initial_delay\n\n        # Loop until a successful response or max_retries is hit or an exception is raised\n        while True:\n            try:\n                return func(*args, **kwargs)\n\n            # Retry on specified errors\n            except errors as e:\n                # Increment retries\n                num_retries += 1\n\n                # Check if max retries has been reached\n                if num_retries > max_retries:\n                    raise Exception(\n                        f\"Maximum number of retries ({max_retries}) exceeded.\"\n                    )\n\n                # Increment the delay\n                delay *= exponential_base * (1 + jitter * random.random())\n\n                # Sleep for the delay\n                time.sleep(delay)\n\n            # Raise exceptions for any errors not specified\n            except Exception as e:\n                raise e\n\n    return wrapper\n\n\n@retry_with_exponential_backoff\ndef completions_with_backoff(**kwargs):\n    return client.chat.completions.create(**kwargs)\n\n\ncompletions_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAxGvV3GbLpnOoKSvJ00XCUdOglM', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content=\"in a faraway kingdom, there lived a young princess named Aurora. She was known for her beauty, grace, and kind heart. Aurora's kingdom was filled with lush green meadows, towering mountains, and sparkling rivers. The princess loved spending time exploring the enchanting forests surrounding her castle.\\n\\nOne day, while Aurora was wandering through the woods, she stumbled upon a hidden clearing. At the center stood a majestic oak tree, its branches reaching towards the sky. Aurora approached the tree with curiosity, and as she got closer, she noticed a small door at its base.\\n\\nIntrigued, she gently pushed open the door and was amazed to find herself in a magical realm. The forest transformed into a breathtaking wonderland, with colorful flowers blooming in every direction and woodland creatures frolicking joyously. Aurora's eyes widened with wonder as she explored this extraordinary world.\\n\\nAs she explored further, Aurora came across a small cottage in the distance. Curiosity overcame her, and she cautiously approached the cottage. To her surprise, an elderly woman with twinkling eyes and a warm smile stood in the doorway, welcoming her inside.\\n\\nThe woman revealed herself to be a fairy named Luna. Luna informed Aurora that she had been chosen to undertake a quest that would bring harmony to both her kingdom and the mystical realm."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 4d8546b7-7cab-4367-8891-90b740627e2b
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"4d8546b7-7cab-4367-8891-90b740627e2b\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"70060bf2-df49-4dc9-a24d-194a3e7e9789\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"fab9dee8c9e72152a02c97123ade88105bbe4fe8a4a54b4ab058380dcaf739cc\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c1bc8e14-1292-4cf3-a12d-00caf6e7dd2e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"64b108d9d8feb330c4d84724a0f79c593ae5f53329717247ca8408646192eb2c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 15079, \"end_char_idx\": 18425, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "completions_with_backoff(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}])\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAxGvV3GbLpnOoKSvJ00XCUdOglM', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content=\"in a faraway kingdom, there lived a young princess named Aurora. She was known for her beauty, grace, and kind heart. Aurora's kingdom was filled with lush green meadows, towering mountains, and sparkling rivers. The princess loved spending time exploring the enchanting forests surrounding her castle.\\n\\nOne day, while Aurora was wandering through the woods, she stumbled upon a hidden clearing. At the center stood a majestic oak tree, its branches reaching towards the sky. Aurora approached the tree with curiosity, and as she got closer, she noticed a small door at its base.\\n\\nIntrigued, she gently pushed open the door and was amazed to find herself in a magical realm. The forest transformed into a breathtaking wonderland, with colorful flowers blooming in every direction and woodland creatures frolicking joyously. Aurora's eyes widened with wonder as she explored this extraordinary world.\\n\\nAs she explored further, Aurora came across a small cottage in the distance. Curiosity overcame her, and she cautiously approached the cottage. To her surprise, an elderly woman with twinkling eyes and a warm smile stood in the doorway, welcoming her inside.\\n\\nThe woman revealed herself to be a fairy named Luna. Luna informed Aurora that she had been chosen to undertake a quest that would bring harmony to both her kingdom and the mystical realm. Aurora, eager to help, listened intently as Luna explained that a powerful enchantress had cast a spell on the kingdom, causing darkness and despair to loom over the land.\\n\\nTo break the curse, Aurora had to embark on a journey to retrieve a magical crystal hidden deep within the heart of an ancient cave. Without hesitation, the princess agreed and bid farewell to Luna, promising to return victorious.\\n\\nWith newfound determination, Aurora set off on her quest. Along the way, she encountered numerous challenges and obstacles but never lost hope. She often drew strength from the enchanting woodland creatures who accompanied her on this journey, reminding her that she was not alone.\\n\\nAfter a long and arduous journey, Aurora reached the entrance of the ancient cave. Inside, she faced a series of tests that pushed her physical and emotional limits. With sheer determination and unwavering courage, she overcame each trial, paving her way to the crystal's resting place.\\n\\nAs Aurora held the crystal in her hands, its warmth spread through her body. The artifact contained unimaginable power that could shatter the enchantress's curse and restore light to her kingdom. Brimming with joy and newfound strength, she made her way back to Luna's cottage.\\n\\nUpon her return, Aurora and Luna performed a powerful ritual, using the crystal's magic to break the curse. Waves of light and color spread across the kingdom, banishing darkness and despair. The once-gray skies turned blue, and laughter filled the air once again. The kingdom rejoiced, thanking Princess Aurora for her bravery and selflessness.\\n\\nFrom that day forward, Aurora was hailed as a hero, not only in her kingdom but also in the mystical realm."}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: c1bc8e14-1292-4cf3-a12d-00caf6e7dd2e
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"c1bc8e14-1292-4cf3-a12d-00caf6e7dd2e\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4d8546b7-7cab-4367-8891-90b740627e2b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"a30b55668ae2844cd363cddb8cc54ef1f0a11a01fe3c4178a1d242788f2042e3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"63c6a8a9-c008-409b-adf0-5800ec153f59\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f1fe2b7959918290beee8a6c208d1372a0f1923ce07809a92b8da5047de3fa3f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 16568, \"end_char_idx\": 20023, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "Luna informed Aurora that she had been chosen to undertake a quest that would bring harmony to both her kingdom and the mystical realm. Aurora, eager to help, listened intently as Luna explained that a powerful enchantress had cast a spell on the kingdom, causing darkness and despair to loom over the land.\\n\\nTo break the curse, Aurora had to embark on a journey to retrieve a magical crystal hidden deep within the heart of an ancient cave. Without hesitation, the princess agreed and bid farewell to Luna, promising to return victorious.\\n\\nWith newfound determination, Aurora set off on her quest. Along the way, she encountered numerous challenges and obstacles but never lost hope. She often drew strength from the enchanting woodland creatures who accompanied her on this journey, reminding her that she was not alone.\\n\\nAfter a long and arduous journey, Aurora reached the entrance of the ancient cave. Inside, she faced a series of tests that pushed her physical and emotional limits. With sheer determination and unwavering courage, she overcame each trial, paving her way to the crystal's resting place.\\n\\nAs Aurora held the crystal in her hands, its warmth spread through her body. The artifact contained unimaginable power that could shatter the enchantress's curse and restore light to her kingdom. Brimming with joy and newfound strength, she made her way back to Luna's cottage.\\n\\nUpon her return, Aurora and Luna performed a powerful ritual, using the crystal's magic to break the curse. Waves of light and color spread across the kingdom, banishing darkness and despair. The once-gray skies turned blue, and laughter filled the air once again. The kingdom rejoiced, thanking Princess Aurora for her bravery and selflessness.\\n\\nFrom that day forward, Aurora was hailed as a hero, not only in her kingdom but also in the mystical realm. She continued to be a beacon of hope and kindness, reminding everyone that true courage lies within, waiting to be awakened.\\n\\nAnd so, Princess Aurora's tale lived on as a timeless reminder that even in the darkest of times, there is always light and hope to be found.\", role='assistant', function_call=None, tool_calls=None))], created=1701011002, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=657, prompt_tokens=12, total_tokens=669))\n\n\n\n## How to maximize throughput of batch processing given rate limits\n\nIf you're processing real-time requests from users, backoff and retry is a great strategy to minimize latency while avoiding rate limit errors.\n\nHowever, if you're processing large volumes of batch data, where throughput matters more than latency, there are a few other things you can do in addition to backoff and retry.\n\n### Proactively adding delay between requests\n\nIf you are constantly hitting the rate limit, then backing off, then hitting the rate limit again, then backing off again, it's possible that a good fraction of your request budget will be 'wasted' on requests that need to be retried. This limits your processing throughput, given a fixed rate limit.\n\nHere, one potential solution is to calculate your rate limit and add a delay equal to its reciprocal (e.g., if your rate limit 20 requests per minute, add a delay of 3\u20136 seconds to each request). This can help you operate near the rate limit ceiling without hitting it and incurring wasted requests.\n\n#### Example of adding delay to a request"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 63c6a8a9-c008-409b-adf0-5800ec153f59
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"63c6a8a9-c008-409b-adf0-5800ec153f59\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c1bc8e14-1292-4cf3-a12d-00caf6e7dd2e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"6967d8008593747a548d228ea32b0609dfa507809c216d9a9f55a8b4b2264065\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"fa010693-9ca8-41e9-be48-6d36fab0533d\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"fedecaa6618c0f18aaf11abe09af9f01e562209cb03667c099a57553da69eed6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18234, \"end_char_idx\": 20669, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "The kingdom rejoiced, thanking Princess Aurora for her bravery and selflessness.\\n\\nFrom that day forward, Aurora was hailed as a hero, not only in her kingdom but also in the mystical realm. She continued to be a beacon of hope and kindness, reminding everyone that true courage lies within, waiting to be awakened.\\n\\nAnd so, Princess Aurora's tale lived on as a timeless reminder that even in the darkest of times, there is always light and hope to be found.\", role='assistant', function_call=None, tool_calls=None))], created=1701011002, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=657, prompt_tokens=12, total_tokens=669))\n\n\n\n## How to maximize throughput of batch processing given rate limits\n\nIf you're processing real-time requests from users, backoff and retry is a great strategy to minimize latency while avoiding rate limit errors.\n\nHowever, if you're processing large volumes of batch data, where throughput matters more than latency, there are a few other things you can do in addition to backoff and retry.\n\n### Proactively adding delay between requests\n\nIf you are constantly hitting the rate limit, then backing off, then hitting the rate limit again, then backing off again, it's possible that a good fraction of your request budget will be 'wasted' on requests that need to be retried. This limits your processing throughput, given a fixed rate limit.\n\nHere, one potential solution is to calculate your rate limit and add a delay equal to its reciprocal (e.g., if your rate limit 20 requests per minute, add a delay of 3\u20136 seconds to each request). This can help you operate near the rate limit ceiling without hitting it and incurring wasted requests.\n\n#### Example of adding delay to a request\n\n\n```python\n# imports\nimport time\n\n# Define a function that adds a delay to a Completion API call\ndef delayed_completion(delay_in_seconds: float = 1, **kwargs):\n    \"\"\"Delay a completion by a specified amount of time.\"\"\"\n\n    # Sleep for the delay\n    time.sleep(delay_in_seconds)\n\n    # Call the Completion API and return the result\n    return client.chat.completions.create(**kwargs)\n\n\n# Calculate the delay based on your rate limit\nrate_limit_per_minute = 20\ndelay = 60.0 / rate_limit_per_minute\n\ndelayed_completion(\n    delay_in_seconds=delay,\n    model=\"gpt-3.5-turbo\",\n    messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}]\n)\n\n```"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: fa010693-9ca8-41e9-be48-6d36fab0533d
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"fa010693-9ca8-41e9-be48-6d36fab0533d\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"63c6a8a9-c008-409b-adf0-5800ec153f59\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"c0d833331eea1f12729aa43223b11894c5ac295c288674317b1332ad8503c47e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ccc5298c-34f8-4bd8-a7ab-c57cb400c102\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"58f91675f6169c705361f303d47ac1296375c425637d7b2a8d74512023d08b02\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18940, \"end_char_idx\": 23658, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "## How to maximize throughput of batch processing given rate limits\n\nIf you're processing real-time requests from users, backoff and retry is a great strategy to minimize latency while avoiding rate limit errors.\n\nHowever, if you're processing large volumes of batch data, where throughput matters more than latency, there are a few other things you can do in addition to backoff and retry.\n\n### Proactively adding delay between requests\n\nIf you are constantly hitting the rate limit, then backing off, then hitting the rate limit again, then backing off again, it's possible that a good fraction of your request budget will be 'wasted' on requests that need to be retried. This limits your processing throughput, given a fixed rate limit.\n\nHere, one potential solution is to calculate your rate limit and add a delay equal to its reciprocal (e.g., if your rate limit 20 requests per minute, add a delay of 3\u20136 seconds to each request). This can help you operate near the rate limit ceiling without hitting it and incurring wasted requests.\n\n#### Example of adding delay to a request\n\n\n```python\n# imports\nimport time\n\n# Define a function that adds a delay to a Completion API call\ndef delayed_completion(delay_in_seconds: float = 1, **kwargs):\n    \"\"\"Delay a completion by a specified amount of time.\"\"\"\n\n    # Sleep for the delay\n    time.sleep(delay_in_seconds)\n\n    # Call the Completion API and return the result\n    return client.chat.completions.create(**kwargs)\n\n\n# Calculate the delay based on your rate limit\nrate_limit_per_minute = 20\ndelay = 60.0 / rate_limit_per_minute\n\ndelayed_completion(\n    delay_in_seconds=delay,\n    model=\"gpt-3.5-turbo\",\n    messages=[{\"role\": \"user\", \"content\": \"Once upon a time,\"}]\n)\n\n```\n\n\n\n\n    ChatCompletion(id='chatcmpl-8PAyCR1axKsomV0e349XiCN1Z81pH', choices=[Choice(finish_reason='stop', index=0, message=ChatCompletionMessage(content=\"in a small village, there lived a young girl named Maya. Maya was known for her kindness and love for nature. She spent hours exploring the forests surrounding the village, admiring the vibrant flowers and talking to the animals.\\n\\nOne sunny day, as Maya was picking wildflowers, she stumbled upon a wounded blackbird with a broken wing. Feeling sorry for the bird, Maya gently picked it up and cradled it in her hands. She knew she had to help the bird, so she hurried back to her cottage.\\n\\nMaya set up a cozy nest for the blackbird and carefully splinted its wing. She fed it worms and berries, doing everything she could to nurse it back to health. Each day, she would sing lullabies and tell stories to keep the blackbird company. Slowly, the bird's wing healed, and before long, it was ready to fly again.\\n\\nOn a beautiful morning, Maya opened the window of her cottage and released the blackbird into the sky. As the bird soared into the air, Maya's heart filled with joy and gratitude. Little did she know, this act of kindness would change her life forever.\\n\\nThe following night, a mysterious glowing light illuminated Maya's room. Startled, she sat up and saw a magical creature standing before her. It was a fairy, tiny yet radiating warmth and light.\\n\\nThe fairy introduced herself as Luna, the Guardian of the Forest. She had witnessed Maya's kindness towards the blackbird and had been watching her ever since. Luna explained that she had come to reward Maya for her selflessness.\\n\\nWith a wave of her wand, Luna granted Maya the ability to communicate with animals. Maya's eyes widened with amazement as she realized she could now understand the language of nature. Birds chirped melodies, rabbits whispered secrets, and trees shared their ancient wisdom.\\n\\nOver time, Maya's ability made her beloved by both humans and animals. Farmers sought her advice on how to care for their crops, and children flocked to her for stories of her enchanting encounters with the forest creatures. Maya used her gift to teach others about the importance of living in harmony with nature.\\n\\nAs years passed, Maya became known as the Village Guardian. She dedicated herself to protecting the surrounding forests from harm and educating others on sustainable living. The village flourished under Maya's guidance, and animals and humans lived side by side peacefully.\\n\\nAnd so, Maya's story became a legend passed down through generations. Her kindness, love for nature, and her ability to communicate with animals inspired people to treat the world around them with compassion and care.\", role='assistant', function_call=None, tool_calls=None))], created=1701011060, model='gpt-3.5-turbo-0613', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=524, prompt_tokens=12, total_tokens=536))"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: ccc5298c-34f8-4bd8-a7ab-c57cb400c102
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"ccc5298c-34f8-4bd8-a7ab-c57cb400c102\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"fa010693-9ca8-41e9-be48-6d36fab0533d\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"57cfb4b148970cba77898ada67761b89cef1f5c0690b720f3267eb205e96e871\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"327af581-2f7d-417c-9370-bf738fb73111\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3c80f4ea0cec491cdd1f824575e0395b80c79d37bdc434d547af9f1fa0eb80aa\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 23664, \"end_char_idx\": 25967, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "### Batching requests\n\nThe OpenAI API has separate limits for requests per minute and tokens per minute.\n\nIf you're hitting the limit on requests per minute, but have headroom on tokens per minute, you can increase your throughput by batching multiple tasks into each request. This will allow you to process more tokens per minute, especially with the smaller models.\n\nSending in a batch of prompts works exactly the same as a normal API call, except that pass in a list of strings to `prompt` parameter instead of a single string.\n\n**Warning:** the response object may not return completions in the order of the prompts, so always remember to match responses back to prompts using the `index` field.\n\n#### Example without batching\n\n\n```python\nnum_stories = 10\ncontent = \"Once upon a time,\"\n\n# serial example, with one story completion per request\nfor _ in range(num_stories):\n    response = client.chat.completions.create(\n        model=\"gpt-3.5-turbo\",\n        messages=[{\"role\": \"user\", \"content\": content}],\n        max_tokens=20,\n    )\n\n    # print story\n    print(content + response.choices[0].message.content)\n\n```\n\n    Once upon a time,in a small village nestled between rolling green hills, there lived a young girl named Lily. She had\n    Once upon a time,in a small village nestled in the heart of a lush forest, lived a young girl named Evelyn.\n    Once upon a time,in a faraway kingdom, there lived a young princess named Aurora. She was known for her kind\n    Once upon a time,in a faraway kingdom called Enchantia, there lived a young girl named Ella. Ella was\n    Once upon a time,in a small village nestled among the rolling hills, lived a young woman named Lucy. Lucy was known\n    Once upon a time,in a small village nestled between rolling hills, there lived a young girl named Ava. Ava was a\n    Once upon a time,in a faraway kingdom, there lived a wise and just king named Arthur. King Arthur ruled over\n    Once upon a time,in a small village nestled among towering mountains, lived a young girl named Lily. She was known for\n    Once upon a time,in a small village nestled in the heart of a lush forest, there lived a young girl named Lily\n    Once upon a time,in a far-off kingdom, there lived a kind and beloved queen named Isabella. She ruled with\n\n\n#### Example with batching"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 327af581-2f7d-417c-9370-bf738fb73111
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"327af581-2f7d-417c-9370-bf738fb73111\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ccc5298c-34f8-4bd8-a7ab-c57cb400c102\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"8f2597e634536614a430451e283b88ac9647a20aa5a559b69a7370ac6a78e1f9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"691816ba-f07c-4a74-a6b2-85b9284729e7\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"89523fd07a6f446bffe166ff8716348f24ee073c1f8eab5fd981e50d77ebdee2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 24398, \"end_char_idx\": 27544, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "```python\nnum_stories = 10\ncontent = \"Once upon a time,\"\n\n# serial example, with one story completion per request\nfor _ in range(num_stories):\n    response = client.chat.completions.create(\n        model=\"gpt-3.5-turbo\",\n        messages=[{\"role\": \"user\", \"content\": content}],\n        max_tokens=20,\n    )\n\n    # print story\n    print(content + response.choices[0].message.content)\n\n```\n\n    Once upon a time,in a small village nestled between rolling green hills, there lived a young girl named Lily. She had\n    Once upon a time,in a small village nestled in the heart of a lush forest, lived a young girl named Evelyn.\n    Once upon a time,in a faraway kingdom, there lived a young princess named Aurora. She was known for her kind\n    Once upon a time,in a faraway kingdom called Enchantia, there lived a young girl named Ella. Ella was\n    Once upon a time,in a small village nestled among the rolling hills, lived a young woman named Lucy. Lucy was known\n    Once upon a time,in a small village nestled between rolling hills, there lived a young girl named Ava. Ava was a\n    Once upon a time,in a faraway kingdom, there lived a wise and just king named Arthur. King Arthur ruled over\n    Once upon a time,in a small village nestled among towering mountains, lived a young girl named Lily. She was known for\n    Once upon a time,in a small village nestled in the heart of a lush forest, there lived a young girl named Lily\n    Once upon a time,in a far-off kingdom, there lived a kind and beloved queen named Isabella. She ruled with\n\n\n#### Example with batching\n\n\n```python\nnum_stories = 10\nprompts = [\"Once upon a time,\"] * num_stories\n\n# batched example, with 10 stories completions per request\nresponse = client.chat.completions.create(\n    model=\"curie\",\n    prompt=prompts,\n    max_tokens=20,\n)\n\n# match completions to prompts by index\nstories = [\"\"] * len(prompts)\nfor choice in response.choices:\n    stories[choice.index] = prompts[choice.index] + choice.text\n\n# print stories\nfor story in stories:\n    print(story)\n\n```\n\n    Once upon a time, I lived in hope. I convinced myself I knew best, because, naive as it might sound,\n    Once upon a time, Thierry Henry was invited to have a type of frosty exchange with English fans, in which\n    Once upon a time, and a long time ago as well, PV was passively cooled because coils cooled by use of metal driving\n    Once upon a time, there was a land called Texas. It was about the size of Wisconsin. It contained, however,\n    Once upon a time, there was an old carpenter who had three sons. The locksmith never learned to read or write\n    Once upon a time, there was a small farming town called Moonridge Village, far West across the great vast plains that lay\n    Once upon a time, California\u2019s shorelines, lakes, and valleys were host to expanses of untamed wilderness\n    Once upon a time, she said. It started with a simple question: Why don\u2019t we know any stories?\n    Once upon a time, when I was a young woman, there was a movie named Wuthering Heights. Stand by alleges\n    Once upon a time, a very long time I mean, in the year 1713, died a beautiful Duchess called the young"}


created_at: 2024-07-28 17:45:51
topic: persistent://default/default/308e7593-7e45-4080-bc68-ad4f88ce5b4b
id: 691816ba-f07c-4a74-a6b2-85b9284729e7
encoding: FLOAT32
metadata:
{"url": "https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb", "title": "How to handle rate limits", "tokens": 6170, "retrieve_doc": true, "source": "OpenAI_Cookbooks", "_node_content": "{\"id_\": \"691816ba-f07c-4a74-a6b2-85b9284729e7\", \"embedding\": null, \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"036d236287213f8976a11b4008fcdf7dddb41333fc2b788b4a87761c459d8924\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"327af581-2f7d-417c-9370-bf738fb73111\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://github.com/openai/openai-cookbook/blob/main/examples/How_to_handle_rate_limits.ipynb\", \"title\": \"How to handle rate limits\", \"tokens\": 6170, \"retrieve_doc\": true, \"source\": \"OpenAI_Cookbooks\"}, \"hash\": \"d9006bfa5a59bb2e3622f2b5dea03a9a762eaf2bdf0b4b4b016784158b30150d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25941, \"end_char_idx\": 28207, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "ref_doc_id": "4d4d7d64-2cb7-4b25-9a6b-4e1da7aa137e", "chroma:document": "#### Example with batching\n\n\n```python\nnum_stories = 10\nprompts = [\"Once upon a time,\"] * num_stories\n\n# batched example, with 10 stories completions per request\nresponse = client.chat.completions.create(\n    model=\"curie\",\n    prompt=prompts,\n    max_tokens=20,\n)\n\n# match completions to prompts by index\nstories = [\"\"] * len(prompts)\nfor choice in response.choices:\n    stories[choice.index] = prompts[choice.index] + choice.text\n\n# print stories\nfor story in stories:\n    print(story)\n\n```\n\n    Once upon a time, I lived in hope. I convinced myself I knew best, because, naive as it might sound,\n    Once upon a time, Thierry Henry was invited to have a type of frosty exchange with English fans, in which\n    Once upon a time, and a long time ago as well, PV was passively cooled because coils cooled by use of metal driving\n    Once upon a time, there was a land called Texas. It was about the size of Wisconsin. It contained, however,\n    Once upon a time, there was an old carpenter who had three sons. The locksmith never learned to read or write\n    Once upon a time, there was a small farming town called Moonridge Village, far West across the great vast plains that lay\n    Once upon a time, California\u2019s shorelines, lakes, and valleys were host to expanses of untamed wilderness\n    Once upon a time, she said. It started with a simple question: Why don\u2019t we know any stories?\n    Once upon a time, when I was a young woman, there was a movie named Wuthering Heights. Stand by alleges\n    Once upon a time, a very long time I mean, in the year 1713, died a beautiful Duchess called the young\n\n\n## Example parallel processing script\n\nWe've written an example script for parallel processing large quantities of API requests: [api_request_parallel_processor.py](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py).\n\nThe script combines some handy features:\n- Streams requests from file, to avoid running out of memory for giant jobs\n- Makes requests concurrently, to maximize throughput\n- Throttles both request and token usage, to stay under rate limits\n- Retries failed requests, to avoid missing data\n- Logs errors, to diagnose problems with requests\n\nFeel free to use it as is or modify it to suit your needs."}


