# embeddings_queue

created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 1eed6757-7ea7-4c4f-8e52-2c333b2a6426
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/mask_generation", "title": "Mask Generation", "tokens": 2117, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"1eed6757-7ea7-4c4f-8e52-2c333b2a6426\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2f393569-3c1a-4607-8af3-f3564d76edea\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"385f9b234ba753f61a1095d2d3cde2181f866dca68ccaf42a2dcf54ca0fb65b2\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"154b05a7-dac6-4a4f-8fb8-f05efc925f02\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6ec150254f33a00b4c0197617151c016cd3de138c6dfe16f684383b42f7ecf07\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3002, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "ref_doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "chroma:document": "# Mask Generation\n\nMask generation is the task of generating semantically meaningful masks for an image. \nThis task is very similar to [image segmentation](semantic_segmentation), but many differences exist. Image segmentation models are trained on labeled datasets and are limited to the classes they have seen during training; they return a set of masks and corresponding classes, given an image. \n\nMask generation models are trained on large amounts of data and operate in two modes. \n- Prompting mode: In this mode, the model takes in an image and a prompt, where a prompt can be a 2D point location (XY coordinates) in the image within an object or a bounding box surrounding an object. In prompting mode, the model only returns the mask over the object \nthat the prompt is pointing out. \n- Segment Everything mode: In segment everything, given an image, the model generates every mask in the image. To do so, a grid of points is generated and overlaid on the image for inference. \n\nMask generation task is supported by [Segment Anything Model (SAM)](model_doc/sam). It's a powerful model that consists of a Vision Transformer-based image encoder, a prompt encoder, and a two-way transformer mask decoder. Images and prompts are encoded, and the decoder takes these embeddings and generates valid masks. \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/sam.png\" alt=\"SAM Architecture\"/>\n</div>\n\nSAM serves as a powerful foundation model for segmentation as it has large data coverage. It is trained on \n[SA-1B](https://ai.meta.com/datasets/segment-anything/), a dataset with 1 million images and 1.1 billion masks. \n\nIn this guide, you will learn how to:\n- Infer in segment everything mode with batching,\n- Infer in point prompting mode,\n- Infer in box prompting mode.\n\nFirst, let's install `transformers`:\n\n```bash\npip install -q transformers\n```\n\n## Mask Generation Pipeline\n\nThe easiest way to infer mask generation models is to use the `mask-generation` pipeline.\n\n```python\n>>> from transformers import pipeline\n\n>>> checkpoint = \"facebook/sam-vit-base\"\n>>> mask_generator = pipeline(model=checkpoint, task=\"mask-generation\")\n```\n\nLet's see the image.\n\n```python\nfrom PIL import Image\nimport requests\n\nimg_url = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\"\nimage = Image.open(requests.get(img_url, stream=True).raw).convert(\"RGB\")\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\" alt=\"Example Image\"/>\n</div>\n\nLet's segment everything. `points-per-batch` enables parallel inference of points in segment everything mode. This enables faster inference, but consumes more memory. Moreover, SAM only enables batching over points and not the images. `pred_iou_thresh` is the IoU confidence threshold where only the masks above that certain threshold are returned."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 154b05a7-dac6-4a4f-8fb8-f05efc925f02
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/mask_generation", "title": "Mask Generation", "tokens": 2117, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"154b05a7-dac6-4a4f-8fb8-f05efc925f02\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2f393569-3c1a-4607-8af3-f3564d76edea\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"385f9b234ba753f61a1095d2d3cde2181f866dca68ccaf42a2dcf54ca0fb65b2\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"1eed6757-7ea7-4c4f-8e52-2c333b2a6426\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"1d4c7c8d31efdaf6fbe969e46d7b92aceb94ca80b9b1f8fdd25159f8178c7eaf\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"aad2f637-21fd-4a76-a407-41a2f5f071f6\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c099248979b7f82b2d3fc917922ca409f250c854bb01029223269d4b40c174c5\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1311, \"end_char_idx\": 4262, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "ref_doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "chroma:document": "<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/sam.png\" alt=\"SAM Architecture\"/>\n</div>\n\nSAM serves as a powerful foundation model for segmentation as it has large data coverage. It is trained on \n[SA-1B](https://ai.meta.com/datasets/segment-anything/), a dataset with 1 million images and 1.1 billion masks. \n\nIn this guide, you will learn how to:\n- Infer in segment everything mode with batching,\n- Infer in point prompting mode,\n- Infer in box prompting mode.\n\nFirst, let's install `transformers`:\n\n```bash\npip install -q transformers\n```\n\n## Mask Generation Pipeline\n\nThe easiest way to infer mask generation models is to use the `mask-generation` pipeline.\n\n```python\n>>> from transformers import pipeline\n\n>>> checkpoint = \"facebook/sam-vit-base\"\n>>> mask_generator = pipeline(model=checkpoint, task=\"mask-generation\")\n```\n\nLet's see the image.\n\n```python\nfrom PIL import Image\nimport requests\n\nimg_url = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\"\nimage = Image.open(requests.get(img_url, stream=True).raw).convert(\"RGB\")\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\" alt=\"Example Image\"/>\n</div>\n\nLet's segment everything. `points-per-batch` enables parallel inference of points in segment everything mode. This enables faster inference, but consumes more memory. Moreover, SAM only enables batching over points and not the images. `pred_iou_thresh` is the IoU confidence threshold where only the masks above that certain threshold are returned.\n\n```python\nmasks = mask_generator(image, points_per_batch=128, pred_iou_thresh=0.88)\n```\n\nThe `masks` looks like the following:\n\n```bash\n{'masks': [array([[False, False, False, ...,  True,  True,  True],\n         [False, False, False, ...,  True,  True,  True],\n         [False, False, False, ...,  True,  True,  True],\n         ...,\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False]]),\n  array([[False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         ...,\n'scores': tensor([0.9972, 0.9917,\n        ...,\n}\n```\n\nWe can visualize them like this:\n\n```python\nimport matplotlib.pyplot as plt\n\nplt.imshow(image, cmap='gray')\n\nfor i, mask in enumerate(masks[\"masks\"]):\n    plt.imshow(mask, cmap='viridis', alpha=0.1, vmin=0, vmax=1)\n\nplt.axis('off')\nplt.show()\n```\n\nBelow is the original image in grayscale with colorful maps overlaid. Very impressive.\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee_segmented.png\" alt=\"Visualized\"/>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: aad2f637-21fd-4a76-a407-41a2f5f071f6
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/mask_generation", "title": "Mask Generation", "tokens": 2117, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"aad2f637-21fd-4a76-a407-41a2f5f071f6\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2f393569-3c1a-4607-8af3-f3564d76edea\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"385f9b234ba753f61a1095d2d3cde2181f866dca68ccaf42a2dcf54ca0fb65b2\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"154b05a7-dac6-4a4f-8fb8-f05efc925f02\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"41fcdf816c00785d00457985ae6f1e79a4aeb18f36cbce7ee4df1c31342bdc17\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"2b97d6d6-8993-46ab-9d8b-6187537c1a13\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f0ea727a0aa4d79b68a673172a4fea6e31fea884671222f24da93b79c0fc4de1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2764, \"end_char_idx\": 6701, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "ref_doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "chroma:document": "This enables faster inference, but consumes more memory. Moreover, SAM only enables batching over points and not the images. `pred_iou_thresh` is the IoU confidence threshold where only the masks above that certain threshold are returned.\n\n```python\nmasks = mask_generator(image, points_per_batch=128, pred_iou_thresh=0.88)\n```\n\nThe `masks` looks like the following:\n\n```bash\n{'masks': [array([[False, False, False, ...,  True,  True,  True],\n         [False, False, False, ...,  True,  True,  True],\n         [False, False, False, ...,  True,  True,  True],\n         ...,\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False]]),\n  array([[False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         [False, False, False, ..., False, False, False],\n         ...,\n'scores': tensor([0.9972, 0.9917,\n        ...,\n}\n```\n\nWe can visualize them like this:\n\n```python\nimport matplotlib.pyplot as plt\n\nplt.imshow(image, cmap='gray')\n\nfor i, mask in enumerate(masks[\"masks\"]):\n    plt.imshow(mask, cmap='viridis', alpha=0.1, vmin=0, vmax=1)\n\nplt.axis('off')\nplt.show()\n```\n\nBelow is the original image in grayscale with colorful maps overlaid. Very impressive.\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee_segmented.png\" alt=\"Visualized\"/>\n</div>\n\n\n## Model Inference\n\n### Point Prompting\n\nYou can also use the model without the pipeline. To do so, initialize the model and\nthe processor.\n\n```python\nfrom transformers import SamModel, SamProcessor\nimport torch\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\nmodel = SamModel.from_pretrained(\"facebook/sam-vit-base\").to(device)\nprocessor = SamProcessor.from_pretrained(\"facebook/sam-vit-base\")\n```\n\nTo do point prompting, pass the input point to the processor, then take the processor output\nand pass it to the model for inference. To post-process the model output, pass the outputs and\n`original_sizes` and `reshaped_input_sizes` we take from the processor's initial output. We need to pass these \nsince the processor resizes the image, and the output needs to be extrapolated.\n\n```python\ninput_points = [[[2592, 1728]]] # point location of the bee\n\ninputs = processor(image, input_points=input_points, return_tensors=\"pt\").to(device)\nwith torch.no_grad():\n    outputs = model(**inputs)\nmasks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs[\"original_sizes\"].cpu(), inputs[\"reshaped_input_sizes\"].cpu())\n```\nWe can visualize the three masks in the `masks`\u00a0output.\n\n```python\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfig, axes = plt.subplots(1, 4, figsize=(15, 5))\n\naxes[0].imshow(image)\naxes[0].set_title('Original Image')\nmask_list = [masks[0][0][0].numpy(), masks[0][0][1].numpy(), masks[0][0][2].numpy()]\n\nfor i, mask in enumerate(mask_list, start=1):\n    overlayed_image = np.array(image).copy()\n\n    overlayed_image[:,:,0] = np.where(mask == 1, 255, overlayed_image[:,:,0])\n    overlayed_image[:,:,1] = np.where(mask == 1, 0, overlayed_image[:,:,1])\n    overlayed_image[:,:,2] = np.where(mask == 1, 0, overlayed_image[:,:,2])\n    \n    axes[i].imshow(overlayed_image)\n    axes[i].set_title(f'Mask {i}')\nfor ax in axes:\n    ax.axis('off')\n\nplt.show()\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/masks.png\" alt=\"Visualized\"/>\n</div>\n\n### Box Prompting\n\nYou can also do box prompting in a similar fashion to point prompting. You can simply pass the input box in the format of a list\n`[x_min, y_min, x_max, y_max]` format along with the image to the `processor`. Take the processor output and directly pass it \nto the model, then post-process the output again."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 2b97d6d6-8993-46ab-9d8b-6187537c1a13
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/mask_generation", "title": "Mask Generation", "tokens": 2117, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"2b97d6d6-8993-46ab-9d8b-6187537c1a13\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2f393569-3c1a-4607-8af3-f3564d76edea\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"385f9b234ba753f61a1095d2d3cde2181f866dca68ccaf42a2dcf54ca0fb65b2\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"aad2f637-21fd-4a76-a407-41a2f5f071f6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/mask_generation\", \"title\": \"Mask Generation\", \"tokens\": 2117, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"1496c6cfa0a677066af684ad48ac8af018ea5f1bfc479890e6b473a0edab61e1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6704, \"end_char_idx\": 7995, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "ref_doc_id": "2f393569-3c1a-4607-8af3-f3564d76edea", "chroma:document": "```python\n# bounding box around the bee\nbox = [2350, 1600, 2850, 2100]\n\ninputs = processor(\n        image,\n        input_boxes=[[[box]]],\n        return_tensors=\"pt\"\n    ).to(\"cuda\")\n\nwith torch.no_grad():\n    outputs = model(**inputs)\n\nmask = processor.image_processor.post_process_masks(\n    outputs.pred_masks.cpu(),\n    inputs[\"original_sizes\"].cpu(),\n    inputs[\"reshaped_input_sizes\"].cpu()\n)[0][0][0].numpy()\n```\n\nYou can visualize the bounding box around the bee as shown below.\n\n```python\nimport matplotlib.patches as patches\n\nfig, ax = plt.subplots()\nax.imshow(image)\n\nrectangle = patches.Rectangle((2350, 1600), 500, 500, linewidth=2, edgecolor='r', facecolor='none')\nax.add_patch(rectangle)\nax.axis(\"off\")\nplt.show()\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/bbox.png\" alt=\"Visualized Bbox\"/>\n</div>\n\nYou can see the inference output below. \n\n```python\nfig, ax = plt.subplots()\nax.imshow(image)\nax.imshow(mask, cmap='viridis', alpha=0.4)\n\nax.axis(\"off\")\nplt.show()\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/box_inference.png\" alt=\"Visualized Inference\"/>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 665ffc4a-3a83-48ce-9e9e-bb4972e23b20
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"665ffc4a-3a83-48ce-9e9e-bb4972e23b20\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"0218814e-3820-48b0-ae4e-20ec36453362\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"57f0f676a9f4b24e604e3b97d0e1f70e4861a8cb761b5e925fbca7da301c5826\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2898, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "# Zero-shot object detection\n\n[[open-in-colab]]\n\nTraditionally, models used for [object detection](object_detection) require labeled image datasets for training,\nand are limited to detecting the set of classes from the training data.\n\nZero-shot object detection is supported by the [OWL-ViT](../model_doc/owlvit) model which uses a different approach. OWL-ViT\nis an open-vocabulary object detector. It means that it can detect objects in images based on free-text queries without\nthe need to fine-tune the model on labeled datasets.\n\nOWL-ViT leverages multi-modal representations to perform open-vocabulary detection. It combines [CLIP](../model_doc/clip) with\nlightweight object classification and localization heads. Open-vocabulary detection is achieved by embedding free-text queries with the text encoder of CLIP and using them as input to the object classification and localization heads.\nassociate images and their corresponding textual descriptions, and ViT processes image patches as inputs. The authors\nof OWL-ViT first trained CLIP from scratch and then fine-tuned OWL-ViT end to end on standard object detection datasets using\na bipartite matching loss.\n\nWith this approach, the model can detect objects based on textual descriptions without prior training on labeled datasets.\n\nIn this guide, you will learn how to use OWL-ViT:\n- to detect objects based on text prompts\n- for batch object detection\n- for image-guided object detection\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install -q transformers\n```\n\n## Zero-shot object detection pipeline\n\nThe simplest way to try out inference with OWL-ViT is to use it in a [`pipeline`]. Instantiate a pipeline\nfor zero-shot object detection from a [checkpoint on the Hugging Face Hub](https://huggingface.co/models?other=owlvit):\n\n```python\n>>> from transformers import pipeline\n\n>>> checkpoint = \"google/owlv2-base-patch16-ensemble\"\n>>> detector = pipeline(model=checkpoint, task=\"zero-shot-object-detection\")\n```\n\nNext, choose an image you'd like to detect objects in. Here we'll use the image of astronaut Eileen Collins that is\na part of the [NASA](https://www.nasa.gov/multimedia/imagegallery/index.html) Great Images dataset.\n\n```py\n>>> import skimage\n>>> import numpy as np\n>>> from PIL import Image\n\n>>> image = skimage.data.astronaut()\n>>> image = Image.fromarray(np.uint8(image)).convert(\"RGB\")\n\n>>> image\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_1.png\" alt=\"Astronaut Eileen Collins\"/>\n</div>\n\nPass the image and the candidate object labels to look for to the pipeline.\nHere we pass the image directly; other suitable options include a local path to an image or an image url. We also pass text descriptions for all items we want to query the image for."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 0218814e-3820-48b0-ae4e-20ec36453362
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"0218814e-3820-48b0-ae4e-20ec36453362\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"665ffc4a-3a83-48ce-9e9e-bb4972e23b20\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"a96066c26dc7602891a516c6e79e22e57ee235d6cbf4bd552a5698c765c0bded\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"61e08514-ded9-4d43-8cdf-48850e141be6\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0bedb9152c57476bafe862cc8c56f580398b1ab2552580308d8d30a181821169\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1167, \"end_char_idx\": 4636, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "With this approach, the model can detect objects based on textual descriptions without prior training on labeled datasets.\n\nIn this guide, you will learn how to use OWL-ViT:\n- to detect objects based on text prompts\n- for batch object detection\n- for image-guided object detection\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install -q transformers\n```\n\n## Zero-shot object detection pipeline\n\nThe simplest way to try out inference with OWL-ViT is to use it in a [`pipeline`]. Instantiate a pipeline\nfor zero-shot object detection from a [checkpoint on the Hugging Face Hub](https://huggingface.co/models?other=owlvit):\n\n```python\n>>> from transformers import pipeline\n\n>>> checkpoint = \"google/owlv2-base-patch16-ensemble\"\n>>> detector = pipeline(model=checkpoint, task=\"zero-shot-object-detection\")\n```\n\nNext, choose an image you'd like to detect objects in. Here we'll use the image of astronaut Eileen Collins that is\na part of the [NASA](https://www.nasa.gov/multimedia/imagegallery/index.html) Great Images dataset.\n\n```py\n>>> import skimage\n>>> import numpy as np\n>>> from PIL import Image\n\n>>> image = skimage.data.astronaut()\n>>> image = Image.fromarray(np.uint8(image)).convert(\"RGB\")\n\n>>> image\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_1.png\" alt=\"Astronaut Eileen Collins\"/>\n</div>\n\nPass the image and the candidate object labels to look for to the pipeline.\nHere we pass the image directly; other suitable options include a local path to an image or an image url. We also pass text descriptions for all items we want to query the image for. \n\n```py\n>>> predictions = detector(\n...     image,\n...     candidate_labels=[\"human face\", \"rocket\", \"nasa badge\", \"star-spangled banner\"],\n... )\n>>> predictions\n[{'score': 0.3571370542049408,\n  'label': 'human face',\n  'box': {'xmin': 180, 'ymin': 71, 'xmax': 271, 'ymax': 178}},\n {'score': 0.28099656105041504,\n  'label': 'nasa badge',\n  'box': {'xmin': 129, 'ymin': 348, 'xmax': 206, 'ymax': 427}},\n {'score': 0.2110239565372467,\n  'label': 'rocket',\n  'box': {'xmin': 350, 'ymin': -1, 'xmax': 468, 'ymax': 288}},\n {'score': 0.13790413737297058,\n  'label': 'star-spangled banner',\n  'box': {'xmin': 1, 'ymin': 1, 'xmax': 105, 'ymax': 509}},\n {'score': 0.11950037628412247,\n  'label': 'nasa badge',\n  'box': {'xmin': 277, 'ymin': 338, 'xmax': 327, 'ymax': 380}},\n {'score': 0.10649408400058746,\n  'label': 'rocket',\n  'box': {'xmin': 358, 'ymin': 64, 'xmax': 424, 'ymax': 280}}]\n```\n\nLet's visualize the predictions:\n\n```py\n>>> from PIL import ImageDraw\n\n>>> draw = ImageDraw.Draw(image)\n\n>>> for prediction in predictions:\n...     box = prediction[\"box\"]\n...     label = prediction[\"label\"]\n...     score = prediction[\"score\"]\n\n...     xmin, ymin, xmax, ymax = box.values()\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"red\", width=1)\n...     draw.text((xmin, ymin), f\"{label}: {round(score,2)}\", fill=\"white\")\n\n>>> image\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_2.png\" alt=\"Visualized predictions on NASA image\"/>\n</div>\n\n## Text-prompted zero-shot object detection by hand\n\nNow that you've seen how to use the zero-shot object detection pipeline, let's replicate the same\nresult manually."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 61e08514-ded9-4d43-8cdf-48850e141be6
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"61e08514-ded9-4d43-8cdf-48850e141be6\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"0218814e-3820-48b0-ae4e-20ec36453362\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"59ba5ca7ff9b5f78b61c152e547936316f203ccbc49a52eea786b8c3333af528\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"fdd191be-2a56-4ed1-865c-f3fc041d401f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f2679b3e1283b8389acede742e3aa00aca7adae05978ae60bf292065ec2d21f9\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4638, \"end_char_idx\": 7506, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "Start by loading the model and associated processor from a [checkpoint on the Hugging Face Hub](https://huggingface.co/models?other=owlvit).\nHere we'll use the same checkpoint as before:\n\n```py\n>>> from transformers import AutoProcessor, AutoModelForZeroShotObjectDetection\n\n>>> model = AutoModelForZeroShotObjectDetection.from_pretrained(checkpoint)\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n```\n\nLet's take a different image to switch things up.\n\n```py\n>>> import requests\n\n>>> url = \"https://unsplash.com/photos/oj0zeY2Ltk4/download?ixid=MnwxMjA3fDB8MXxzZWFyY2h8MTR8fHBpY25pY3xlbnwwfHx8fDE2Nzc0OTE1NDk&force=true&w=640\"\n>>> im = Image.open(requests.get(url, stream=True).raw)\n>>> im\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_3.png\" alt=\"Beach photo\"/>\n</div>\n\nUse the processor to prepare the inputs for the model. The processor combines an image processor that prepares the\nimage for the model by resizing and normalizing it, and a [`CLIPTokenizer`] that takes care of the text inputs.\n\n```py\n>>> text_queries = [\"hat\", \"book\", \"sunglasses\", \"camera\"]\n>>> inputs = processor(text=text_queries, images=im, return_tensors=\"pt\")\n```\n\nPass the inputs through the model, post-process, and visualize the results. Since the image processor resized images before\nfeeding them to the model, you need to use the [`~OwlViTImageProcessor.post_process_object_detection`] method to make sure the predicted bounding\nboxes have the correct coordinates relative to the original image:\n\n```py\n>>> import torch\n\n>>> with torch.no_grad():\n...     outputs = model(**inputs)\n...     target_sizes = torch.tensor([im.size[::-1]])\n...     results = processor.post_process_object_detection(outputs, threshold=0.1, target_sizes=target_sizes)[0]\n\n>>> draw = ImageDraw.Draw(im)\n\n>>> scores = results[\"scores\"].tolist()\n>>> labels = results[\"labels\"].tolist()\n>>> boxes = results[\"boxes\"].tolist()\n\n>>> for box, score, label in zip(boxes, scores, labels):\n...     xmin, ymin, xmax, ymax = box\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"red\", width=1)\n...     draw.text((xmin, ymin), f\"{text_queries[label]}: {round(score,2)}\", fill=\"white\")\n\n>>> im\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_4.png\" alt=\"Beach photo with detected objects\"/>\n</div>\n\n## Batch processing\n\nYou can pass multiple sets of images and text queries to search for different (or same) objects in several images.\nLet's use both an astronaut image and the beach image together.\nFor batch processing, you should pass text queries as a nested list to the processor and images as lists of PIL images,\nPyTorch tensors, or NumPy arrays."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: fdd191be-2a56-4ed1-865c-f3fc041d401f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"fdd191be-2a56-4ed1-865c-f3fc041d401f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"61e08514-ded9-4d43-8cdf-48850e141be6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"74ea051195fba5a8758d9926433f244481b93edc6ab1cf3aa5d61e59e9cea028\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b14089cf-e861-46ff-b319-961d1deeafbe\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7ec40982ef3dc2111c501cdfbc6684f788849a2fc270c6be455be431fb0c863c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5996, \"end_char_idx\": 9215, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "Since the image processor resized images before\nfeeding them to the model, you need to use the [`~OwlViTImageProcessor.post_process_object_detection`] method to make sure the predicted bounding\nboxes have the correct coordinates relative to the original image:\n\n```py\n>>> import torch\n\n>>> with torch.no_grad():\n...     outputs = model(**inputs)\n...     target_sizes = torch.tensor([im.size[::-1]])\n...     results = processor.post_process_object_detection(outputs, threshold=0.1, target_sizes=target_sizes)[0]\n\n>>> draw = ImageDraw.Draw(im)\n\n>>> scores = results[\"scores\"].tolist()\n>>> labels = results[\"labels\"].tolist()\n>>> boxes = results[\"boxes\"].tolist()\n\n>>> for box, score, label in zip(boxes, scores, labels):\n...     xmin, ymin, xmax, ymax = box\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"red\", width=1)\n...     draw.text((xmin, ymin), f\"{text_queries[label]}: {round(score,2)}\", fill=\"white\")\n\n>>> im\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_4.png\" alt=\"Beach photo with detected objects\"/>\n</div>\n\n## Batch processing\n\nYou can pass multiple sets of images and text queries to search for different (or same) objects in several images.\nLet's use both an astronaut image and the beach image together.\nFor batch processing, you should pass text queries as a nested list to the processor and images as lists of PIL images,\nPyTorch tensors, or NumPy arrays.\n\n```py\n>>> images = [image, im]\n>>> text_queries = [\n...     [\"human face\", \"rocket\", \"nasa badge\", \"star-spangled banner\"],\n...     [\"hat\", \"book\", \"sunglasses\", \"camera\"],\n... ]\n>>> inputs = processor(text=text_queries, images=images, return_tensors=\"pt\")\n```\n\nPreviously for post-processing you passed the single image's size as a tensor, but you can also pass a tuple, or, in case\nof several images, a list of tuples. Let's create predictions for the two examples, and visualize the second one (`image_idx = 1`).\n\n```py\n>>> with torch.no_grad():\n...     outputs = model(**inputs)\n...     target_sizes = [x.size[::-1] for x in images]\n...     results = processor.post_process_object_detection(outputs, threshold=0.1, target_sizes=target_sizes)\n\n>>> image_idx = 1\n>>> draw = ImageDraw.Draw(images[image_idx])\n\n>>> scores = results[image_idx][\"scores\"].tolist()\n>>> labels = results[image_idx][\"labels\"].tolist()\n>>> boxes = results[image_idx][\"boxes\"].tolist()\n\n>>> for box, score, label in zip(boxes, scores, labels):\n...     xmin, ymin, xmax, ymax = box\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"red\", width=1)\n...     draw.text((xmin, ymin), f\"{text_queries[image_idx][label]}: {round(score,2)}\", fill=\"white\")\n\n>>> images[image_idx]\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_4.png\" alt=\"Beach photo with detected objects\"/>\n</div>\n\n## Image-guided object detection\n\nIn addition to zero-shot object detection with text queries, OWL-ViT offers image-guided object detection. This means\nyou can use an image query to find similar objects in the target image."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b14089cf-e861-46ff-b319-961d1deeafbe
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"b14089cf-e861-46ff-b319-961d1deeafbe\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"fdd191be-2a56-4ed1-865c-f3fc041d401f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"270f0c224ab48347ff571c1b6ad5e00539daa4382277a026662cc95b58b4d108\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"75c88cf8-3437-4a39-a1e3-1e776b6a5a92\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"579fbd9f59083f4ea1aafad225d23c489c474f135b633e9077cf401783306302\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7929, \"end_char_idx\": 10432, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "Let's create predictions for the two examples, and visualize the second one (`image_idx = 1`).\n\n```py\n>>> with torch.no_grad():\n...     outputs = model(**inputs)\n...     target_sizes = [x.size[::-1] for x in images]\n...     results = processor.post_process_object_detection(outputs, threshold=0.1, target_sizes=target_sizes)\n\n>>> image_idx = 1\n>>> draw = ImageDraw.Draw(images[image_idx])\n\n>>> scores = results[image_idx][\"scores\"].tolist()\n>>> labels = results[image_idx][\"labels\"].tolist()\n>>> boxes = results[image_idx][\"boxes\"].tolist()\n\n>>> for box, score, label in zip(boxes, scores, labels):\n...     xmin, ymin, xmax, ymax = box\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"red\", width=1)\n...     draw.text((xmin, ymin), f\"{text_queries[image_idx][label]}: {round(score,2)}\", fill=\"white\")\n\n>>> images[image_idx]\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_4.png\" alt=\"Beach photo with detected objects\"/>\n</div>\n\n## Image-guided object detection\n\nIn addition to zero-shot object detection with text queries, OWL-ViT offers image-guided object detection. This means\nyou can use an image query to find similar objects in the target image.\nUnlike text queries, only a single example image is allowed.\n\nLet's take an image with two cats on a couch as a target image, and an image of a single cat\nas a query:\n\n```py\n>>> url = \"http://images.cocodataset.org/val2017/000000039769.jpg\"\n>>> image_target = Image.open(requests.get(url, stream=True).raw)\n\n>>> query_url = \"http://images.cocodataset.org/val2017/000000524280.jpg\"\n>>> query_image = Image.open(requests.get(query_url, stream=True).raw)\n```\n\nLet's take a quick look at the images:\n\n```py\n>>> import matplotlib.pyplot as plt\n\n>>> fig, ax = plt.subplots(1, 2)\n>>> ax[0].imshow(image_target)\n>>> ax[1].imshow(query_image)\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_5.png\" alt=\"Cats\"/>\n</div>\n\nIn the preprocessing step, instead of text queries, you now need to use `query_images`:\n\n```py\n>>> inputs = processor(images=image_target, query_images=query_image, return_tensors=\"pt\")\n```\n\nFor predictions, instead of passing the inputs to the model, pass them to [`~OwlViTForObjectDetection.image_guided_detection`]. Draw the predictions\nas before except now there are no labels."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 75c88cf8-3437-4a39-a1e3-1e776b6a5a92
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection", "title": "Zero-shot object detection", "tokens": 2993, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"75c88cf8-3437-4a39-a1e3-1e776b6a5a92\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"af3b8454-6c3e-40da-96bc-81ddf1e7406f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7e8db6ca78f64dab105a404ac6bb403a05c857981260dd057dd944018eb8cee3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b14089cf-e861-46ff-b319-961d1deeafbe\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection\", \"title\": \"Zero-shot object detection\", \"tokens\": 2993, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2041335e14cf1710e017cb57d3a6b32fc465d0fb5a16145e7fc042e1f682fbdd\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9133, \"end_char_idx\": 11224, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "ref_doc_id": "af3b8454-6c3e-40da-96bc-81ddf1e7406f", "chroma:document": "This means\nyou can use an image query to find similar objects in the target image.\nUnlike text queries, only a single example image is allowed.\n\nLet's take an image with two cats on a couch as a target image, and an image of a single cat\nas a query:\n\n```py\n>>> url = \"http://images.cocodataset.org/val2017/000000039769.jpg\"\n>>> image_target = Image.open(requests.get(url, stream=True).raw)\n\n>>> query_url = \"http://images.cocodataset.org/val2017/000000524280.jpg\"\n>>> query_image = Image.open(requests.get(query_url, stream=True).raw)\n```\n\nLet's take a quick look at the images:\n\n```py\n>>> import matplotlib.pyplot as plt\n\n>>> fig, ax = plt.subplots(1, 2)\n>>> ax[0].imshow(image_target)\n>>> ax[1].imshow(query_image)\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_5.png\" alt=\"Cats\"/>\n</div>\n\nIn the preprocessing step, instead of text queries, you now need to use `query_images`:\n\n```py\n>>> inputs = processor(images=image_target, query_images=query_image, return_tensors=\"pt\")\n```\n\nFor predictions, instead of passing the inputs to the model, pass them to [`~OwlViTForObjectDetection.image_guided_detection`]. Draw the predictions\nas before except now there are no labels.\n\n```py\n>>> with torch.no_grad():\n...     outputs = model.image_guided_detection(**inputs)\n...     target_sizes = torch.tensor([image_target.size[::-1]])\n...     results = processor.post_process_image_guided_detection(outputs=outputs, target_sizes=target_sizes)[0]\n\n>>> draw = ImageDraw.Draw(image_target)\n\n>>> scores = results[\"scores\"].tolist()\n>>> boxes = results[\"boxes\"].tolist()\n\n>>> for box, score, label in zip(boxes, scores, labels):\n...     xmin, ymin, xmax, ymax = box\n...     draw.rectangle((xmin, ymin, xmax, ymax), outline=\"white\", width=4)\n\n>>> image_target\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/zero-sh-obj-detection_6.png\" alt=\"Cats with bounding boxes\"/>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 8eb8007b-f189-4791-a5bf-a76dd40616f2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"8eb8007b-f189-4791-a5bf-a76dd40616f2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"20b271f3-cf33-4948-a786-e45ba2bff738\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6124ef1c5d9dbf649af3c2f030fd53cd1b84bcb0c53bccb420bccefdfb79a230\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2327, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "# Image Segmentation\n\n[[open-in-colab]]\n\n<Youtube id=\"dKE8SIt9C-w\"/>\n\nImage segmentation models separate areas corresponding to different areas of interest in an image. These models work by assigning a label to each pixel. There are several types of segmentation: semantic segmentation, instance segmentation, and panoptic segmentation.\n\nIn this guide, we will:\n1. [Take a look at different types of segmentation](#types-of-segmentation).\n2. [Have an end-to-end fine-tuning example for semantic segmentation](#fine-tuning-a-model-for-segmentation).\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```py\n# uncomment to install the necessary libraries\n!pip install -q datasets transformers evaluate accelerate\n```\n\nWe encourage you to log in to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\n## Types of Segmentation\n\nSemantic segmentation assigns a label or class to every single pixel in an image. Let's take a look at a semantic segmentation model output. It will assign the same class to every instance of an object it comes across in an image, for example, all cats will be labeled as \"cat\" instead of \"cat-1\", \"cat-2\".\nWe can use transformers' image segmentation pipeline to quickly infer a semantic segmentation model. Let's take a look at the example image.\n\n```python\nfrom transformers import pipeline\nfrom PIL import Image\nimport requests\n\nurl = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation_input.jpg\"\nimage = Image.open(requests.get(url, stream=True).raw)\nimage\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation_input.jpg\" alt=\"Segmentation Input\"/>\n</div>\n\nWe will use [nvidia/segformer-b1-finetuned-cityscapes-1024-1024](https://huggingface.co/nvidia/segformer-b1-finetuned-cityscapes-1024-1024).\n\n```python\nsemantic_segmentation = pipeline(\"image-segmentation\", \"nvidia/segformer-b1-finetuned-cityscapes-1024-1024\")\nresults = semantic_segmentation(image)\nresults\n```\n\nThe segmentation pipeline output includes a mask for every predicted class."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 20b271f3-cf33-4948-a786-e45ba2bff738
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"20b271f3-cf33-4948-a786-e45ba2bff738\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8eb8007b-f189-4791-a5bf-a76dd40616f2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"5ba99c88e389ec3cde8471d8e4d55cc3cbe451f531d7ef37b11b4175f1e1b630\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a6a3c46e-fc80-412d-9e3a-84d46d36e5f4\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"332e480efc41cae58485573f76a0402fbc065de4b38e5b72408088e6d4f58cff\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 858, \"end_char_idx\": 3383, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "When prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\n## Types of Segmentation\n\nSemantic segmentation assigns a label or class to every single pixel in an image. Let's take a look at a semantic segmentation model output. It will assign the same class to every instance of an object it comes across in an image, for example, all cats will be labeled as \"cat\" instead of \"cat-1\", \"cat-2\".\nWe can use transformers' image segmentation pipeline to quickly infer a semantic segmentation model. Let's take a look at the example image.\n\n```python\nfrom transformers import pipeline\nfrom PIL import Image\nimport requests\n\nurl = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation_input.jpg\"\nimage = Image.open(requests.get(url, stream=True).raw)\nimage\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation_input.jpg\" alt=\"Segmentation Input\"/>\n</div>\n\nWe will use [nvidia/segformer-b1-finetuned-cityscapes-1024-1024](https://huggingface.co/nvidia/segformer-b1-finetuned-cityscapes-1024-1024).\n\n```python\nsemantic_segmentation = pipeline(\"image-segmentation\", \"nvidia/segformer-b1-finetuned-cityscapes-1024-1024\")\nresults = semantic_segmentation(image)\nresults\n```\n\nThe segmentation pipeline output includes a mask for every predicted class.\n```bash\n[{'score': None,\n  'label': 'road',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'sidewalk',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'building',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'wall',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'pole',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'traffic sign',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'vegetation',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'terrain',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'sky',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\n\nTaking a look at the mask for the car class, we can see every car is classified with the same mask."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a6a3c46e-fc80-412d-9e3a-84d46d36e5f4
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"a6a3c46e-fc80-412d-9e3a-84d46d36e5f4\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"20b271f3-cf33-4948-a786-e45ba2bff738\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"e8f03516ae1599e5d062fef92a3b681da7cf8e67f1c8ecda8a046f45860c268c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d5514e7c-de77-4203-ba5e-ba69e9f09828\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d19353615ce3434092b174583dda05256f54f985c7c13b5ac4fbb23fc4319713\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2328, \"end_char_idx\": 4836, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "```bash\n[{'score': None,\n  'label': 'road',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'sidewalk',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'building',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'wall',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'pole',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'traffic sign',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'vegetation',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'terrain',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'sky',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': None,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\n\nTaking a look at the mask for the car class, we can see every car is classified with the same mask.\n\n```python\nresults[-1][\"mask\"]\n```\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/semantic_segmentation_output.png\" alt=\"Semantic Segmentation Output\"/>\n</div>\n\nIn instance segmentation, the goal is not to classify every pixel, but to predict a mask for **every instance of an object** in a given image. It works very similar to object detection, where there is a bounding box for every instance, there's a segmentation mask instead. We will use [facebook/mask2former-swin-large-cityscapes-instance](https://huggingface.co/facebook/mask2former-swin-large-cityscapes-instance) for this.\n\n```python\ninstance_segmentation = pipeline(\"image-segmentation\", \"facebook/mask2former-swin-large-cityscapes-instance\")\nresults = instance_segmentation(image)\nresults\n```\n\nAs you can see below, there are multiple cars classified, and there's no classification for pixels other than pixels that belong to car and person instances.\n\n```bash\n[{'score': 0.999944,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999945,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999652,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.903529,\n  'label': 'person',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\nChecking out one of the car masks below."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d5514e7c-de77-4203-ba5e-ba69e9f09828
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d5514e7c-de77-4203-ba5e-ba69e9f09828\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a6a3c46e-fc80-412d-9e3a-84d46d36e5f4\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"1f825b0efe8e261f6f3bde900d9d62ebdec615d5fa4be2cb43f08d9b1750307a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"09c7cf61-8a8b-465f-9766-d8a560b4ec8a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e48832e7c3c106d2f31feb8510cce7b1d819460b8dfeda004749a8a783d24f53\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3385, \"end_char_idx\": 5765, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "```python\nresults[-1][\"mask\"]\n```\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/semantic_segmentation_output.png\" alt=\"Semantic Segmentation Output\"/>\n</div>\n\nIn instance segmentation, the goal is not to classify every pixel, but to predict a mask for **every instance of an object** in a given image. It works very similar to object detection, where there is a bounding box for every instance, there's a segmentation mask instead. We will use [facebook/mask2former-swin-large-cityscapes-instance](https://huggingface.co/facebook/mask2former-swin-large-cityscapes-instance) for this.\n\n```python\ninstance_segmentation = pipeline(\"image-segmentation\", \"facebook/mask2former-swin-large-cityscapes-instance\")\nresults = instance_segmentation(image)\nresults\n```\n\nAs you can see below, there are multiple cars classified, and there's no classification for pixels other than pixels that belong to car and person instances.\n\n```bash\n[{'score': 0.999944,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999945,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999652,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.903529,\n  'label': 'person',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\nChecking out one of the car masks below.\n\n```python\nresults[2][\"mask\"]\n```\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/instance_segmentation_output.png\" alt=\"Semantic Segmentation Output\"/>\n</div>\n\nPanoptic segmentation combines semantic segmentation and instance segmentation, where every pixel is classified into a class and an instance of that class, and there are multiple masks for each instance of a class. We can use [facebook/mask2former-swin-large-cityscapes-panoptic](https://huggingface.co/facebook/mask2former-swin-large-cityscapes-panoptic) for this.\n\n```python\npanoptic_segmentation = pipeline(\"image-segmentation\", \"facebook/mask2former-swin-large-cityscapes-panoptic\")\nresults = panoptic_segmentation(image)\nresults\n```\nAs you can see below, we have more classes. We will later illustrate to see that every pixel is classified into one of the classes."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 09c7cf61-8a8b-465f-9766-d8a560b4ec8a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"09c7cf61-8a8b-465f-9766-d8a560b4ec8a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d5514e7c-de77-4203-ba5e-ba69e9f09828\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"9781af54940e88f586b3d844232b5c8dfda380abc9d858caf034ed50da57bb99\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"46d69b62-6b0f-4708-a807-22fd760cc552\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"27b255274b441150644516f516d93eea7eb1c96885cdeb37882b514bd088ccc1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4401, \"end_char_idx\": 7045, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "```bash\n[{'score': 0.999944,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999945,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999652,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.903529,\n  'label': 'person',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\nChecking out one of the car masks below.\n\n```python\nresults[2][\"mask\"]\n```\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/instance_segmentation_output.png\" alt=\"Semantic Segmentation Output\"/>\n</div>\n\nPanoptic segmentation combines semantic segmentation and instance segmentation, where every pixel is classified into a class and an instance of that class, and there are multiple masks for each instance of a class. We can use [facebook/mask2former-swin-large-cityscapes-panoptic](https://huggingface.co/facebook/mask2former-swin-large-cityscapes-panoptic) for this.\n\n```python\npanoptic_segmentation = pipeline(\"image-segmentation\", \"facebook/mask2former-swin-large-cityscapes-panoptic\")\nresults = panoptic_segmentation(image)\nresults\n```\nAs you can see below, we have more classes. We will later illustrate to see that every pixel is classified into one of the classes.\n\n```bash\n[{'score': 0.999981,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999958,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.99997,\n  'label': 'vegetation',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999575,\n  'label': 'pole',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999958,\n  'label': 'building',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999634,\n  'label': 'road',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.996092,\n  'label': 'sidewalk',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.999221,\n  'label': 'car',\n  'mask': <PIL.Image.Image image mode=L size=612x415>},\n {'score': 0.99987,\n  'label': 'sky',\n  'mask': <PIL.Image.Image image mode=L size=612x415>}]\n```\n\nLet's have a side by side comparison for all types of segmentation.\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation-comparison.png\" alt=\"Segmentation Maps Compared\"/>\n</div>\n\nSeeing all types of segmentation, let's have a deep dive on fine-tuning a model for semantic segmentation."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 46d69b62-6b0f-4708-a807-22fd760cc552
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"46d69b62-6b0f-4708-a807-22fd760cc552\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"09c7cf61-8a8b-465f-9766-d8a560b4ec8a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"43c8a8cd4ee7f004e637d86c2997a65ee5ed3d66279d994c9d554840111c7649\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"14fb1e06-fb53-40a0-b1bc-3fe2f9b49b6e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"889e3b3d6625a79644a3da7340abdbaa71aa3916a57d49cd34bb75056a3950e2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6721, \"end_char_idx\": 9110, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/segmentation-comparison.png\" alt=\"Segmentation Maps Compared\"/>\n</div>\n\nSeeing all types of segmentation, let's have a deep dive on fine-tuning a model for semantic segmentation.\n\nCommon real-world applications of semantic segmentation include training self-driving cars to identify pedestrians and important traffic information, identifying cells and abnormalities in medical imagery, and monitoring environmental changes from satellite imagery.\n\n## Fine-tuning a Model for Segmentation\n\nWe will now:\n\n1. Finetune [SegFormer](https://huggingface.co/docs/transformers/main/en/model_doc/segformer#segformer) on the [SceneParse150](https://huggingface.co/datasets/scene_parse_150) dataset.\n2. Use your fine-tuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/image-segmentation)\n\n</Tip>\n\n\n### Load SceneParse150 dataset\n\nStart by loading a smaller subset of the SceneParse150 dataset from the \ud83e\udd17 Datasets library. This'll give you a chance to experiment and make sure everything works before spending more time training on the full dataset.\n\n```py\n>>> from datasets import load_dataset\n\n>>> ds = load_dataset(\"scene_parse_150\", split=\"train[:50]\")\n```\n\nSplit the dataset's `train` split into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n```py\n>>> ds = ds.train_test_split(test_size=0.2)\n>>> train_ds = ds[\"train\"]\n>>> test_ds = ds[\"test\"]\n```\n\nThen take a look at an example:\n\n```py\n>>> train_ds[0]\n{'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=512x683 at 0x7F9B0C201F90>,\n 'annotation': <PIL.PngImagePlugin.PngImageFile image mode=L size=512x683 at 0x7F9B0C201DD0>,\n 'scene_category': 368}\n\n# view the image\n>>> train_ds[0][\"image\"]\n```\n\n- `image`: a PIL image of the scene.\n- `annotation`: a PIL image of the segmentation map, which is also the model's target.\n- `scene_category`: a category id that describes the image scene like \"kitchen\" or \"office\". In this guide, you'll only need `image` and `annotation`, both of which are PIL images.\n\nYou'll also want to create a dictionary that maps a label id to a label class which will be useful when you set up the model later."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 14fb1e06-fb53-40a0-b1bc-3fe2f9b49b6e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"14fb1e06-fb53-40a0-b1bc-3fe2f9b49b6e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"46d69b62-6b0f-4708-a807-22fd760cc552\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"997df885267a99b92a30e5b060c6a9a7b95111e27c1d5a44784f8932016f6fc7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e1bccd4a-7522-45ff-83c5-c7f3afdc5fff\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1f9d8ee40e2d2fbf13dd9983ec089b2be97e662ce07b844c9119aa24aba5ec27\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7600, \"end_char_idx\": 9916, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/image-segmentation)\n\n</Tip>\n\n\n### Load SceneParse150 dataset\n\nStart by loading a smaller subset of the SceneParse150 dataset from the \ud83e\udd17 Datasets library. This'll give you a chance to experiment and make sure everything works before spending more time training on the full dataset.\n\n```py\n>>> from datasets import load_dataset\n\n>>> ds = load_dataset(\"scene_parse_150\", split=\"train[:50]\")\n```\n\nSplit the dataset's `train` split into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n```py\n>>> ds = ds.train_test_split(test_size=0.2)\n>>> train_ds = ds[\"train\"]\n>>> test_ds = ds[\"test\"]\n```\n\nThen take a look at an example:\n\n```py\n>>> train_ds[0]\n{'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=512x683 at 0x7F9B0C201F90>,\n 'annotation': <PIL.PngImagePlugin.PngImageFile image mode=L size=512x683 at 0x7F9B0C201DD0>,\n 'scene_category': 368}\n\n# view the image\n>>> train_ds[0][\"image\"]\n```\n\n- `image`: a PIL image of the scene.\n- `annotation`: a PIL image of the segmentation map, which is also the model's target.\n- `scene_category`: a category id that describes the image scene like \"kitchen\" or \"office\". In this guide, you'll only need `image` and `annotation`, both of which are PIL images.\n\nYou'll also want to create a dictionary that maps a label id to a label class which will be useful when you set up the model later. Download the mappings from the Hub and create the `id2label` and `label2id` dictionaries:\n\n```py\n>>> import json\n>>> from pathlib import Path\n>>> from huggingface_hub import hf_hub_download\n\n>>> repo_id = \"huggingface/label-files\"\n>>> filename = \"ade20k-id2label.json\"\n>>> id2label = json.loads(Path(hf_hub_download(repo_id, filename, repo_type=\"dataset\")).read_text())\n>>> id2label = {int(k): v for k, v in id2label.items()}\n>>> label2id = {v: k for k, v in id2label.items()}\n>>> num_labels = len(id2label)\n```\n\n#### Custom dataset\n\nYou could also create and use your own dataset if you prefer to train with the [run_semantic_segmentation.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/semantic-segmentation/run_semantic_segmentation.py) script instead of a notebook instance."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: e1bccd4a-7522-45ff-83c5-c7f3afdc5fff
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"e1bccd4a-7522-45ff-83c5-c7f3afdc5fff\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"14fb1e06-fb53-40a0-b1bc-3fe2f9b49b6e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"6b6290eec4cf3f4dcf021af475cdd0db68fafac2a081cea325d85bac3780327b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f7ff712f-bcb7-4b07-b602-322b67b301b5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"42d94ed0b47c326fd19a6ff0cea5f0ad6679e4b4386896470be65edb34dbec02\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8710, \"end_char_idx\": 11901, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "- `annotation`: a PIL image of the segmentation map, which is also the model's target.\n- `scene_category`: a category id that describes the image scene like \"kitchen\" or \"office\". In this guide, you'll only need `image` and `annotation`, both of which are PIL images.\n\nYou'll also want to create a dictionary that maps a label id to a label class which will be useful when you set up the model later. Download the mappings from the Hub and create the `id2label` and `label2id` dictionaries:\n\n```py\n>>> import json\n>>> from pathlib import Path\n>>> from huggingface_hub import hf_hub_download\n\n>>> repo_id = \"huggingface/label-files\"\n>>> filename = \"ade20k-id2label.json\"\n>>> id2label = json.loads(Path(hf_hub_download(repo_id, filename, repo_type=\"dataset\")).read_text())\n>>> id2label = {int(k): v for k, v in id2label.items()}\n>>> label2id = {v: k for k, v in id2label.items()}\n>>> num_labels = len(id2label)\n```\n\n#### Custom dataset\n\nYou could also create and use your own dataset if you prefer to train with the [run_semantic_segmentation.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/semantic-segmentation/run_semantic_segmentation.py) script instead of a notebook instance. The script requires:\n\n1. a [`~datasets.DatasetDict`] with two [`~datasets.Image`] columns, \"image\" and \"label\"\n\n     ```py\n     from datasets import Dataset, DatasetDict, Image\n\n     image_paths_train = [\"path/to/image_1.jpg/jpg\", \"path/to/image_2.jpg/jpg\", ..., \"path/to/image_n.jpg/jpg\"]\n     label_paths_train = [\"path/to/annotation_1.png\", \"path/to/annotation_2.png\", ..., \"path/to/annotation_n.png\"]\n\n     image_paths_validation = [...]\n     label_paths_validation = [...]\n\n     def create_dataset(image_paths, label_paths):\n         dataset = Dataset.from_dict({\"image\": sorted(image_paths),\n                                     \"label\": sorted(label_paths)})\n         dataset = dataset.cast_column(\"image\", Image())\n         dataset = dataset.cast_column(\"label\", Image())\n         return dataset\n\n     # step 1: create Dataset objects\n     train_dataset = create_dataset(image_paths_train, label_paths_train)\n     validation_dataset = create_dataset(image_paths_validation, label_paths_validation)\n\n     # step 2: create DatasetDict\n     dataset = DatasetDict({\n          \"train\": train_dataset,\n          \"validation\": validation_dataset,\n          }\n     )\n\n     # step 3: push to Hub (assumes you have ran the huggingface-cli login command in a terminal/notebook)\n     dataset.push_to_hub(\"your-name/dataset-repo\")\n\n     # optionally, you can push to a private repo on the Hub\n     # dataset.push_to_hub(\"name of repo on the hub\", private=True)\n     ```\n\n2. an id2label dictionary mapping the class integers to their class names\n\n     ```py\n     import json\n     # simple example\n     id2label = {0: 'cat', 1: 'dog'}\n     with open('id2label.json', 'w') as fp:\n     json.dump(id2label, fp)\n     ```\n\nAs an example, take a look at this [example dataset](https://huggingface.co/datasets/nielsr/ade20k-demo) which was created with the steps shown above.\n\n### Preprocess\n\nThe next step is to load a SegFormer image processor to prepare the images and annotations for the model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: f7ff712f-bcb7-4b07-b602-322b67b301b5
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"f7ff712f-bcb7-4b07-b602-322b67b301b5\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e1bccd4a-7522-45ff-83c5-c7f3afdc5fff\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"aa7869a99e353b4c272ba282649499def73dfcb786cb358bcc096639f9bfc973\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"677b98c0-b079-4be7-bf4f-8364e89868d5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0c9d533a75316f71ec98e4ecc3c9aa81d70fea132a66dc3e87d91e1ed3be6646\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11780, \"end_char_idx\": 14827, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "### Preprocess\n\nThe next step is to load a SegFormer image processor to prepare the images and annotations for the model. Some datasets, like this one, use the zero-index as the background class. However, the background class isn't actually included in the 150 classes, so you'll need to set `do_reduce_labels=True` to subtract one from all the labels. The zero-index is replaced by `255` so it's ignored by SegFormer's loss function:\n\n```py\n>>> from transformers import AutoImageProcessor\n\n>>> checkpoint = \"nvidia/mit-b0\"\n>>> image_processor = AutoImageProcessor.from_pretrained(checkpoint, do_reduce_labels=True)\n```\n\n<frameworkcontent>\n<pt>\n\nIt is common to apply some data augmentations to an image dataset to make a model more robust against overfitting. In this guide, you'll use the [`ColorJitter`](https://pytorch.org/vision/stable/generated/torchvision.transforms.ColorJitter.html) function from [torchvision](https://pytorch.org/vision/stable/index.html) to randomly change the color properties of an image, but you can also use any image library you like.\n\n```py\n>>> from torchvision.transforms import ColorJitter\n\n>>> jitter = ColorJitter(brightness=0.25, contrast=0.25, saturation=0.25, hue=0.1)\n```\n\nNow create two preprocessing functions to prepare the images and annotations for the model. These functions convert the images into `pixel_values` and annotations to `labels`. For the training set, `jitter` is applied before providing the images to the image processor. For the test set, the image processor crops and normalizes the `images`, and only crops the `labels` because no data augmentation is applied during testing.\n\n```py\n>>> def train_transforms(example_batch):\n...     images = [jitter(x) for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n\n\n>>> def val_transforms(example_batch):\n...     images = [x for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n```\n\nTo apply the `jitter` over the entire dataset, use the \ud83e\udd17 Datasets [`~datasets.Dataset.set_transform`] function. The transform is applied on the fly which is faster and consumes less disk space:\n\n```py\n>>> train_ds.set_transform(train_transforms)\n>>> test_ds.set_transform(val_transforms)\n```\n\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\nIt is common to apply some data augmentations to an image dataset to make a model more robust against overfitting.\nIn this guide, you'll use [`tf.image`](https://www.tensorflow.org/api_docs/python/tf/image) to randomly change the color properties of an image, but you can also use any image\nlibrary you like.\nDefine two separate transformation functions:\n- training data transformations that include image augmentation\n- validation data transformations that only transpose the images, since computer vision models in \ud83e\udd17 Transformers expect channels-first layout\n\n```py\n>>> import tensorflow as tf"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 677b98c0-b079-4be7-bf4f-8364e89868d5
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"677b98c0-b079-4be7-bf4f-8364e89868d5\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f7ff712f-bcb7-4b07-b602-322b67b301b5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"0d08270beb704ce04c29d4470ee78a61335bad2e1bfb8310d66a8a615c20a488\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"611b2f8a-f94e-4664-b3f4-852054844982\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"32889705aca16e332ef49ef69a1e05e88bdfef77fdf4ac66a6fa3b27f58b8bb9\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13087, \"end_char_idx\": 15955, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "These functions convert the images into `pixel_values` and annotations to `labels`. For the training set, `jitter` is applied before providing the images to the image processor. For the test set, the image processor crops and normalizes the `images`, and only crops the `labels` because no data augmentation is applied during testing.\n\n```py\n>>> def train_transforms(example_batch):\n...     images = [jitter(x) for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n\n\n>>> def val_transforms(example_batch):\n...     images = [x for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n```\n\nTo apply the `jitter` over the entire dataset, use the \ud83e\udd17 Datasets [`~datasets.Dataset.set_transform`] function. The transform is applied on the fly which is faster and consumes less disk space:\n\n```py\n>>> train_ds.set_transform(train_transforms)\n>>> test_ds.set_transform(val_transforms)\n```\n\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\nIt is common to apply some data augmentations to an image dataset to make a model more robust against overfitting.\nIn this guide, you'll use [`tf.image`](https://www.tensorflow.org/api_docs/python/tf/image) to randomly change the color properties of an image, but you can also use any image\nlibrary you like.\nDefine two separate transformation functions:\n- training data transformations that include image augmentation\n- validation data transformations that only transpose the images, since computer vision models in \ud83e\udd17 Transformers expect channels-first layout\n\n```py\n>>> import tensorflow as tf\n\n\n>>> def aug_transforms(image):\n...     image = tf.keras.utils.img_to_array(image)\n...     image = tf.image.random_brightness(image, 0.25)\n...     image = tf.image.random_contrast(image, 0.5, 2.0)\n...     image = tf.image.random_saturation(image, 0.75, 1.25)\n...     image = tf.image.random_hue(image, 0.1)\n...     image = tf.transpose(image, (2, 0, 1))\n...     return image\n\n\n>>> def transforms(image):\n...     image = tf.keras.utils.img_to_array(image)\n...     image = tf.transpose(image, (2, 0, 1))\n...     return image\n```\n\nNext, create two preprocessing functions to prepare batches of images and annotations for the model. These functions apply\nthe image transformations and use the earlier loaded `image_processor` to convert the images into `pixel_values` and\nannotations to `labels`. `ImageProcessor` also takes care of resizing and normalizing the images.\n\n```py\n>>> def train_transforms(example_batch):\n...     images = [aug_transforms(x.convert(\"RGB\")) for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 611b2f8a-f94e-4664-b3f4-852054844982
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"611b2f8a-f94e-4664-b3f4-852054844982\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"677b98c0-b079-4be7-bf4f-8364e89868d5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"24a96b9c6e47a6a34ce21abf93dc701d6e2e255c230f69f59bc6608b6e77b5c3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3827d503-59df-4053-8798-d94fca662e62\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f64919519d54ee46dcc789813149f5d5cf857cdbd88e3dde70f071cc4d69c7f8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 14830, \"end_char_idx\": 18339, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": ">>> def aug_transforms(image):\n...     image = tf.keras.utils.img_to_array(image)\n...     image = tf.image.random_brightness(image, 0.25)\n...     image = tf.image.random_contrast(image, 0.5, 2.0)\n...     image = tf.image.random_saturation(image, 0.75, 1.25)\n...     image = tf.image.random_hue(image, 0.1)\n...     image = tf.transpose(image, (2, 0, 1))\n...     return image\n\n\n>>> def transforms(image):\n...     image = tf.keras.utils.img_to_array(image)\n...     image = tf.transpose(image, (2, 0, 1))\n...     return image\n```\n\nNext, create two preprocessing functions to prepare batches of images and annotations for the model. These functions apply\nthe image transformations and use the earlier loaded `image_processor` to convert the images into `pixel_values` and\nannotations to `labels`. `ImageProcessor` also takes care of resizing and normalizing the images.\n\n```py\n>>> def train_transforms(example_batch):\n...     images = [aug_transforms(x.convert(\"RGB\")) for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n\n\n>>> def val_transforms(example_batch):\n...     images = [transforms(x.convert(\"RGB\")) for x in example_batch[\"image\"]]\n...     labels = [x for x in example_batch[\"annotation\"]]\n...     inputs = image_processor(images, labels)\n...     return inputs\n```\n\nTo apply the preprocessing transformations over the entire dataset, use the \ud83e\udd17 Datasets [`~datasets.Dataset.set_transform`] function.\nThe transform is applied on the fly which is faster and consumes less disk space:\n\n```py\n>>> train_ds.set_transform(train_transforms)\n>>> test_ds.set_transform(val_transforms)\n```\n</tf>\n</frameworkcontent>\n\n### Evaluate\n\nIncluding a metric during training is often helpful for evaluating your model's performance. You can quickly load an evaluation method with the \ud83e\udd17 [Evaluate](https://huggingface.co/docs/evaluate/index) library. For this task, load the [mean Intersection over Union](https://huggingface.co/spaces/evaluate-metric/accuracy) (IoU) metric (see the \ud83e\udd17 Evaluate [quick tour](https://huggingface.co/docs/evaluate/a_quick_tour) to learn more about how to load and compute a metric):\n\n```py\n>>> import evaluate\n\n>>> metric = evaluate.load(\"mean_iou\")\n```\n\nThen create a function to [`~evaluate.EvaluationModule.compute`] the metrics. Your predictions need to be converted to\nlogits first, and then reshaped to match the size of the labels before you can call [`~evaluate.EvaluationModule.compute`]:\n\n<frameworkcontent>\n<pt>\n\n```py\n>>> import numpy as np\n>>> import torch\n>>> from torch import nn\n\n>>> def compute_metrics(eval_pred):\n...     with torch.no_grad():\n...         logits, labels = eval_pred\n...         logits_tensor = torch.from_numpy(logits)\n...         logits_tensor = nn.functional.interpolate(\n...             logits_tensor,\n...             size=labels.shape[-2:],\n...             mode=\"bilinear\",\n...             align_corners=False,\n...         ).argmax(dim=1)\n\n...         pred_labels = logits_tensor.detach().cpu().numpy()\n...         metrics = metric.compute(\n...             predictions=pred_labels,\n...             references=labels,\n...             num_labels=num_labels,\n...             ignore_index=255,\n...             reduce_labels=False,\n...         )\n...         for key, value in metrics.items():\n...             if isinstance(value, np.ndarray):\n...                 metrics[key] = value.tolist()\n...         return metrics\n```\n\n</pt>\n</frameworkcontent>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 3827d503-59df-4053-8798-d94fca662e62
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"3827d503-59df-4053-8798-d94fca662e62\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"611b2f8a-f94e-4664-b3f4-852054844982\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"33c991f84f091f412b0d91f7bb70a35f8ca8b77f7d1a2221faa1253eb1db12ff\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"0d01e300-31dd-456d-9dcd-7f6be89e1250\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"ee45fd45fe330e33ea671d3741c48858a1725e958bad49742727131c00d3fac7\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18342, \"end_char_idx\": 20884, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "<frameworkcontent>\n<tf>\n\n```py\n>>> def compute_metrics(eval_pred):\n...     logits, labels = eval_pred\n...     logits = tf.transpose(logits, perm=[0, 2, 3, 1])\n...     logits_resized = tf.image.resize(\n...         logits,\n...         size=tf.shape(labels)[1:],\n...         method=\"bilinear\",\n...     )\n\n...     pred_labels = tf.argmax(logits_resized, axis=-1)\n...     metrics = metric.compute(\n...         predictions=pred_labels,\n...         references=labels,\n...         num_labels=num_labels,\n...         ignore_index=-1,\n...         reduce_labels=image_processor.do_reduce_labels,\n...     )\n\n...     per_category_accuracy = metrics.pop(\"per_category_accuracy\").tolist()\n...     per_category_iou = metrics.pop(\"per_category_iou\").tolist()\n\n...     metrics.update({f\"accuracy_{id2label[i]}\": v for i, v in enumerate(per_category_accuracy)})\n...     metrics.update({f\"iou_{id2label[i]}\": v for i, v in enumerate(per_category_iou)})\n...     return {\"val_\" + k: v for k, v in metrics.items()}\n```\n\n</tf>\n</frameworkcontent>\n\nYour `compute_metrics` function is ready to go now, and you'll return to it when you setup your training.\n\n### Train\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#finetune-with-trainer)!\n\n</Tip>\n\nYou're ready to start training your model now! Load SegFormer with [`AutoModelForSemanticSegmentation`], and pass the model the mapping between label ids and label classes:\n\n```py\n>>> from transformers import AutoModelForSemanticSegmentation, TrainingArguments, Trainer\n\n>>> model = AutoModelForSemanticSegmentation.from_pretrained(checkpoint, id2label=id2label, label2id=label2id)\n```\n\nAt this point, only three steps remain:\n\n1. Define your training hyperparameters in [`TrainingArguments`]. It is important you don't remove unused columns because this'll drop the `image` column. Without the `image` column, you can't create `pixel_values`. Set `remove_unused_columns=False` to prevent this behavior! The only other required parameter is `output_dir` which specifies where to save your model. You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the IoU metric and save the training checkpoint.\n2. Pass the training arguments to [`Trainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 0d01e300-31dd-456d-9dcd-7f6be89e1250
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"0d01e300-31dd-456d-9dcd-7f6be89e1250\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3827d503-59df-4053-8798-d94fca662e62\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"aab139918b1c21b544f60d4e39a5acc5b3ad675b1896b056747981c6fa69c115\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c48dd07c-026a-4a78-ba2a-260bae119a78\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"99650b7cf77a92208f6a77c57a65980fa003762a9a1fe65c26224f0e021c9099\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 19473, \"end_char_idx\": 22350, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "### Train\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#finetune-with-trainer)!\n\n</Tip>\n\nYou're ready to start training your model now! Load SegFormer with [`AutoModelForSemanticSegmentation`], and pass the model the mapping between label ids and label classes:\n\n```py\n>>> from transformers import AutoModelForSemanticSegmentation, TrainingArguments, Trainer\n\n>>> model = AutoModelForSemanticSegmentation.from_pretrained(checkpoint, id2label=id2label, label2id=label2id)\n```\n\nAt this point, only three steps remain:\n\n1. Define your training hyperparameters in [`TrainingArguments`]. It is important you don't remove unused columns because this'll drop the `image` column. Without the `image` column, you can't create `pixel_values`. Set `remove_unused_columns=False` to prevent this behavior! The only other required parameter is `output_dir` which specifies where to save your model. You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the IoU metric and save the training checkpoint.\n2. Pass the training arguments to [`Trainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = TrainingArguments(\n...     output_dir=\"segformer-b0-scene-parse-150\",\n...     learning_rate=6e-5,\n...     num_train_epochs=50,\n...     per_device_train_batch_size=2,\n...     per_device_eval_batch_size=2,\n...     save_total_limit=3,\n...     eval_strategy=\"steps\",\n...     save_strategy=\"steps\",\n...     save_steps=20,\n...     eval_steps=20,\n...     logging_steps=1,\n...     eval_accumulation_steps=5,\n...     remove_unused_columns=False,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Trainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=train_ds,\n...     eval_dataset=test_ds,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\n<Tip>\n\nIf you are unfamiliar with fine-tuning a model with Keras, check out the [basic tutorial](./training#train-a-tensorflow-model-with-keras) first!\n\n</Tip>\n\nTo fine-tune a model in TensorFlow, follow these steps:\n1. Define the training hyperparameters, and set up an optimizer and a learning rate schedule.\n2. Instantiate a pretrained model.\n3. Convert a \ud83e\udd17 Dataset to a `tf.data.Dataset`.\n4. Compile your model.\n5. Add callbacks to calculate metrics and upload your model to \ud83e\udd17 Hub\n6. Use the `fit()` method to run the training."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c48dd07c-026a-4a78-ba2a-260bae119a78
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"c48dd07c-026a-4a78-ba2a-260bae119a78\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"0d01e300-31dd-456d-9dcd-7f6be89e1250\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"e8d9d759fea8cd60ef4148356d4bba788fe62b2a841f312e0f3f0d9fe1937586\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4f019c86-884e-4b4f-997e-863a0a9d9ff7\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7029ad31362defd4ebe54fe57f2d3d563e0cab9abf5c0c5e242e680f5e19f722\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 20834, \"end_char_idx\": 23366, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = TrainingArguments(\n...     output_dir=\"segformer-b0-scene-parse-150\",\n...     learning_rate=6e-5,\n...     num_train_epochs=50,\n...     per_device_train_batch_size=2,\n...     per_device_eval_batch_size=2,\n...     save_total_limit=3,\n...     eval_strategy=\"steps\",\n...     save_strategy=\"steps\",\n...     save_steps=20,\n...     eval_steps=20,\n...     logging_steps=1,\n...     eval_accumulation_steps=5,\n...     remove_unused_columns=False,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Trainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=train_ds,\n...     eval_dataset=test_ds,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\n<Tip>\n\nIf you are unfamiliar with fine-tuning a model with Keras, check out the [basic tutorial](./training#train-a-tensorflow-model-with-keras) first!\n\n</Tip>\n\nTo fine-tune a model in TensorFlow, follow these steps:\n1. Define the training hyperparameters, and set up an optimizer and a learning rate schedule.\n2. Instantiate a pretrained model.\n3. Convert a \ud83e\udd17 Dataset to a `tf.data.Dataset`.\n4. Compile your model.\n5. Add callbacks to calculate metrics and upload your model to \ud83e\udd17 Hub\n6. Use the `fit()` method to run the training.\n\nStart by defining the hyperparameters, optimizer and learning rate schedule:\n\n```py\n>>> from transformers import create_optimizer\n\n>>> batch_size = 2\n>>> num_epochs = 50\n>>> num_train_steps = len(train_ds) * num_epochs\n>>> learning_rate = 6e-5\n>>> weight_decay_rate = 0.01\n\n>>> optimizer, lr_schedule = create_optimizer(\n...     init_lr=learning_rate,\n...     num_train_steps=num_train_steps,\n...     weight_decay_rate=weight_decay_rate,\n...     num_warmup_steps=0,\n... )\n```\n\nThen, load SegFormer with [`TFAutoModelForSemanticSegmentation`] along with the label mappings, and compile it with the\noptimizer. Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> from transformers import TFAutoModelForSemanticSegmentation\n\n>>> model = TFAutoModelForSemanticSegmentation.from_pretrained(\n...     checkpoint,\n...     id2label=id2label,\n...     label2id=label2id,\n... )\n>>> model.compile(optimizer=optimizer)  # No loss argument!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4f019c86-884e-4b4f-997e-863a0a9d9ff7
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"4f019c86-884e-4b4f-997e-863a0a9d9ff7\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c48dd07c-026a-4a78-ba2a-260bae119a78\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"65218c2ea38b83146cb36a8927bb2d78a8d5a1d2f373600b1943939059f8e5d4\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"2f3b46e1-7953-4f53-bb18-5cd3fc855c71\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"9a49d4c667aab8e8cff7116217554b6f2008b61206e7c039915cb2d739b64817\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 21972, \"end_char_idx\": 25213, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "</Tip>\n\nTo fine-tune a model in TensorFlow, follow these steps:\n1. Define the training hyperparameters, and set up an optimizer and a learning rate schedule.\n2. Instantiate a pretrained model.\n3. Convert a \ud83e\udd17 Dataset to a `tf.data.Dataset`.\n4. Compile your model.\n5. Add callbacks to calculate metrics and upload your model to \ud83e\udd17 Hub\n6. Use the `fit()` method to run the training.\n\nStart by defining the hyperparameters, optimizer and learning rate schedule:\n\n```py\n>>> from transformers import create_optimizer\n\n>>> batch_size = 2\n>>> num_epochs = 50\n>>> num_train_steps = len(train_ds) * num_epochs\n>>> learning_rate = 6e-5\n>>> weight_decay_rate = 0.01\n\n>>> optimizer, lr_schedule = create_optimizer(\n...     init_lr=learning_rate,\n...     num_train_steps=num_train_steps,\n...     weight_decay_rate=weight_decay_rate,\n...     num_warmup_steps=0,\n... )\n```\n\nThen, load SegFormer with [`TFAutoModelForSemanticSegmentation`] along with the label mappings, and compile it with the\noptimizer. Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> from transformers import TFAutoModelForSemanticSegmentation\n\n>>> model = TFAutoModelForSemanticSegmentation.from_pretrained(\n...     checkpoint,\n...     id2label=id2label,\n...     label2id=label2id,\n... )\n>>> model.compile(optimizer=optimizer)  # No loss argument!\n```\n\nConvert your datasets to the `tf.data.Dataset` format using the [`~datasets.Dataset.to_tf_dataset`] and the [`DefaultDataCollator`]:\n\n```py\n>>> from transformers import DefaultDataCollator\n\n>>> data_collator = DefaultDataCollator(return_tensors=\"tf\")\n\n>>> tf_train_dataset = train_ds.to_tf_dataset(\n...     columns=[\"pixel_values\", \"label\"],\n...     shuffle=True,\n...     batch_size=batch_size,\n...     collate_fn=data_collator,\n... )\n\n>>> tf_eval_dataset = test_ds.to_tf_dataset(\n...     columns=[\"pixel_values\", \"label\"],\n...     shuffle=True,\n...     batch_size=batch_size,\n...     collate_fn=data_collator,\n... )\n```\n\nTo compute the accuracy from the predictions and push your model to the \ud83e\udd17 Hub, use [Keras callbacks](../main_classes/keras_callbacks).\nPass your `compute_metrics` function to [`KerasMetricCallback`],\nand use the [`PushToHubCallback`] to upload the model:\n\n```py\n>>> from transformers.keras_callbacks import KerasMetricCallback, PushToHubCallback\n\n>>> metric_callback = KerasMetricCallback(\n...     metric_fn=compute_metrics, eval_dataset=tf_eval_dataset, batch_size=batch_size, label_cols=[\"labels\"]\n... )\n\n>>> push_to_hub_callback = PushToHubCallback(output_dir=\"scene_segmentation\", tokenizer=image_processor)\n\n>>> callbacks = [metric_callback, push_to_hub_callback]\n```\n\nFinally, you are ready to train your model! Call `fit()` with your training and validation datasets, the number of epochs,\nand your callbacks to fine-tune the model:\n\n```py\n>>> model.fit(\n...     tf_train_dataset,\n...     validation_data=tf_eval_dataset,\n...     callbacks=callbacks,\n...     epochs=num_epochs,\n... )\n```\n\nCongratulations! You have fine-tuned your model and shared it on the \ud83e\udd17 Hub. You can now use it for inference!\n</tf>\n</frameworkcontent>\n\n### Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 2f3b46e1-7953-4f53-bb18-5cd3fc855c71
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"2f3b46e1-7953-4f53-bb18-5cd3fc855c71\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4f019c86-884e-4b4f-997e-863a0a9d9ff7\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"fe5eefbf1a32d7c8903d578a103ed34e84bb98770f76cd1ff9d5d6f4af66f6d7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b666c1f0-6d88-4fb4-b894-c1d4006d2c2d\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7b6c332f760af9acedcc19b651959283d44cfbfdb2f81cdbc404410fce0a7921\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 24129, \"end_char_idx\": 25749, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "Pass your `compute_metrics` function to [`KerasMetricCallback`],\nand use the [`PushToHubCallback`] to upload the model:\n\n```py\n>>> from transformers.keras_callbacks import KerasMetricCallback, PushToHubCallback\n\n>>> metric_callback = KerasMetricCallback(\n...     metric_fn=compute_metrics, eval_dataset=tf_eval_dataset, batch_size=batch_size, label_cols=[\"labels\"]\n... )\n\n>>> push_to_hub_callback = PushToHubCallback(output_dir=\"scene_segmentation\", tokenizer=image_processor)\n\n>>> callbacks = [metric_callback, push_to_hub_callback]\n```\n\nFinally, you are ready to train your model! Call `fit()` with your training and validation datasets, the number of epochs,\nand your callbacks to fine-tune the model:\n\n```py\n>>> model.fit(\n...     tf_train_dataset,\n...     validation_data=tf_eval_dataset,\n...     callbacks=callbacks,\n...     epochs=num_epochs,\n... )\n```\n\nCongratulations! You have fine-tuned your model and shared it on the \ud83e\udd17 Hub. You can now use it for inference!\n</tf>\n</frameworkcontent>\n\n### Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nReload the dataset and load an image for inference.\n\n```py\n>>> from datasets import load_dataset\n\n>>> ds = load_dataset(\"scene_parse_150\", split=\"train[:50]\")\n>>> ds = ds.train_test_split(test_size=0.2)\n>>> test_ds = ds[\"test\"]\n>>> image = ds[\"test\"][0][\"image\"]\n>>> image\n```\n\n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/semantic-seg-image.png\" alt=\"Image of bedroom\"/>\n</div>\n\n<frameworkcontent>\n<pt>\n\nWe will now see how to infer without a pipeline."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b666c1f0-6d88-4fb4-b894-c1d4006d2c2d
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"b666c1f0-6d88-4fb4-b894-c1d4006d2c2d\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"2f3b46e1-7953-4f53-bb18-5cd3fc855c71\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"c5ad54cd33f49a77c0a9c777d832b8d5cc1519c815b4031a48d750a2c273684a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d2d30549-53cb-4cc0-a8bf-bd7be49b1ea5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"654a3cc059ce368316d40b0afbe0907eb3c10a56d33a23321eaed3c70be3c3ea\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 24129, \"end_char_idx\": 27390, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "Pass your `compute_metrics` function to [`KerasMetricCallback`],\nand use the [`PushToHubCallback`] to upload the model:\n\n```py\n>>> from transformers.keras_callbacks import KerasMetricCallback, PushToHubCallback\n\n>>> metric_callback = KerasMetricCallback(\n...     metric_fn=compute_metrics, eval_dataset=tf_eval_dataset, batch_size=batch_size, label_cols=[\"labels\"]\n... )\n\n>>> push_to_hub_callback = PushToHubCallback(output_dir=\"scene_segmentation\", tokenizer=image_processor)\n\n>>> callbacks = [metric_callback, push_to_hub_callback]\n```\n\nFinally, you are ready to train your model! Call `fit()` with your training and validation datasets, the number of epochs,\nand your callbacks to fine-tune the model:\n\n```py\n>>> model.fit(\n...     tf_train_dataset,\n...     validation_data=tf_eval_dataset,\n...     callbacks=callbacks,\n...     epochs=num_epochs,\n... )\n```\n\nCongratulations! You have fine-tuned your model and shared it on the \ud83e\udd17 Hub. You can now use it for inference!\n</tf>\n</frameworkcontent>\n\n### Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nReload the dataset and load an image for inference.\n\n```py\n>>> from datasets import load_dataset\n\n>>> ds = load_dataset(\"scene_parse_150\", split=\"train[:50]\")\n>>> ds = ds.train_test_split(test_size=0.2)\n>>> test_ds = ds[\"test\"]\n>>> image = ds[\"test\"][0][\"image\"]\n>>> image\n```\n\n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/semantic-seg-image.png\" alt=\"Image of bedroom\"/>\n</div>\n\n<frameworkcontent>\n<pt>\n\nWe will now see how to infer without a pipeline. Process the image with an image processor and place the `pixel_values` on a GPU:\n\n```py\n>>> device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")  # use GPU if available, otherwise use a CPU\n>>> encoding = image_processor(image, return_tensors=\"pt\")\n>>> pixel_values = encoding.pixel_values.to(device)\n```\n\nPass your input to the model and return the `logits`:\n\n```py\n>>> outputs = model(pixel_values=pixel_values)\n>>> logits = outputs.logits.cpu()\n```\n\nNext, rescale the logits to the original image size:\n\n```py\n>>> upsampled_logits = nn.functional.interpolate(\n...     logits,\n...     size=image.size[::-1],\n...     mode=\"bilinear\",\n...     align_corners=False,\n... )\n\n>>> pred_seg = upsampled_logits.argmax(dim=1)[0]\n```\n\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\nLoad an image processor to preprocess the image and return the input as TensorFlow tensors:\n\n```py\n>>> from transformers import AutoImageProcessor\n\n>>> image_processor = AutoImageProcessor.from_pretrained(\"MariaK/scene_segmentation\")\n>>> inputs = image_processor(image, return_tensors=\"tf\")\n```\n\nPass your input to the model and return the `logits`:\n\n```py\n>>> from transformers import TFAutoModelForSemanticSegmentation\n\n>>> model = TFAutoModelForSemanticSegmentation.from_pretrained(\"MariaK/scene_segmentation\")\n>>> logits = model(**inputs).logits\n```\n\nNext, rescale the logits to the original image size and apply argmax on the class dimension:\n```py\n>>> logits = tf.transpose(logits, [0, 2, 3, 1])\n\n>>> upsampled_logits = tf.image.resize(\n...     logits,\n...     # We reverse the shape of `image` because `image.size` returns width and height."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d2d30549-53cb-4cc0-a8bf-bd7be49b1ea5
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d2d30549-53cb-4cc0-a8bf-bd7be49b1ea5\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b666c1f0-6d88-4fb4-b894-c1d4006d2c2d\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"7081e74c336b02ae379a944be528f76fd07cad6ca41171517c8a11ba526095e0\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4ae58a73-f028-49ff-a89d-30002b1056e1\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c76b62c8021d262578030a4d6397245ab30aeafd02a7d38df60b9c3d9e7f4d1c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25750, \"end_char_idx\": 28315, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "Process the image with an image processor and place the `pixel_values` on a GPU:\n\n```py\n>>> device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")  # use GPU if available, otherwise use a CPU\n>>> encoding = image_processor(image, return_tensors=\"pt\")\n>>> pixel_values = encoding.pixel_values.to(device)\n```\n\nPass your input to the model and return the `logits`:\n\n```py\n>>> outputs = model(pixel_values=pixel_values)\n>>> logits = outputs.logits.cpu()\n```\n\nNext, rescale the logits to the original image size:\n\n```py\n>>> upsampled_logits = nn.functional.interpolate(\n...     logits,\n...     size=image.size[::-1],\n...     mode=\"bilinear\",\n...     align_corners=False,\n... )\n\n>>> pred_seg = upsampled_logits.argmax(dim=1)[0]\n```\n\n</pt>\n</frameworkcontent>\n\n<frameworkcontent>\n<tf>\nLoad an image processor to preprocess the image and return the input as TensorFlow tensors:\n\n```py\n>>> from transformers import AutoImageProcessor\n\n>>> image_processor = AutoImageProcessor.from_pretrained(\"MariaK/scene_segmentation\")\n>>> inputs = image_processor(image, return_tensors=\"tf\")\n```\n\nPass your input to the model and return the `logits`:\n\n```py\n>>> from transformers import TFAutoModelForSemanticSegmentation\n\n>>> model = TFAutoModelForSemanticSegmentation.from_pretrained(\"MariaK/scene_segmentation\")\n>>> logits = model(**inputs).logits\n```\n\nNext, rescale the logits to the original image size and apply argmax on the class dimension:\n```py\n>>> logits = tf.transpose(logits, [0, 2, 3, 1])\n\n>>> upsampled_logits = tf.image.resize(\n...     logits,\n...     # We reverse the shape of `image` because `image.size` returns width and height.\n...     image.size[::-1],\n... )\n\n>>> pred_seg = tf.math.argmax(upsampled_logits, axis=-1)[0]\n```\n\n</tf>\n</frameworkcontent>\n\nTo visualize the results, load the [dataset color palette](https://github.com/tensorflow/models/blob/3f1ca33afe3c1631b733ea7e40c294273b9e406d/research/deeplab/utils/get_dataset_colormap.py#L51) as `ade_palette()` that maps each class to their RGB values.\n\n```py\ndef ade_palette():\n  return np.asarray([\n      [0, 0, 0],\n      [120, 120, 120],\n      [180, 120, 120],\n      [6, 230, 230],\n      [80, 50, 50],\n      [4, 200, 3],\n      [120, 120, 80],\n      [140, 140, 140],\n      [204, 5, 255],\n      [230, 230, 230],\n      [4, 250, 7],\n      [224, 5, 255],\n      [235, 255, 7],\n      [150, 5, 61],\n      [120, 120, 70],\n      [8, 255, 51],\n      [255, 6, 82],\n      [143, 255, 140],\n      [204, 255, 4],\n      [255, 51, 7],\n      [204, 70, 3],\n      [0, 102, 200],\n      [61, 230, 250],\n      [255, 6,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4ae58a73-f028-49ff-a89d-30002b1056e1
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"4ae58a73-f028-49ff-a89d-30002b1056e1\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d2d30549-53cb-4cc0-a8bf-bd7be49b1ea5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"b210651afc2275c70c55697fe703c97d5ca985c691b2515dcf5b30ecad6d9245\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9f61f805-6e1c-4647-a451-98c7a4a17cf3\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8a2243df478a5ee3610ff7bb2c6dfe62393290c9fe20d73c944b8f4849da0583\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 27391, \"end_char_idx\": 29063, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "...     image.size[::-1],\n... )\n\n>>> pred_seg = tf.math.argmax(upsampled_logits, axis=-1)[0]\n```\n\n</tf>\n</frameworkcontent>\n\nTo visualize the results, load the [dataset color palette](https://github.com/tensorflow/models/blob/3f1ca33afe3c1631b733ea7e40c294273b9e406d/research/deeplab/utils/get_dataset_colormap.py#L51) as `ade_palette()` that maps each class to their RGB values.\n\n```py\ndef ade_palette():\n  return np.asarray([\n      [0, 0, 0],\n      [120, 120, 120],\n      [180, 120, 120],\n      [6, 230, 230],\n      [80, 50, 50],\n      [4, 200, 3],\n      [120, 120, 80],\n      [140, 140, 140],\n      [204, 5, 255],\n      [230, 230, 230],\n      [4, 250, 7],\n      [224, 5, 255],\n      [235, 255, 7],\n      [150, 5, 61],\n      [120, 120, 70],\n      [8, 255, 51],\n      [255, 6, 82],\n      [143, 255, 140],\n      [204, 255, 4],\n      [255, 51, 7],\n      [204, 70, 3],\n      [0, 102, 200],\n      [61, 230, 250],\n      [255, 6, 51],\n      [11, 102, 255],\n      [255, 7, 71],\n      [255, 9, 224],\n      [9, 7, 230],\n      [220, 220, 220],\n      [255, 9, 92],\n      [112, 9, 255],\n      [8, 255, 214],\n      [7, 255, 224],\n      [255, 184, 6],\n      [10, 255, 71],\n      [255, 41, 10],\n      [7, 255, 255],\n      [224, 255, 8],\n      [102, 8, 255],\n      [255, 61, 6],\n      [255, 194, 7],\n      [255, 122, 8],\n      [0, 255, 20],\n      [255, 8, 41],\n      [255, 5, 153],\n      [6, 51, 255],\n      [235, 12, 255],\n      [160, 150, 20],\n      [0, 163, 255],\n      [140, 140, 140],\n      [250, 10, 15],\n      [20, 255, 0],\n      [31, 255, 0],\n      [255, 31, 0],\n      [255, 224, 0],\n      [153, 255, 0],\n      [0, 0, 255],\n      [255, 71, 0],\n      [0, 235, 255],\n      [0, 173,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 9f61f805-6e1c-4647-a451-98c7a4a17cf3
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"9f61f805-6e1c-4647-a451-98c7a4a17cf3\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4ae58a73-f028-49ff-a89d-30002b1056e1\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"4d233c881ff7a7be0af0e1bfda61a66d05e8d1a5465a510ebb03d40931a169dc\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"13c0d9bd-f675-4329-9ec8-2d7e637a65c8\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"65527d728b9d77eff1556d1fdd0bdec48b564fcdf2bf2e970072adae620c89b2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 28313, \"end_char_idx\": 29781, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "6, 51],\n      [11, 102, 255],\n      [255, 7, 71],\n      [255, 9, 224],\n      [9, 7, 230],\n      [220, 220, 220],\n      [255, 9, 92],\n      [112, 9, 255],\n      [8, 255, 214],\n      [7, 255, 224],\n      [255, 184, 6],\n      [10, 255, 71],\n      [255, 41, 10],\n      [7, 255, 255],\n      [224, 255, 8],\n      [102, 8, 255],\n      [255, 61, 6],\n      [255, 194, 7],\n      [255, 122, 8],\n      [0, 255, 20],\n      [255, 8, 41],\n      [255, 5, 153],\n      [6, 51, 255],\n      [235, 12, 255],\n      [160, 150, 20],\n      [0, 163, 255],\n      [140, 140, 140],\n      [250, 10, 15],\n      [20, 255, 0],\n      [31, 255, 0],\n      [255, 31, 0],\n      [255, 224, 0],\n      [153, 255, 0],\n      [0, 0, 255],\n      [255, 71, 0],\n      [0, 235, 255],\n      [0, 173, 255],\n      [31, 0, 255],\n      [11, 200, 200],\n      [255, 82, 0],\n      [0, 255, 245],\n      [0, 61, 255],\n      [0, 255, 112],\n      [0, 255, 133],\n      [255, 0, 0],\n      [255, 163, 0],\n      [255, 102, 0],\n      [194, 255, 0],\n      [0, 143, 255],\n      [51, 255, 0],\n      [0, 82, 255],\n      [0, 255, 41],\n      [0, 255, 173],\n      [10, 0, 255],\n      [173, 255, 0],\n      [0, 255, 153],\n      [255, 92, 0],\n      [255, 0, 255],\n      [255, 0, 245],\n      [255, 0, 102],\n      [255, 173, 0],\n      [255, 0, 20],\n      [255, 184, 184],\n      [0, 31, 255],\n      [0, 255, 61],\n      [0, 71, 255],\n      [255, 0, 204],\n      [0, 255, 194],\n      [0, 255, 82],\n      [0, 10, 255],\n      [0, 112, 255],\n      [51,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 13c0d9bd-f675-4329-9ec8-2d7e637a65c8
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"13c0d9bd-f675-4329-9ec8-2d7e637a65c8\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9f61f805-6e1c-4647-a451-98c7a4a17cf3\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"a7eea018a59e9df652e1d36c2338730de37257cdd20f842f82c2bea79a299056\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3cbd0fd9-4502-4210-b051-61d2308c158e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"a1e219b327f2f232ebb37965203f0aa2e13799f883996ce398724fc32606ed52\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 29038, \"end_char_idx\": 30512, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "235, 255],\n      [0, 173, 255],\n      [31, 0, 255],\n      [11, 200, 200],\n      [255, 82, 0],\n      [0, 255, 245],\n      [0, 61, 255],\n      [0, 255, 112],\n      [0, 255, 133],\n      [255, 0, 0],\n      [255, 163, 0],\n      [255, 102, 0],\n      [194, 255, 0],\n      [0, 143, 255],\n      [51, 255, 0],\n      [0, 82, 255],\n      [0, 255, 41],\n      [0, 255, 173],\n      [10, 0, 255],\n      [173, 255, 0],\n      [0, 255, 153],\n      [255, 92, 0],\n      [255, 0, 255],\n      [255, 0, 245],\n      [255, 0, 102],\n      [255, 173, 0],\n      [255, 0, 20],\n      [255, 184, 184],\n      [0, 31, 255],\n      [0, 255, 61],\n      [0, 71, 255],\n      [255, 0, 204],\n      [0, 255, 194],\n      [0, 255, 82],\n      [0, 10, 255],\n      [0, 112, 255],\n      [51, 0, 255],\n      [0, 194, 255],\n      [0, 122, 255],\n      [0, 255, 163],\n      [255, 153, 0],\n      [0, 255, 10],\n      [255, 112, 0],\n      [143, 255, 0],\n      [82, 0, 255],\n      [163, 255, 0],\n      [255, 235, 0],\n      [8, 184, 170],\n      [133, 0, 255],\n      [0, 255, 92],\n      [184, 0, 255],\n      [255, 0, 31],\n      [0, 184, 255],\n      [0, 214, 255],\n      [255, 0, 112],\n      [92, 255, 0],\n      [0, 224, 255],\n      [112, 224, 255],\n      [70, 184, 160],\n      [163, 0, 255],\n      [153, 0, 255],\n      [71, 255, 0],\n      [255, 0, 163],\n      [255, 204, 0],\n      [255, 0, 143],\n      [0, 255, 235],\n      [133, 255, 0],\n      [255, 0, 235],\n      [245, 0, 255],\n      [255, 0, 122],\n      [255, 245, 0],\n      [10,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 3cbd0fd9-4502-4210-b051-61d2308c158e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"3cbd0fd9-4502-4210-b051-61d2308c158e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"13c0d9bd-f675-4329-9ec8-2d7e637a65c8\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"419c9751cc3eea09f8fb17102abd2990196f60f772d5328fbcbfe85cf96742a7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"eecd007d-bc92-4299-a6d7-7fa24fb4f1ca\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8820c311ab122c4c017562b695f18cc3f2cf85f41d7f71e754c9f88f6b5c3036\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "112, 255],\n      [51, 0, 255],\n      [0, 194, 255],\n      [0, 122, 255],\n      [0, 255, 163],\n      [255, 153, 0],\n      [0, 255, 10],\n      [255, 112, 0],\n      [143, 255, 0],\n      [82, 0, 255],\n      [163, 255, 0],\n      [255, 235, 0],\n      [8, 184, 170],\n      [133, 0, 255],\n      [0, 255, 92],\n      [184, 0, 255],\n      [255, 0, 31],\n      [0, 184, 255],\n      [0, 214, 255],\n      [255, 0, 112],\n      [92, 255, 0],\n      [0, 224, 255],\n      [112, 224, 255],\n      [70, 184, 160],\n      [163, 0, 255],\n      [153, 0, 255],\n      [71, 255, 0],\n      [255, 0, 163],\n      [255, 204, 0],\n      [255, 0, 143],\n      [0, 255, 235],\n      [133, 255, 0],\n      [255, 0, 235],\n      [245, 0, 255],\n      [255, 0, 122],\n      [255, 245, 0],\n      [10, 190, 212],\n      [214, 255, 0],\n      [0, 204, 255],\n      [20, 0, 255],\n      [255, 255, 0],\n      [0, 153, 255],\n      [0, 41, 255],\n      [0, 255, 204],\n      [41, 0, 255],\n      [41, 255, 0],\n      [173, 0, 255],\n      [0, 245, 255],\n      [71, 0, 255],\n      [122, 0, 255],\n      [0, 255, 184],\n      [0, 92, 255],\n      [184, 255, 0],\n      [0, 133, 255],\n      [255, 214, 0],\n      [25, 194, 194],\n      [102, 255, 0],\n      [92, 0, 255],\n  ])\n```\n\nThen you can combine and plot your image and the predicted segmentation map:\n\n```py\n>>> import matplotlib.pyplot as plt\n>>> import numpy as np\n\n>>> color_seg = np.zeros((pred_seg.shape[0], pred_seg.shape[1], 3), dtype=np.uint8)\n>>> palette = np.array(ade_palette())\n>>> for label, color in enumerate(palette):\n.     color_seg[pred_seg == label, :] = color\n>>> color_seg = color_seg[. ::-1]  # convert to BGR\n\n>>> img = np.array(image) * 0.5 + color_seg * 0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: eecd007d-bc92-4299-a6d7-7fa24fb4f1ca
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/semantic_segmentation", "title": "Image Segmentation", "tokens": 8705, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"eecd007d-bc92-4299-a6d7-7fa24fb4f1ca\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"431596ee-838d-47b9-81a2-0942ac2907eb\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29eb63b4cdaebee7daa5ed72185c11d247fa7bd9fe25b32d4a9cc31bd3bfea2d\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3cbd0fd9-4502-4210-b051-61d2308c158e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/semantic_segmentation\", \"title\": \"Image Segmentation\", \"tokens\": 8705, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"9489197698d5a35cd49a44d8cf073705044b39c44004ca34f708e170daa19e87\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "ref_doc_id": "431596ee-838d-47b9-81a2-0942ac2907eb", "chroma:document": "0, 122],\n      [255, 245, 0],\n      [10, 190, 212],\n      [214, 255, 0],\n      [0, 204, 255],\n      [20, 0, 255],\n      [255, 255, 0],\n      [0, 153, 255],\n      [0, 41, 255],\n      [0, 255, 204],\n      [41, 0, 255],\n      [41, 255, 0],\n      [173, 0, 255],\n      [0, 245, 255],\n      [71, 0, 255],\n      [122, 0, 255],\n      [0, 255, 184],\n      [0, 92, 255],\n      [184, 255, 0],\n      [0, 133, 255],\n      [255, 214, 0],\n      [25, 194, 194],\n      [102, 255, 0],\n      [92, 0, 255],\n  ])\n```\n\nThen you can combine and plot your image and the predicted segmentation map:\n\n```py\n>>> import matplotlib.pyplot as plt\n>>> import numpy as np\n\n>>> color_seg = np.zeros((pred_seg.shape[0], pred_seg.shape[1], 3), dtype=np.uint8)\n>>> palette = np.array(ade_palette())\n>>> for label, color in enumerate(palette):\n.     color_seg[pred_seg == label, :] = color\n>>> color_seg = color_seg[. ::-1]  # convert to BGR\n\n>>> img = np.array(image) * 0.5 + color_seg * 0.5  # plot the image with the segmentation map\n>>> img = img.astype(np.uint8)\n\n>>> plt.figure(figsize=(15, 10))\n>>> plt.imshow(img)\n>>> plt.show()\n```\n\n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/semantic-seg-preds.png\" alt=\"Image of bedroom overlaid with segmentation map\"/>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c8559089-b7b2-4e0a-b47d-b4815242bc2b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_captioning", "title": "Image captioning", "tokens": 1900, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"c8559089-b7b2-4e0a-b47d-b4815242bc2b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"a6d70f2c-878f-4499-88e7-6f7c8d3ed260\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"91c7f9bca67878ef01d86e644d523e7e994743923414f046881da32189729c0f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a9a3874c-daec-4849-b765-582ca2921d35\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c469c64c686b38b6063bb340fd071e6777e3cc9aa72c916bf687c08dc232085d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2354, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "ref_doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "chroma:document": "# Image captioning\n\n[[open-in-colab]]\n\nImage captioning is the task of predicting a caption for a given image. Common real world applications of it include\naiding visually impaired people that can help them navigate through different situations. Therefore, image captioning\nhelps to improve content accessibility for people by describing images to them.\n\nThis guide will show you how to:\n\n* Fine-tune an image captioning model.\n* Use the fine-tuned model for inference. \n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install transformers datasets evaluate -q\npip install jiwer -q\n```\n\nWe encourage you to log in to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to log in:\n\n\n```python\nfrom huggingface_hub import notebook_login\n\nnotebook_login()\n```\n\n## Load the Pok\u00e9mon BLIP captions dataset\n\nUse the \ud83e\udd17 Dataset library to load a dataset that consists of {image-caption} pairs. To create your own image captioning dataset\nin PyTorch, you can follow [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/GIT/Fine_tune_GIT_on_an_image_captioning_dataset.ipynb). \n\n\n```python\nfrom datasets import load_dataset\n\nds = load_dataset(\"lambdalabs/pokemon-blip-captions\")\nds\n```\n```bash\nDatasetDict({\n    train: Dataset({\n        features: ['image', 'text'],\n        num_rows: 833\n    })\n})\n```\n\nThe dataset has two features, `image` and `text`.\n\n<Tip>\n\nMany image captioning datasets contain multiple captions per image. In those cases, a common strategy is to randomly sample a caption amongst the available ones during training. \n\n</Tip>\n\nSplit the dataset\u2019s train split into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n\n```python\nds = ds[\"train\"].train_test_split(test_size=0.1)\ntrain_ds = ds[\"train\"]\ntest_ds = ds[\"test\"]\n```\n\nLet's visualize a couple of samples from the training set. \n\n\n```python\nfrom textwrap import wrap\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n\ndef plot_images(images, captions):\n    plt.figure(figsize=(20, 20))\n    for i in range(len(images)):\n        ax = plt.subplot(1, len(images), i + 1)\n        caption = captions[i]\n        caption = \"\\n\".join(wrap(caption, 12))\n        plt.title(caption)\n        plt.imshow(images[i])\n        plt.axis(\"off\")"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a9a3874c-daec-4849-b765-582ca2921d35
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_captioning", "title": "Image captioning", "tokens": 1900, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a9a3874c-daec-4849-b765-582ca2921d35\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"a6d70f2c-878f-4499-88e7-6f7c8d3ed260\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"91c7f9bca67878ef01d86e644d523e7e994743923414f046881da32189729c0f\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c8559089-b7b2-4e0a-b47d-b4815242bc2b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"63084dee75d6f3ef67a51b598853a291a78897a734d023accc3622aab45067a0\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"03b3e4a2-494e-43d8-9a04-f5cc4b9d7708\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f49ccd7582a09ead0a203446ff9618ecaa9be430bd6807e6068076d0b6a013ae\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 789, \"end_char_idx\": 3953, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "ref_doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "chroma:document": "```python\nfrom huggingface_hub import notebook_login\n\nnotebook_login()\n```\n\n## Load the Pok\u00e9mon BLIP captions dataset\n\nUse the \ud83e\udd17 Dataset library to load a dataset that consists of {image-caption} pairs. To create your own image captioning dataset\nin PyTorch, you can follow [this notebook](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/GIT/Fine_tune_GIT_on_an_image_captioning_dataset.ipynb). \n\n\n```python\nfrom datasets import load_dataset\n\nds = load_dataset(\"lambdalabs/pokemon-blip-captions\")\nds\n```\n```bash\nDatasetDict({\n    train: Dataset({\n        features: ['image', 'text'],\n        num_rows: 833\n    })\n})\n```\n\nThe dataset has two features, `image` and `text`.\n\n<Tip>\n\nMany image captioning datasets contain multiple captions per image. In those cases, a common strategy is to randomly sample a caption amongst the available ones during training. \n\n</Tip>\n\nSplit the dataset\u2019s train split into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n\n```python\nds = ds[\"train\"].train_test_split(test_size=0.1)\ntrain_ds = ds[\"train\"]\ntest_ds = ds[\"test\"]\n```\n\nLet's visualize a couple of samples from the training set. \n\n\n```python\nfrom textwrap import wrap\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n\ndef plot_images(images, captions):\n    plt.figure(figsize=(20, 20))\n    for i in range(len(images)):\n        ax = plt.subplot(1, len(images), i + 1)\n        caption = captions[i]\n        caption = \"\\n\".join(wrap(caption, 12))\n        plt.title(caption)\n        plt.imshow(images[i])\n        plt.axis(\"off\")\n\n\nsample_images_to_visualize = [np.array(train_ds[i][\"image\"]) for i in range(5)]\nsample_captions = [train_ds[i][\"text\"] for i in range(5)]\nplot_images(sample_images_to_visualize, sample_captions)\n```\n    \n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/sample_training_images_image_cap.png\" alt=\"Sample training images\"/>\n</div>\n\n## Preprocess the dataset\n\nSince the dataset has two modalities (image and text), the pre-processing pipeline will preprocess images and the captions.\n\nTo do so, load the processor class associated with the model you are about to fine-tune. \n\n```python\nfrom transformers import AutoProcessor\n\ncheckpoint = \"microsoft/git-base\"\nprocessor = AutoProcessor.from_pretrained(checkpoint)\n```\n\nThe processor will internally pre-process the image (which includes resizing, and pixel scaling) and tokenize the caption. \n\n```python\ndef transforms(example_batch):\n    images = [x for x in example_batch[\"image\"]]\n    captions = [x for x in example_batch[\"text\"]]\n    inputs = processor(images=images, text=captions, padding=\"max_length\")\n    inputs.update({\"labels\": inputs[\"input_ids\"]})\n    return inputs\n\n\ntrain_ds.set_transform(transforms)\ntest_ds.set_transform(transforms)\n```\n\nWith the dataset ready, you can now set up the model for fine-tuning. \n\n## Load a base model\n\nLoad the [\"microsoft/git-base\"](https://huggingface.co/microsoft/git-base) into a [`AutoModelForCausalLM`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM) object."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 03b3e4a2-494e-43d8-9a04-f5cc4b9d7708
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_captioning", "title": "Image captioning", "tokens": 1900, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"03b3e4a2-494e-43d8-9a04-f5cc4b9d7708\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"a6d70f2c-878f-4499-88e7-6f7c8d3ed260\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"91c7f9bca67878ef01d86e644d523e7e994743923414f046881da32189729c0f\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a9a3874c-daec-4849-b765-582ca2921d35\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"a449ca6da5e097c1f201cd14a1cc2d886e1eed3bfd8267933b816b49bfae5acc\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"905da11c-3040-463d-afe4-625a566200ab\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1f1680a6ba28c4f5e499e7141a37120323fd0a735223608b73bd21b35475c047\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2357, \"end_char_idx\": 5146, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "ref_doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "chroma:document": "sample_images_to_visualize = [np.array(train_ds[i][\"image\"]) for i in range(5)]\nsample_captions = [train_ds[i][\"text\"] for i in range(5)]\nplot_images(sample_images_to_visualize, sample_captions)\n```\n    \n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/sample_training_images_image_cap.png\" alt=\"Sample training images\"/>\n</div>\n\n## Preprocess the dataset\n\nSince the dataset has two modalities (image and text), the pre-processing pipeline will preprocess images and the captions.\n\nTo do so, load the processor class associated with the model you are about to fine-tune. \n\n```python\nfrom transformers import AutoProcessor\n\ncheckpoint = \"microsoft/git-base\"\nprocessor = AutoProcessor.from_pretrained(checkpoint)\n```\n\nThe processor will internally pre-process the image (which includes resizing, and pixel scaling) and tokenize the caption. \n\n```python\ndef transforms(example_batch):\n    images = [x for x in example_batch[\"image\"]]\n    captions = [x for x in example_batch[\"text\"]]\n    inputs = processor(images=images, text=captions, padding=\"max_length\")\n    inputs.update({\"labels\": inputs[\"input_ids\"]})\n    return inputs\n\n\ntrain_ds.set_transform(transforms)\ntest_ds.set_transform(transforms)\n```\n\nWith the dataset ready, you can now set up the model for fine-tuning. \n\n## Load a base model\n\nLoad the [\"microsoft/git-base\"](https://huggingface.co/microsoft/git-base) into a [`AutoModelForCausalLM`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM) object.\n\n\n```python\nfrom transformers import AutoModelForCausalLM\n\nmodel = AutoModelForCausalLM.from_pretrained(checkpoint)\n```\n\n## Evaluate\n\nImage captioning models are typically evaluated with the [Rouge Score](https://huggingface.co/spaces/evaluate-metric/rouge) or [Word Error Rate](https://huggingface.co/spaces/evaluate-metric/wer). For this guide, you will use the Word Error Rate (WER). \n\nWe use the \ud83e\udd17 Evaluate library to do so. For potential limitations and other gotchas of the WER, refer to [this guide](https://huggingface.co/spaces/evaluate-metric/wer). \n\n\n```python\nfrom evaluate import load\nimport torch\n\nwer = load(\"wer\")\n\n\ndef compute_metrics(eval_pred):\n    logits, labels = eval_pred\n    predicted = logits.argmax(-1)\n    decoded_labels = processor.batch_decode(labels, skip_special_tokens=True)\n    decoded_predictions = processor.batch_decode(predicted, skip_special_tokens=True)\n    wer_score = wer.compute(predictions=decoded_predictions, references=decoded_labels)\n    return {\"wer_score\": wer_score}\n```\n\n## Train!\n\nNow, you are ready to start fine-tuning the model. You will use the \ud83e\udd17 [`Trainer`] for this. \n\nFirst, define the training arguments using [`TrainingArguments`]."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 905da11c-3040-463d-afe4-625a566200ab
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_captioning", "title": "Image captioning", "tokens": 1900, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"905da11c-3040-463d-afe4-625a566200ab\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"a6d70f2c-878f-4499-88e7-6f7c8d3ed260\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"91c7f9bca67878ef01d86e644d523e7e994743923414f046881da32189729c0f\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"03b3e4a2-494e-43d8-9a04-f5cc4b9d7708\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"1da929197ddfb4dce336d1b008e4ce305de8d79028943830ef5ea9c780dd479f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"868648dc-b8b3-4eaa-b337-b086d60edd70\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d47d08787f0017b2e6b4f1b639a84997b6effc52688873af0686e1c9e9071aed\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3580, \"end_char_idx\": 6411, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "ref_doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "chroma:document": "train_ds.set_transform(transforms)\ntest_ds.set_transform(transforms)\n```\n\nWith the dataset ready, you can now set up the model for fine-tuning. \n\n## Load a base model\n\nLoad the [\"microsoft/git-base\"](https://huggingface.co/microsoft/git-base) into a [`AutoModelForCausalLM`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM) object.\n\n\n```python\nfrom transformers import AutoModelForCausalLM\n\nmodel = AutoModelForCausalLM.from_pretrained(checkpoint)\n```\n\n## Evaluate\n\nImage captioning models are typically evaluated with the [Rouge Score](https://huggingface.co/spaces/evaluate-metric/rouge) or [Word Error Rate](https://huggingface.co/spaces/evaluate-metric/wer). For this guide, you will use the Word Error Rate (WER). \n\nWe use the \ud83e\udd17 Evaluate library to do so. For potential limitations and other gotchas of the WER, refer to [this guide](https://huggingface.co/spaces/evaluate-metric/wer). \n\n\n```python\nfrom evaluate import load\nimport torch\n\nwer = load(\"wer\")\n\n\ndef compute_metrics(eval_pred):\n    logits, labels = eval_pred\n    predicted = logits.argmax(-1)\n    decoded_labels = processor.batch_decode(labels, skip_special_tokens=True)\n    decoded_predictions = processor.batch_decode(predicted, skip_special_tokens=True)\n    wer_score = wer.compute(predictions=decoded_predictions, references=decoded_labels)\n    return {\"wer_score\": wer_score}\n```\n\n## Train!\n\nNow, you are ready to start fine-tuning the model. You will use the \ud83e\udd17 [`Trainer`] for this. \n\nFirst, define the training arguments using [`TrainingArguments`].\n\n\n```python\nfrom transformers import TrainingArguments, Trainer\n\nmodel_name = checkpoint.split(\"/\")[1]\n\ntraining_args = TrainingArguments(\n    output_dir=f\"{model_name}-pokemon\",\n    learning_rate=5e-5,\n    num_train_epochs=50,\n    fp16=True,\n    per_device_train_batch_size=32,\n    per_device_eval_batch_size=32,\n    gradient_accumulation_steps=2,\n    save_total_limit=3,\n    eval_strategy=\"steps\",\n    eval_steps=50,\n    save_strategy=\"steps\",\n    save_steps=50,\n    logging_steps=50,\n    remove_unused_columns=False,\n    push_to_hub=True,\n    label_names=[\"labels\"],\n    load_best_model_at_end=True,\n)\n```\n\nThen pass them along with the datasets and the model to \ud83e\udd17 Trainer. \n\n```python\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=train_ds,\n    eval_dataset=test_ds,\n    compute_metrics=compute_metrics,\n)\n```\n\nTo start training, simply call [`~Trainer.train`] on the [`Trainer`] object.\n\n```python \ntrainer.train()\n```\n\nYou should see the training loss drop smoothly as training progresses.\n\nOnce training is completed, share your model to the Hub with the [`~Trainer.push_to_hub`] method so everyone can use your model:\n\n\n```python\ntrainer.push_to_hub()\n```\n\n## Inference\n\nTake a sample image from `test_ds` to test the model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 868648dc-b8b3-4eaa-b337-b086d60edd70
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_captioning", "title": "Image captioning", "tokens": 1900, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"868648dc-b8b3-4eaa-b337-b086d60edd70\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"a6d70f2c-878f-4499-88e7-6f7c8d3ed260\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"91c7f9bca67878ef01d86e644d523e7e994743923414f046881da32189729c0f\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"905da11c-3040-463d-afe4-625a566200ab\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_captioning\", \"title\": \"Image captioning\", \"tokens\": 1900, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"6d836cea26a90adaeb76467296216abbe5e6145c18c4aba064226f0d6000dbfb\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5149, \"end_char_idx\": 7385, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "ref_doc_id": "a6d70f2c-878f-4499-88e7-6f7c8d3ed260", "chroma:document": "```python\nfrom transformers import TrainingArguments, Trainer\n\nmodel_name = checkpoint.split(\"/\")[1]\n\ntraining_args = TrainingArguments(\n    output_dir=f\"{model_name}-pokemon\",\n    learning_rate=5e-5,\n    num_train_epochs=50,\n    fp16=True,\n    per_device_train_batch_size=32,\n    per_device_eval_batch_size=32,\n    gradient_accumulation_steps=2,\n    save_total_limit=3,\n    eval_strategy=\"steps\",\n    eval_steps=50,\n    save_strategy=\"steps\",\n    save_steps=50,\n    logging_steps=50,\n    remove_unused_columns=False,\n    push_to_hub=True,\n    label_names=[\"labels\"],\n    load_best_model_at_end=True,\n)\n```\n\nThen pass them along with the datasets and the model to \ud83e\udd17 Trainer. \n\n```python\ntrainer = Trainer(\n    model=model,\n    args=training_args,\n    train_dataset=train_ds,\n    eval_dataset=test_ds,\n    compute_metrics=compute_metrics,\n)\n```\n\nTo start training, simply call [`~Trainer.train`] on the [`Trainer`] object.\n\n```python \ntrainer.train()\n```\n\nYou should see the training loss drop smoothly as training progresses.\n\nOnce training is completed, share your model to the Hub with the [`~Trainer.push_to_hub`] method so everyone can use your model:\n\n\n```python\ntrainer.push_to_hub()\n```\n\n## Inference\n\nTake a sample image from `test_ds` to test the model.\n\n\n```python\nfrom PIL import Image\nimport requests\n\nurl = \"https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/pokemon.png\"\nimage = Image.open(requests.get(url, stream=True).raw)\nimage\n```\n\n<div class=\"flex justify-center\">\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/test_image_image_cap.png\" alt=\"Test image\"/>\n</div>\n    \nPrepare image for the model.\n\n```python\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\ninputs = processor(images=image, return_tensors=\"pt\").to(device)\npixel_values = inputs.pixel_values\n```\n\nCall [`generate`] and decode the predictions. \n\n```python\ngenerated_ids = model.generate(pixel_values=pixel_values, max_length=50)\ngenerated_caption = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]\nprint(generated_caption)\n```\n```bash\na drawing of a pink and blue pokemon\n```\n\nLooks like the fine-tuned model generated a pretty good caption!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 40b94134-6662-49cc-bcc1-b89cbdb03f25
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_text_to_text", "title": "Image-text-to-text", "tokens": 1936, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"40b94134-6662-49cc-bcc1-b89cbdb03f25\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e961e63f-f1fb-4f09-90b9-78b506870bf2\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e9c2f2f6611fc285f25ce1b46f84a11985fff7798a572e3349601e8720a6d5de\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9379c6f2-325b-4871-9b36-7ff1fb801e22\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"dfff0139cabab5dd294e0f21af1aacc0a2872d281258f9ba9804f68f50435191\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2561, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "ref_doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "chroma:document": "# Image-text-to-text\n\n[[open-in-colab]]\n\nImage-text-to-text models, also known as vision language models (VLMs), are language models that take an image input. These models can tackle various tasks, from visual question answering to image segmentation. This task shares many similarities with image-to-text, but\u00a0with some overlapping use cases like image captioning. Image-to-text models only take image inputs and often accomplish a specific task, whereas VLMs take open-ended text and image inputs and are more generalist models.\n\nIn this guide, we provide a brief overview of VLMs and show how to use them with Transformers for inference.\n\nTo begin with, there are multiple types of VLMs:\n- base models used for fine-tuning\n- chat fine-tuned models for conversation\n- instruction fine-tuned models\n\nThis guide focuses on inference with an instruction-tuned model. \n\nLet's begin installing the dependencies.\n\n```bash\npip install -q transformers accelerate flash_attn \n```\n\nLet's initialize the model and the processor. \n\n```python\nfrom transformers import AutoProcessor, Idefics2ForConditionalGeneration\nimport torch\n\ndevice = torch.device(\"cuda\")\nmodel = Idefics2ForConditionalGeneration.from_pretrained(\n    \"HuggingFaceM4/idefics2-8b\",\n    torch_dtype=torch.bfloat16,\n    attn_implementation=\"flash_attention_2\",\n).to(device)\n\nprocessor = AutoProcessor.from_pretrained(\"HuggingFaceM4/idefics2-8b\")\n```\n\nThis model has a [chat template](./chat_templating) that helps user parse chat outputs. Moreover, the model can also accept multiple images as input in a single conversation or message. We will now prepare the inputs. \n\nThe image inputs look like the following.\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png\" alt=\"Two cats sitting on a net\"/>\n</div>\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\" alt=\"A bee on a pink flower\"/>\n</div>\n\n\n```python\nfrom PIL import Image\nimport requests\n\nimg_urls =[\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png\",\n           \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\"]\nimages = [Image.open(requests.get(img_urls[0], stream=True).raw),\n          Image.open(requests.get(img_urls[1], stream=True).raw)]\n```\n\nBelow is an example of the chat template. We can feed conversation turns and the last message as an input by appending it at the end of the template."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 9379c6f2-325b-4871-9b36-7ff1fb801e22
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_text_to_text", "title": "Image-text-to-text", "tokens": 1936, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"9379c6f2-325b-4871-9b36-7ff1fb801e22\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e961e63f-f1fb-4f09-90b9-78b506870bf2\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e9c2f2f6611fc285f25ce1b46f84a11985fff7798a572e3349601e8720a6d5de\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"40b94134-6662-49cc-bcc1-b89cbdb03f25\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"afc3fac2f189afead04497a5d766e332c068412ac730efba1e48513a73a56145\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a36d5132-7a95-41ce-9407-bb61db646b0f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"fbd0a5c8b592fadba8b90674eb436326c6b9d909a205e7c764876ad2e3e6cc31\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2025, \"end_char_idx\": 4500, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "ref_doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "chroma:document": "```python\nfrom PIL import Image\nimport requests\n\nimg_urls =[\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/cats.png\",\n           \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg\"]\nimages = [Image.open(requests.get(img_urls[0], stream=True).raw),\n          Image.open(requests.get(img_urls[1], stream=True).raw)]\n```\n\nBelow is an example of the chat template. We can feed conversation turns and the last message as an input by appending it at the end of the template. \n\n\n```python\nmessages = [\n    {\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"image\"},\n            {\"type\": \"text\", \"text\": \"What do we see in this image?\"},\n        ]\n    },\n    {\n        \"role\": \"assistant\",\n        \"content\": [\n            {\"type\": \"text\", \"text\": \"In this image we can see two cats on the nets.\"},\n        ]\n    },\n    {\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"image\"},\n            {\"type\": \"text\", \"text\": \"And how about this image?\"},\n        ]\n    },       \n]\n```\n\nWe will now call the processors' [`~ProcessorMixin.apply_chat_template`] method to preprocess its output along with the image inputs.\n\n```python\nprompt = processor.apply_chat_template(messages, add_generation_prompt=True)\ninputs = processor(text=prompt, images=[images[0], images[1]], return_tensors=\"pt\").to(device)\n```\n\nWe can now pass the preprocessed inputs to the model.\n\n```python\nwith torch.no_grad():\n    generated_ids = model.generate(**inputs, max_new_tokens=500)\ngenerated_texts = processor.batch_decode(generated_ids, skip_special_tokens=True)\n\nprint(generated_texts)\n## ['User: What do we see in this image? \\nAssistant: In this image we can see two cats on the nets. \\nUser: And how about this image? \\nAssistant: In this image we can see flowers, plants and insect.']\n```\n\n## Streaming\n\nWe can use [text streaming](./generation_strategies#streaming) for a better generation experience. Transformers supports streaming with the [`TextStreamer`] or [`TextIteratorStreamer`] classes. We will use the [`TextIteratorStreamer`] with IDEFICS-8B.\n\nAssume we have an application that keeps chat history and takes in the new user input. We will preprocess the inputs as usual and initialize [`TextIteratorStreamer`] to handle the generation in a separate thread. This allows you to stream the generated text tokens in real-time. Any generation arguments can be passed to [`TextIteratorStreamer`]."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a36d5132-7a95-41ce-9407-bb61db646b0f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/image_text_to_text", "title": "Image-text-to-text", "tokens": 1936, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a36d5132-7a95-41ce-9407-bb61db646b0f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"e961e63f-f1fb-4f09-90b9-78b506870bf2\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e9c2f2f6611fc285f25ce1b46f84a11985fff7798a572e3349601e8720a6d5de\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9379c6f2-325b-4871-9b36-7ff1fb801e22\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/image_text_to_text\", \"title\": \"Image-text-to-text\", \"tokens\": 1936, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fc27437c836012644ef191fb31ea21a13ee5e167f68bb367f5654f8d76088f1b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4503, \"end_char_idx\": 7704, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "ref_doc_id": "e961e63f-f1fb-4f09-90b9-78b506870bf2", "chroma:document": "```python\nimport time\nfrom transformers import TextIteratorStreamer\nfrom threading import Thread\n\ndef model_inference(\n    user_prompt,\n    chat_history,\n    max_new_tokens,\n    images\n):\n    user_prompt = {\n        \"role\": \"user\",\n        \"content\": [\n            {\"type\": \"image\"},\n            {\"type\": \"text\", \"text\": user_prompt},\n        ]\n    }\n    chat_history.append(user_prompt)\n    streamer = TextIteratorStreamer(\n        processor.tokenizer,\n        skip_prompt=True,\n        timeout=5.0,\n    )\n\n    generation_args = {\n        \"max_new_tokens\": max_new_tokens,\n        \"streamer\": streamer,\n        \"do_sample\": False\n    }\n\n    # add_generation_prompt=True makes model generate bot response\n    prompt = processor.apply_chat_template(chat_history, add_generation_prompt=True)\n    inputs = processor(\n        text=prompt,\n        images=images,\n        return_tensors=\"pt\",\n    ).to(device)\n    generation_args.update(inputs)\n\n    thread = Thread(\n        target=model.generate,\n        kwargs=generation_args,\n    )\n    thread.start()\n\n    acc_text = \"\"\n    for text_token in streamer:\n        time.sleep(0.04)\n        acc_text += text_token\n        if acc_text.endswith(\"<end_of_utterance>\"):\n            acc_text = acc_text[:-18]\n        yield acc_text\n    \n    thread.join()\n```\n\nNow let's call the `model_inference` function we created and stream the values. \n\n```python\ngenerator = model_inference(\n    user_prompt=\"And what is in this image?\",\n    chat_history=messages,\n    max_new_tokens=100,\n    images=images\n)\n\nfor value in generator:\n  print(value)\n\n# In\n# In this\n# In this image ...\n```\n\n## Fit models in smaller hardware\n\nVLMs are often large and need to be optimized to fit in smaller hardware. Transformers supports many model quantization libraries, and here we will only show int8 quantization with [Quanto](./quantization/quanto#quanto). int8 quantization offers memory improvements up to 75 percent (if all weights are quantized). However it is no free lunch, since 8-bit is not a CUDA-native precision, the weights are quantized back and forth on the fly, which adds up to latency. \n\nFirst, install dependencies.\n\n```bash\npip install -U quanto bitsandbytes\n```\n\nTo quantize a model during loading, we need to first create [`QuantoConfig`]. Then load the model as usual, but pass `quantization_config`\u00a0during model initialization.\n\n```python\nfrom transformers import Idefics2ForConditionalGeneration, AutoTokenizer, QuantoConfig\n\nmodel_id = \"HuggingFaceM4/idefics2-8b\"\nquantization_config = QuantoConfig(weights=\"int8\")\nquantized_model = Idefics2ForConditionalGeneration.from_pretrained(model_id, device_map=\"cuda\", quantization_config=quantization_config)\n```\n\nAnd that's it, we can use the model the same way with no changes. \n\n## Further Reading\n\nHere are some more resources for the image-text-to-text task.\n\n- [Image-text-to-text\u00a0task page](https://huggingface.co/tasks/image-text-to-text) covers model types, use cases, datasets, and more. \n- [Vision Language Models Explained](https://huggingface.co/blog/vlms) is a blog post that covers everything about vision language models and supervised fine-tuning using [TRL](https://huggingface.co/docs/trl/en/index)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b3f86279-afb9-45b9-aace-b5f0778cdefb
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"b3f86279-afb9-45b9-aace-b5f0778cdefb\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"6bd61024-ef61-4bfe-9634-923328b97af4\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6a5369e3d9198abbec82957c2ef70bb255363843ea67c8b218315278721eb334\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2401, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "# Object detection\n\n[[open-in-colab]]\n\nObject detection is the computer vision task of detecting instances (such as humans, buildings, or cars) in an image. Object detection models receive an image as input and output\ncoordinates of the bounding boxes and associated labels of the detected objects. An image can contain multiple objects,\neach with its own bounding box and a label (e.g. it can have a car and a building), and each object can\nbe present in different parts of an image (e.g. the image can have several cars).\nThis task is commonly used in autonomous driving for detecting things like pedestrians, road signs, and traffic lights.\nOther applications include counting objects in images, image search, and more.\n\nIn this guide, you will learn how to:\n\n 1. Finetune [DETR](https://huggingface.co/docs/transformers/model_doc/detr), a model that combines a convolutional\n backbone with an encoder-decoder Transformer, on the [CPPE-5](https://huggingface.co/datasets/cppe-5)\n dataset.\n 2. Use your finetuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/object-detection)\n\n</Tip>\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install -q datasets transformers accelerate timm\npip install -q -U albumentations>=1.4.5 torchmetrics pycocotools\n```\n\nYou'll use \ud83e\udd17 Datasets to load a dataset from the Hugging Face Hub, \ud83e\udd17 Transformers to train your model,\nand `albumentations` to augment the data.\n\nWe encourage you to share your model with the community. Log in to your Hugging Face account to upload it to the Hub.\nWhen prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\nTo get started, we'll define global constants, namely the model name and image size. For this tutorial, we'll use the conditional DETR model due to its faster convergence. Feel free to select any object detection model available in the `transformers` library.\n\n```py\n>>> MODEL_NAME = \"microsoft/conditional-detr-resnet-50\"  # or \"facebook/detr-resnet-50\"\n>>> IMAGE_SIZE = 480\n```\n\n## Load the CPPE-5 dataset\n\nThe [CPPE-5 dataset](https://huggingface.co/datasets/cppe-5) contains images with\nannotations identifying medical personal protective equipment (PPE) in the context of the COVID-19 pandemic."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 6bd61024-ef61-4bfe-9634-923328b97af4
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"6bd61024-ef61-4bfe-9634-923328b97af4\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b3f86279-afb9-45b9-aace-b5f0778cdefb\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d6f38ba8fbdfced4ca39ad1b5c30477f6e4672eeadb518e179c25fa6edc3d9b1\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"322cae12-0bfe-49d5-a55b-56d22a7f9832\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e42d4eb927780ad0d43cef68cecf1a16d035757554871ad9b7283b7aef6b443e\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 993, \"end_char_idx\": 3318, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "2. Use your finetuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/object-detection)\n\n</Tip>\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install -q datasets transformers accelerate timm\npip install -q -U albumentations>=1.4.5 torchmetrics pycocotools\n```\n\nYou'll use \ud83e\udd17 Datasets to load a dataset from the Hugging Face Hub, \ud83e\udd17 Transformers to train your model,\nand `albumentations` to augment the data.\n\nWe encourage you to share your model with the community. Log in to your Hugging Face account to upload it to the Hub.\nWhen prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\nTo get started, we'll define global constants, namely the model name and image size. For this tutorial, we'll use the conditional DETR model due to its faster convergence. Feel free to select any object detection model available in the `transformers` library.\n\n```py\n>>> MODEL_NAME = \"microsoft/conditional-detr-resnet-50\"  # or \"facebook/detr-resnet-50\"\n>>> IMAGE_SIZE = 480\n```\n\n## Load the CPPE-5 dataset\n\nThe [CPPE-5 dataset](https://huggingface.co/datasets/cppe-5) contains images with\nannotations identifying medical personal protective equipment (PPE) in the context of the COVID-19 pandemic.\n\nStart by loading the dataset and creating a `validation` split from `train`:\n\n```py\n>>> from datasets import load_dataset\n\n>>> cppe5 = load_dataset(\"cppe-5\")\n\n>>> if \"validation\" not in cppe5:\n...     split = cppe5[\"train\"].train_test_split(0.15, seed=1337)\n...     cppe5[\"train\"] = split[\"train\"]\n...     cppe5[\"validation\"] = split[\"test\"]\n\n>>> cppe5\nDatasetDict({\n    train: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 850\n    })\n    test: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 29\n    })\n    validation: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 150\n    })\n})\n```\n\nYou'll see that this dataset has 1000 images for train and validation sets and a test set with 29 images.\n\nTo get familiar with the data, explore what the examples look like."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 322cae12-0bfe-49d5-a55b-56d22a7f9832
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"322cae12-0bfe-49d5-a55b-56d22a7f9832\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"6bd61024-ef61-4bfe-9634-923328b97af4\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"fb5e992bfefa7a687936e33d77a88565fb9b8a01c9ba28f2c330dc418ec6161b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e63bddf5-d0c5-473e-bf3e-482f93871b7b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d0a64fc7743877eee49195caa32541c90469e2b61e9763f75539e033815242cf\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1887, \"end_char_idx\": 4475, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "For this tutorial, we'll use the conditional DETR model due to its faster convergence. Feel free to select any object detection model available in the `transformers` library.\n\n```py\n>>> MODEL_NAME = \"microsoft/conditional-detr-resnet-50\"  # or \"facebook/detr-resnet-50\"\n>>> IMAGE_SIZE = 480\n```\n\n## Load the CPPE-5 dataset\n\nThe [CPPE-5 dataset](https://huggingface.co/datasets/cppe-5) contains images with\nannotations identifying medical personal protective equipment (PPE) in the context of the COVID-19 pandemic.\n\nStart by loading the dataset and creating a `validation` split from `train`:\n\n```py\n>>> from datasets import load_dataset\n\n>>> cppe5 = load_dataset(\"cppe-5\")\n\n>>> if \"validation\" not in cppe5:\n...     split = cppe5[\"train\"].train_test_split(0.15, seed=1337)\n...     cppe5[\"train\"] = split[\"train\"]\n...     cppe5[\"validation\"] = split[\"test\"]\n\n>>> cppe5\nDatasetDict({\n    train: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 850\n    })\n    test: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 29\n    })\n    validation: Dataset({\n        features: ['image_id', 'image', 'width', 'height', 'objects'],\n        num_rows: 150\n    })\n})\n```\n\nYou'll see that this dataset has 1000 images for train and validation sets and a test set with 29 images.\n\nTo get familiar with the data, explore what the examples look like.\n\n```py\n>>> cppe5[\"train\"][0]\n{\n  'image_id': 366,\n  'image': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=500x290>,\n  'width': 500,\n  'height': 500,\n  'objects': {\n    'id': [1932, 1933, 1934],\n    'area': [27063, 34200, 32431],\n    'bbox': [[29.0, 11.0, 97.0, 279.0],\n      [201.0, 1.0, 120.0, 285.0],\n      [382.0, 0.0, 113.0, 287.0]],\n    'category': [0, 0, 0]\n  }\n}\n```\n\nThe examples in the dataset have the following fields:\n- `image_id`: the example image id\n- `image`: a `PIL.Image.Image` object containing the image\n- `width`: width of the image\n- `height`: height of the image\n- `objects`: a dictionary containing bounding box metadata for the objects in the image:\n  - `id`: the annotation id\n  - `area`: the area of the bounding box\n  - `bbox`: the object's bounding box (in the [COCO format](https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/#coco) )\n  - `category`: the object's category, with possible values including `Coverall (0)`, `Face_Shield (1)`, `Gloves (2)`, `Goggles (3)` and `Mask (4)`\n\nYou may notice that the `bbox` field follows the COCO format, which is the format that the DETR model expects."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: e63bddf5-d0c5-473e-bf3e-482f93871b7b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"e63bddf5-d0c5-473e-bf3e-482f93871b7b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"322cae12-0bfe-49d5-a55b-56d22a7f9832\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"4d4a5b58470ff6062871fbb8018f2a047431d7f0c606b47279d78be270b5ee10\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3eeecc86-7735-4faf-8a92-d38e796a264c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"76dcc5b53250fae83d0ef8ef2392f35fbdc19b1934b867946bb7c4b697025fe5\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3320, \"end_char_idx\": 4760, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "```py\n>>> cppe5[\"train\"][0]\n{\n  'image_id': 366,\n  'image': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=500x290>,\n  'width': 500,\n  'height': 500,\n  'objects': {\n    'id': [1932, 1933, 1934],\n    'area': [27063, 34200, 32431],\n    'bbox': [[29.0, 11.0, 97.0, 279.0],\n      [201.0, 1.0, 120.0, 285.0],\n      [382.0, 0.0, 113.0, 287.0]],\n    'category': [0, 0, 0]\n  }\n}\n```\n\nThe examples in the dataset have the following fields:\n- `image_id`: the example image id\n- `image`: a `PIL.Image.Image` object containing the image\n- `width`: width of the image\n- `height`: height of the image\n- `objects`: a dictionary containing bounding box metadata for the objects in the image:\n  - `id`: the annotation id\n  - `area`: the area of the bounding box\n  - `bbox`: the object's bounding box (in the [COCO format](https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/#coco) )\n  - `category`: the object's category, with possible values including `Coverall (0)`, `Face_Shield (1)`, `Gloves (2)`, `Goggles (3)` and `Mask (4)`\n\nYou may notice that the `bbox` field follows the COCO format, which is the format that the DETR model expects.\nHowever, the grouping of the fields inside `objects` differs from the annotation format DETR requires. You will\nneed to apply some preprocessing transformations before using this data for training.\n\nTo get an even better understanding of the data, visualize an example in the dataset."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 3eeecc86-7735-4faf-8a92-d38e796a264c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"3eeecc86-7735-4faf-8a92-d38e796a264c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e63bddf5-d0c5-473e-bf3e-482f93871b7b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"f1143e19fc5aefe896eeb089aa35651e4823c2f7beabacf48633fa7509579910\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"39656372-3ea5-4ee0-99cb-284aa6a2ca29\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7c29284067bf392a3b88dae6f2f99a9958154e4b108ac43616af81f2d9b5b4ac\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4476, \"end_char_idx\": 7433, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "However, the grouping of the fields inside `objects` differs from the annotation format DETR requires. You will\nneed to apply some preprocessing transformations before using this data for training.\n\nTo get an even better understanding of the data, visualize an example in the dataset.\n\n```py\n>>> import numpy as np\n>>> import os\n>>> from PIL import Image, ImageDraw\n\n>>> image = cppe5[\"train\"][2][\"image\"]\n>>> annotations = cppe5[\"train\"][2][\"objects\"]\n>>> draw = ImageDraw.Draw(image)\n\n>>> categories = cppe5[\"train\"].features[\"objects\"].feature[\"category\"].names\n\n>>> id2label = {index: x for index, x in enumerate(categories, start=0)}\n>>> label2id = {v: k for k, v in id2label.items()}\n\n>>> for i in range(len(annotations[\"id\"])):\n...     box = annotations[\"bbox\"][i]\n...     class_idx = annotations[\"category\"][i]\n...     x, y, w, h = tuple(box)\n...     # Check if coordinates are normalized or not\n...     if max(box) > 1.0:\n...         # Coordinates are un-normalized, no need to re-scale them\n...         x1, y1 = int(x), int(y)\n...         x2, y2 = int(x + w), int(y + h)\n...     else:\n...         # Coordinates are normalized, re-scale them\n...         x1 = int(x * width)\n...         y1 = int(y * height)\n...         x2 = int((x + w) * width)\n...         y2 = int((y + h) * height)\n...     draw.rectangle((x, y, x + w, y + h), outline=\"red\", width=1)\n...     draw.text((x, y), id2label[class_idx], fill=\"white\")\n\n>>> image\n```\n<div class=\"flex justify-center\">\n    <img src=\"https://i.imgur.com/oVQb9SF.png\" alt=\"CPPE-5 Image Example\"/>\n</div>\n\n\nTo visualize the bounding boxes with associated labels, you can get the labels from the dataset's metadata, specifically\nthe `category` field.\nYou'll also want to create dictionaries that map a label id to a label class (`id2label`) and the other way around (`label2id`).\nYou can use them later when setting up the model. Including these maps will make your model reusable by others if you share\nit on the Hugging Face Hub. Please note that, the part of above code that draws the bounding boxes assume that it is in `COCO` format `(x_min, y_min, width, height)`. It has to be adjusted to work for other formats like `(x_min, y_min, x_max, y_max)`.\n\nAs a final step of getting familiar with the data, explore it for potential issues. One common problem with datasets for\nobject detection is bounding boxes that \"stretch\" beyond the edge of the image. Such \"runaway\" bounding boxes can raise\nerrors during training and should be addressed. There are a few examples with this issue in this dataset.\nTo keep things simple in this guide, we will set `clip=True` for `BboxParams` in transformations below.\n\n## Preprocess the data\n\nTo finetune a model, you must preprocess the data you plan to use to match precisely the approach used for the pre-trained model.\n[`AutoImageProcessor`] takes care of processing image data to create `pixel_values`, `pixel_mask`, and\n`labels` that a DETR model can train with."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 39656372-3ea5-4ee0-99cb-284aa6a2ca29
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"39656372-3ea5-4ee0-99cb-284aa6a2ca29\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3eeecc86-7735-4faf-8a92-d38e796a264c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"081c3c7d56afcca8c583c9c0d912b4f336db28f1e7091f448695bf4369200b45\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"2a2e2edf-8ba6-4639-b366-80bf16bbd7b0\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c35e953b9512e753411c4004b1f2b62e16c042dfb529d7f77cc8dea243add8fc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6033, \"end_char_idx\": 9114, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "To visualize the bounding boxes with associated labels, you can get the labels from the dataset's metadata, specifically\nthe `category` field.\nYou'll also want to create dictionaries that map a label id to a label class (`id2label`) and the other way around (`label2id`).\nYou can use them later when setting up the model. Including these maps will make your model reusable by others if you share\nit on the Hugging Face Hub. Please note that, the part of above code that draws the bounding boxes assume that it is in `COCO` format `(x_min, y_min, width, height)`. It has to be adjusted to work for other formats like `(x_min, y_min, x_max, y_max)`.\n\nAs a final step of getting familiar with the data, explore it for potential issues. One common problem with datasets for\nobject detection is bounding boxes that \"stretch\" beyond the edge of the image. Such \"runaway\" bounding boxes can raise\nerrors during training and should be addressed. There are a few examples with this issue in this dataset.\nTo keep things simple in this guide, we will set `clip=True` for `BboxParams` in transformations below.\n\n## Preprocess the data\n\nTo finetune a model, you must preprocess the data you plan to use to match precisely the approach used for the pre-trained model.\n[`AutoImageProcessor`] takes care of processing image data to create `pixel_values`, `pixel_mask`, and\n`labels` that a DETR model can train with. The image processor has some attributes that you won't have to worry about:\n\n- `image_mean = [0.485, 0.456, 0.406 ]`\n- `image_std = [0.229, 0.224, 0.225]`\n\nThese are the mean and standard deviation used to normalize images during the model pre-training. These values are crucial\nto replicate when doing inference or finetuning a pre-trained image model.\n\nInstantiate the image processor from the same checkpoint as the model you want to finetune.\n\n```py\n>>> from transformers import AutoImageProcessor\n\n>>> MAX_SIZE = IMAGE_SIZE\n\n>>> image_processor = AutoImageProcessor.from_pretrained(\n...     MODEL_NAME,\n...     do_resize=True,\n...     size={\"max_height\": MAX_SIZE, \"max_width\": MAX_SIZE},\n...     do_pad=True,\n...     pad_size={\"height\": MAX_SIZE, \"width\": MAX_SIZE},\n... )\n```\n\nBefore passing the images to the `image_processor`, apply two preprocessing transformations to the dataset:\n- Augmenting images\n- Reformatting annotations to meet DETR expectations\n\nFirst, to make sure the model does not overfit on the training data, you can apply image augmentation with any data augmentation library. Here we use [Albumentations](https://albumentations.ai/docs/).\nThis library ensures that transformations affect the image and update the bounding boxes accordingly.\nThe \ud83e\udd17 Datasets library documentation has a detailed [guide on how to augment images for object detection](https://huggingface.co/docs/datasets/object_detection),\nand it uses the exact same dataset as an example. Apply some geometric and color transformations to the image. For additional augmentation options, explore the [Albumentations Demo Space](https://huggingface.co/spaces/qubvel-hf/albumentations-demo)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 2a2e2edf-8ba6-4639-b366-80bf16bbd7b0
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"2a2e2edf-8ba6-4639-b366-80bf16bbd7b0\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"39656372-3ea5-4ee0-99cb-284aa6a2ca29\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"549ff75ee709b05712c0812ff314a076a1619e8dc17cd01a0e13ed5ac7146eae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a39a56aa-00ef-46ce-b30b-855a2008d743\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e3d3408c28ef45c5a0f556e3d1aa06cacde680bf44f90dd18b57b2524787728a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7434, \"end_char_idx\": 10095, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "The image processor has some attributes that you won't have to worry about:\n\n- `image_mean = [0.485, 0.456, 0.406 ]`\n- `image_std = [0.229, 0.224, 0.225]`\n\nThese are the mean and standard deviation used to normalize images during the model pre-training. These values are crucial\nto replicate when doing inference or finetuning a pre-trained image model.\n\nInstantiate the image processor from the same checkpoint as the model you want to finetune.\n\n```py\n>>> from transformers import AutoImageProcessor\n\n>>> MAX_SIZE = IMAGE_SIZE\n\n>>> image_processor = AutoImageProcessor.from_pretrained(\n...     MODEL_NAME,\n...     do_resize=True,\n...     size={\"max_height\": MAX_SIZE, \"max_width\": MAX_SIZE},\n...     do_pad=True,\n...     pad_size={\"height\": MAX_SIZE, \"width\": MAX_SIZE},\n... )\n```\n\nBefore passing the images to the `image_processor`, apply two preprocessing transformations to the dataset:\n- Augmenting images\n- Reformatting annotations to meet DETR expectations\n\nFirst, to make sure the model does not overfit on the training data, you can apply image augmentation with any data augmentation library. Here we use [Albumentations](https://albumentations.ai/docs/).\nThis library ensures that transformations affect the image and update the bounding boxes accordingly.\nThe \ud83e\udd17 Datasets library documentation has a detailed [guide on how to augment images for object detection](https://huggingface.co/docs/datasets/object_detection),\nand it uses the exact same dataset as an example. Apply some geometric and color transformations to the image. For additional augmentation options, explore the [Albumentations Demo Space](https://huggingface.co/spaces/qubvel-hf/albumentations-demo).\n\n```py\n>>> import albumentations as A\n\n>>> train_augment_and_transform = A.Compose(\n...     [\n...         A.Perspective(p=0.1),\n...         A.HorizontalFlip(p=0.5),\n...         A.RandomBrightnessContrast(p=0.5),\n...         A.HueSaturationValue(p=0.1),\n...     ],\n...     bbox_params=A.BboxParams(format=\"coco\", label_fields=[\"category\"], clip=True, min_area=25),\n... )\n\n>>> validation_transform = A.Compose(\n...     [A.NoOp()],\n...     bbox_params=A.BboxParams(format=\"coco\", label_fields=[\"category\"], clip=True),\n... )\n```\n\nThe `image_processor` expects the annotations to be in the following format: `{'image_id': int, 'annotations': List[Dict]}`,\n where each dictionary is a COCO object annotation. Let's add a function to reformat annotations for a single example:\n\n```py\n>>> def format_image_annotations_as_coco(image_id, categories, areas, bboxes):\n...     \"\"\"Format one set of image annotations to the COCO format\n\n...     Args:\n...         image_id (str): image id. e.g."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a39a56aa-00ef-46ce-b30b-855a2008d743
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"a39a56aa-00ef-46ce-b30b-855a2008d743\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"2a2e2edf-8ba6-4639-b366-80bf16bbd7b0\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"eb1f571d66366d801c6728c9cc06ff1d48326836ad03c9c2b6b16b9ad1c94739\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"131f9b57-21c9-425f-98b0-fd32dc011033\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b84914bda320b3f08e5e432afb8242660428b333d68de5ea81aa07cef3ea7ced\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8538, \"end_char_idx\": 12459, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "Here we use [Albumentations](https://albumentations.ai/docs/).\nThis library ensures that transformations affect the image and update the bounding boxes accordingly.\nThe \ud83e\udd17 Datasets library documentation has a detailed [guide on how to augment images for object detection](https://huggingface.co/docs/datasets/object_detection),\nand it uses the exact same dataset as an example. Apply some geometric and color transformations to the image. For additional augmentation options, explore the [Albumentations Demo Space](https://huggingface.co/spaces/qubvel-hf/albumentations-demo).\n\n```py\n>>> import albumentations as A\n\n>>> train_augment_and_transform = A.Compose(\n...     [\n...         A.Perspective(p=0.1),\n...         A.HorizontalFlip(p=0.5),\n...         A.RandomBrightnessContrast(p=0.5),\n...         A.HueSaturationValue(p=0.1),\n...     ],\n...     bbox_params=A.BboxParams(format=\"coco\", label_fields=[\"category\"], clip=True, min_area=25),\n... )\n\n>>> validation_transform = A.Compose(\n...     [A.NoOp()],\n...     bbox_params=A.BboxParams(format=\"coco\", label_fields=[\"category\"], clip=True),\n... )\n```\n\nThe `image_processor` expects the annotations to be in the following format: `{'image_id': int, 'annotations': List[Dict]}`,\n where each dictionary is a COCO object annotation. Let's add a function to reformat annotations for a single example:\n\n```py\n>>> def format_image_annotations_as_coco(image_id, categories, areas, bboxes):\n...     \"\"\"Format one set of image annotations to the COCO format\n\n...     Args:\n...         image_id (str): image id. e.g. \"0001\"\n...         categories (List[int]): list of categories/class labels corresponding to provided bounding boxes\n...         areas (List[float]): list of corresponding areas to provided bounding boxes\n...         bboxes (List[Tuple[float]]): list of bounding boxes provided in COCO format\n...             ([center_x, center_y, width, height] in absolute coordinates)\n\n...     Returns:\n...         dict: {\n...             \"image_id\": image id,\n...             \"annotations\": list of formatted annotations\n...         }\n...     \"\"\"\n...     annotations = []\n...     for category, area, bbox in zip(categories, areas, bboxes):\n...         formatted_annotation = {\n...             \"image_id\": image_id,\n...             \"category_id\": category,\n...             \"iscrowd\": 0,\n...             \"area\": area,\n...             \"bbox\": list(bbox),\n...         }\n...         annotations.append(formatted_annotation)\n\n...     return {\n...         \"image_id\": image_id,\n...         \"annotations\": annotations,\n...     }\n\n```\n\nNow you can combine the image and annotation transformations to use on a batch of examples:\n\n```py\n>>> def augment_and_transform_batch(examples, transform, image_processor, return_pixel_mask=False):\n...     \"\"\"Apply augmentations and format annotations in COCO format for object detection task\"\"\"\n\n...     images = []\n...     annotations = []\n...     for image_id, image, objects in zip(examples[\"image_id\"], examples[\"image\"], examples[\"objects\"]):\n...         image = np.array(image.convert(\"RGB\"))\n\n...         # apply augmentations\n...         output = transform(image=image, bboxes=objects[\"bbox\"], category=objects[\"category\"])\n...         images.append(output[\"image\"])\n\n...         # format annotations in COCO format\n...         formatted_annotations = format_image_annotations_as_coco(\n...             image_id, output[\"category\"], objects[\"area\"], output[\"bboxes\"]\n...         )\n...         annotations.append(formatted_annotations)\n\n...     # Apply the image processor transformations: resizing, rescaling, normalization\n...     result = image_processor(images=images, annotations=annotations, return_tensors=\"pt\")\n\n...     if not return_pixel_mask:\n...         result.pop(\"pixel_mask\", None)\n\n...     return result\n```\n\nApply this preprocessing function to the entire dataset using \ud83e\udd17 Datasets [`~datasets.Dataset.with_transform`] method."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 131f9b57-21c9-425f-98b0-fd32dc011033
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"131f9b57-21c9-425f-98b0-fd32dc011033\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a39a56aa-00ef-46ce-b30b-855a2008d743\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"fe71bd0699a05148e04169c9045199cf2337e343f4d0aa148ecb27078aa3dc53\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"6d9bb842-005a-40db-ba41-8b8a8855e43c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"16362efe4a5f5d6d55d670328bfffcfbc9133716ef95184389c0dc299ad84d30\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "This method applies\ntransformations on the fly when you load an element of the dataset.\n\nAt this point, you can check what an example from the dataset looks like after the transformations. You should see a tensor\nwith `pixel_values`, a tensor with `pixel_mask`, and `labels`.\n\n```py\n>>> from functools import partial\n\n>>> # Make transform functions for batch and apply for dataset splits\n>>> train_transform_batch = partial(\n.     augment_and_transform_batch, transform=train_augment_and_transform, image_processor=image_processor\n. )\n>>> validation_transform_batch = partial(\n.     augment_and_transform_batch, transform=validation_transform, image_processor=image_processor\n. )\n\n>>> cppe5[\"train\"] = cppe5[\"train\"].with_transform(train_transform_batch)\n>>> cppe5[\"validation\"] = cppe5[\"validation\"].with_transform(validation_transform_batch)\n>>> cppe5[\"test\"] = cppe5[\"test\"].with_transform(validation_transform_batch)\n\n>>> cppe5[\"train\"][15]\n{'pixel_values': tensor([[[ 1.9235,  1.9407,  1.9749,  . -0.7822, -0.7479, -0.6965],\n          [ 1.9578,  1.9749,  1.9920,  . -0.7993, -0.7650, -0.7308],\n          [ 2.0092,  2.0092,  2.0263,  . -0.8507, -0.8164, -0.7822],\n          .\n          [ 0.0741,  0.0741,  0.0741,  .  0.0741,  0.0741,  0.0741],\n          [ 0.0741,  0.0741,  0.0741,  .  0.0741,  0.0741,  0.0741],\n          [ 0.0741,  0.0741,  0.0741,  .  0.0741,  0.0741,  0.0741]],\n  \n          [[ 1.6232,  1.6408,  1.6583,  .  0.8704,  1.0105,  1.1331],\n          [ 1.6408,  1.6583,  1.6758,  .  0.8529,  0.9930,  1.0980],\n          [ 1.6933,  1.6933,  1.7108,  .  0.8179,  0.9580,  1.0630],\n          .\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052],\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052],\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052]],\n  \n          [[ 1.8905,  1.9080,  1.9428,  ."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 6d9bb842-005a-40db-ba41-8b8a8855e43c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"6d9bb842-005a-40db-ba41-8b8a8855e43c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"131f9b57-21c9-425f-98b0-fd32dc011033\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"8ecec3def1f295328ef58b900262969b60b75961d859f60bfba51f7f7367b3ae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"97945ae9-9a4d-48ac-8744-ab159077c91c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"18d08ee815e7860e2de3464b851dacbeb4a07d76cc200bc0f4f0ffa7f265a47e\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "0741],\n          [ 0.0741,  0.0741,  0.0741,  .  0.0741,  0.0741,  0.0741],\n          [ 0.0741,  0.0741,  0.0741,  .  0.0741,  0.0741,  0.0741]],\n  \n          [[ 1.6232,  1.6408,  1.6583,  .  0.8704,  1.0105,  1.1331],\n          [ 1.6408,  1.6583,  1.6758,  .  0.8529,  0.9930,  1.0980],\n          [ 1.6933,  1.6933,  1.7108,  .  0.8179,  0.9580,  1.0630],\n          .\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052],\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052],\n          [ 0.2052,  0.2052,  0.2052,  .  0.2052,  0.2052,  0.2052]],\n  \n          [[ 1.8905,  1.9080,  1.9428,  . -0.1487, -0.0964, -0.0615],\n          [ 1.9254,  1.9428,  1.9603,  . -0.1661, -0.1138, -0.0790],\n          [ 1.9777,  1.9777,  1.9951,  . -0.2010, -0.1138, -0.0790],\n          .\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265],\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265],\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265]]]),\n  'labels': {'image_id': tensor([688]), 'class_labels': tensor([3, 4, 2, 0, 0]), 'boxes': tensor([[0.4700, 0.1933, 0.1467, 0.0767],\n          [0.4858, 0.2600, 0.1150, 0.1000],\n          [0.4042, 0.4517, 0.1217, 0.1300],\n          [0.4242, 0.3217, 0.3617, 0.5567],"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 97945ae9-9a4d-48ac-8744-ab159077c91c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"97945ae9-9a4d-48ac-8744-ab159077c91c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"6d9bb842-005a-40db-ba41-8b8a8855e43c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"465d8093f52c6ddf2c020f65f2241f59bc65da76bf92d35b2dded3765073efc9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"061d199f-8baf-4c5b-9015-23e4d4d6685a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e58d179bc248edbe2e04351c151c238e22920aa8707ea729758619aa449148a8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "9080,  1.9428,  . -0.1487, -0.0964, -0.0615],\n          [ 1.9254,  1.9428,  1.9603,  . -0.1661, -0.1138, -0.0790],\n          [ 1.9777,  1.9777,  1.9951,  . -0.2010, -0.1138, -0.0790],\n          .\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265],\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265],\n          [ 0.4265,  0.4265,  0.4265,  .  0.4265,  0.4265,  0.4265]]]),\n  'labels': {'image_id': tensor([688]), 'class_labels': tensor([3, 4, 2, 0, 0]), 'boxes': tensor([[0.4700, 0.1933, 0.1467, 0.0767],\n          [0.4858, 0.2600, 0.1150, 0.1000],\n          [0.4042, 0.4517, 0.1217, 0.1300],\n          [0.4242, 0.3217, 0.3617, 0.5567],\n          [0.6617, 0.4033, 0.5400, 0.4533]]), 'area': tensor([ 4048.  4140.  5694. 72478. 88128.]), 'iscrowd': tensor([0, 0, 0, 0, 0]), 'orig_size': tensor([480, 480])}}\n```\n\nYou have successfully augmented the individual images and prepared their annotations. However, preprocessing isn't\ncomplete yet. In the final step, create a custom `collate_fn` to batch images together.\nPad images (which are now `pixel_values`) to the largest image in a batch, and create a corresponding `pixel_mask`\nto indicate which pixels are real (1) and which are padding (0).\n\n```py\n>>> import torch\n\n>>> def collate_fn(batch):\n...     data = {}\n...     data[\"pixel_values\"] = torch.stack([x[\"pixel_values\"] for x in batch])\n...     data[\"labels\"] = [x[\"labels\"] for x in batch]\n...     if \"pixel_mask\" in batch[0]:\n...         data[\"pixel_mask\"] = torch.stack([x[\"pixel_mask\"] for x in batch])\n...     return data\n\n```\n\n## Preparing function to compute mAP\n\nObject detection models are commonly evaluated with a set of <a href=\"https://cocodataset.org/#detection-eval\">COCO-style metrics</a>. We are going to use `torchmetrics` to compute `mAP` (mean average precision) and `mAR` (mean average recall) metrics and will wrap it to `compute_metrics` function in order to use in [`Trainer`] for evaluation."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 061d199f-8baf-4c5b-9015-23e4d4d6685a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"061d199f-8baf-4c5b-9015-23e4d4d6685a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"97945ae9-9a4d-48ac-8744-ab159077c91c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"78542935591fa9e23c7707e33976bab8c19775aae3daa42733bd8b5537a37d08\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f814b9d8-b58d-4929-9e0c-95d3154bd15e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e8654c2503d832dfc38d850656525f4459f74a861f477d5a338907750ae2e01f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "0.1300],\n          [0.4242, 0.3217, 0.3617, 0.5567],\n          [0.6617, 0.4033, 0.5400, 0.4533]]), 'area': tensor([ 4048.  4140.  5694. 72478. 88128.]), 'iscrowd': tensor([0, 0, 0, 0, 0]), 'orig_size': tensor([480, 480])}}\n```\n\nYou have successfully augmented the individual images and prepared their annotations. However, preprocessing isn't\ncomplete yet. In the final step, create a custom `collate_fn` to batch images together.\nPad images (which are now `pixel_values`) to the largest image in a batch, and create a corresponding `pixel_mask`\nto indicate which pixels are real (1) and which are padding (0).\n\n```py\n>>> import torch\n\n>>> def collate_fn(batch):\n...     data = {}\n...     data[\"pixel_values\"] = torch.stack([x[\"pixel_values\"] for x in batch])\n...     data[\"labels\"] = [x[\"labels\"] for x in batch]\n...     if \"pixel_mask\" in batch[0]:\n...         data[\"pixel_mask\"] = torch.stack([x[\"pixel_mask\"] for x in batch])\n...     return data\n\n```\n\n## Preparing function to compute mAP\n\nObject detection models are commonly evaluated with a set of <a href=\"https://cocodataset.org/#detection-eval\">COCO-style metrics</a>. We are going to use `torchmetrics` to compute `mAP` (mean average precision) and `mAR` (mean average recall) metrics and will wrap it to `compute_metrics` function in order to use in [`Trainer`] for evaluation.\n\nIntermediate format of boxes used for training is `YOLO` (normalized) but we will compute metrics for boxes in `Pascal VOC` (absolute) format in order to correctly handle box areas. Let's define a function that converts bounding boxes to `Pascal VOC` format:\n\n```py\n>>> from transformers.image_transforms import center_to_corners_format\n\n>>> def convert_bbox_yolo_to_pascal(boxes, image_size):\n...     \"\"\"\n...     Convert bounding boxes from YOLO format (x_center, y_center, width, height) in range [0, 1]\n...     to Pascal VOC format (x_min, y_min, x_max, y_max) in absolute coordinates.\n\n...     Args:\n...         boxes (torch.Tensor): Bounding boxes in YOLO format\n...         image_size (Tuple[int, int]): Image size in format (height, width)\n\n...     Returns:\n...         torch.Tensor: Bounding boxes in Pascal VOC format (x_min, y_min, x_max, y_max)\n...     \"\"\"\n...     # convert center to corners format\n...     boxes = center_to_corners_format(boxes)\n\n...     # convert to absolute coordinates\n...     height, width = image_size\n...     boxes = boxes * torch.tensor([[width, height, width, height]])\n\n...     return boxes\n```\n\nThen, in `compute_metrics` function we collect `predicted` and `target` bounding boxes, scores and labels from evaluation loop results and pass it to the scoring function.\n\n```py\n>>> import numpy as np\n>>> from dataclasses import dataclass\n>>> from torchmetrics.detection.mean_ap import MeanAveragePrecision\n\n\n>>> @dataclass\n>>> class ModelOutput:\n...     logits: torch.Tensor\n...     pred_boxes: torch.Tensor"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: f814b9d8-b58d-4929-9e0c-95d3154bd15e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"f814b9d8-b58d-4929-9e0c-95d3154bd15e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"061d199f-8baf-4c5b-9015-23e4d4d6685a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"a9df26324f1551da173a70883d3e2bb812122accfe25835fb4317135a8f3fd97\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"85e20a81-9509-405c-b415-5c28a0245a33\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1ffb0299a3d99bff0a66034bb4407641898738e0867c0aa7315f300ca0ea0d4b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 16339, \"end_char_idx\": 21366, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "Intermediate format of boxes used for training is `YOLO` (normalized) but we will compute metrics for boxes in `Pascal VOC` (absolute) format in order to correctly handle box areas. Let's define a function that converts bounding boxes to `Pascal VOC` format:\n\n```py\n>>> from transformers.image_transforms import center_to_corners_format\n\n>>> def convert_bbox_yolo_to_pascal(boxes, image_size):\n...     \"\"\"\n...     Convert bounding boxes from YOLO format (x_center, y_center, width, height) in range [0, 1]\n...     to Pascal VOC format (x_min, y_min, x_max, y_max) in absolute coordinates.\n\n...     Args:\n...         boxes (torch.Tensor): Bounding boxes in YOLO format\n...         image_size (Tuple[int, int]): Image size in format (height, width)\n\n...     Returns:\n...         torch.Tensor: Bounding boxes in Pascal VOC format (x_min, y_min, x_max, y_max)\n...     \"\"\"\n...     # convert center to corners format\n...     boxes = center_to_corners_format(boxes)\n\n...     # convert to absolute coordinates\n...     height, width = image_size\n...     boxes = boxes * torch.tensor([[width, height, width, height]])\n\n...     return boxes\n```\n\nThen, in `compute_metrics` function we collect `predicted` and `target` bounding boxes, scores and labels from evaluation loop results and pass it to the scoring function.\n\n```py\n>>> import numpy as np\n>>> from dataclasses import dataclass\n>>> from torchmetrics.detection.mean_ap import MeanAveragePrecision\n\n\n>>> @dataclass\n>>> class ModelOutput:\n...     logits: torch.Tensor\n...     pred_boxes: torch.Tensor\n\n\n>>> @torch.no_grad()\n>>> def compute_metrics(evaluation_results, image_processor, threshold=0.0, id2label=None):\n...     \"\"\"\n...     Compute mean average mAP, mAR and their variants for the object detection task.\n\n...     Args:\n...         evaluation_results (EvalPrediction): Predictions and targets from evaluation.\n...         threshold (float, optional): Threshold to filter predicted boxes by confidence. Defaults to 0.0.\n...         id2label (Optional[dict], optional): Mapping from class id to class name. Defaults to None.\n\n...     Returns:\n...         Mapping[str, float]: Metrics in a form of dictionary {<metric_name>: <metric_value>}\n...     \"\"\"\n\n...     predictions, targets = evaluation_results.predictions, evaluation_results.label_ids\n\n...     # For metric computation we need to provide:\n...     #  - targets in a form of list of dictionaries with keys \"boxes\", \"labels\"\n...     #  - predictions in a form of list of dictionaries with keys \"boxes\", \"scores\", \"labels\"\n\n...     image_sizes = []\n...     post_processed_targets = []\n...     post_processed_predictions = []\n\n...     # Collect targets in the required format for metric computation\n...     for batch in targets:\n...         # collect image sizes, we will need them for predictions post processing\n...         batch_image_sizes = torch.tensor(np.array([x[\"orig_size\"] for x in batch]))\n...         image_sizes.append(batch_image_sizes)\n...         # collect targets in the required format for metric computation\n...         # boxes were converted to YOLO format needed for model training\n...         # here we will convert them to Pascal VOC format (x_min, y_min, x_max, y_max)\n...         for image_target in batch:\n...             boxes = torch.tensor(image_target[\"boxes\"])\n...             boxes = convert_bbox_yolo_to_pascal(boxes, image_target[\"orig_size\"])\n...             labels = torch.tensor(image_target[\"class_labels\"])\n...             post_processed_targets.append({\"boxes\": boxes, \"labels\": labels})\n\n...     # Collect predictions in the required format for metric computation,\n...     # model produce boxes in YOLO format, then image_processor convert them to Pascal VOC format\n...     for batch, target_sizes in zip(predictions, image_sizes):\n...         batch_logits, batch_boxes = batch[1], batch[2]\n...         output = ModelOutput(logits=torch.tensor(batch_logits), pred_boxes=torch.tensor(batch_boxes))\n...         post_processed_output = image_processor.post_process_object_detection(\n...             output, threshold=threshold, target_sizes=target_sizes\n...         )\n...         post_processed_predictions.extend(post_processed_output)\n\n...     # Compute metrics\n...     metric = MeanAveragePrecision(box_format=\"xyxy\", class_metrics=True)\n...     metric.update(post_processed_predictions, post_processed_targets)\n...     metrics = metric.compute()\n\n...     # Replace list of per class metrics with separate metric for each class\n...     classes = metrics.pop(\"classes\")\n...     map_per_class = metrics.pop(\"map_per_class\")\n...     mar_100_per_class = metrics.pop(\"mar_100_per_class\")\n...     for class_id, class_map, class_mar in zip(classes, map_per_class, mar_100_per_class):\n...         class_name = id2label[class_id.item()] if id2label is not None else class_id.item()\n...         metrics[f\"map_{class_name}\"] = class_map\n...         metrics[f\"mar_100_{class_name}\"] = class_mar\n\n...     metrics = {k: round(v.item(), 4) for k, v in metrics.items()}\n\n...     return metrics"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 85e20a81-9509-405c-b415-5c28a0245a33
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"85e20a81-9509-405c-b415-5c28a0245a33\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f814b9d8-b58d-4929-9e0c-95d3154bd15e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"55c79b23536ee0c2c5e518c276850ba6c3369decb88493b9a435605d883a70f3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e8280077-a9cb-4286-803d-5da03f0b140b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"203b9167d068489e551c4d0a452a1bf3986c1b59b1342a2e5024b962b79e769e\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": ">>> eval_compute_metrics_fn = partial(\n...     compute_metrics, image_processor=image_processor, id2label=id2label, threshold=0.0\n... )\n```\n\n## Training the detection model\n\nYou have done most of the heavy lifting in the previous sections, so now you are ready to train your model!\nThe images in this dataset are still quite large, even after resizing. This means that finetuning this model will\nrequire at least one GPU.\n\nTraining involves the following steps:\n1. Load the model with [`AutoModelForObjectDetection`] using the same checkpoint as in the preprocessing.\n2. Define your training hyperparameters in [`TrainingArguments`].\n3. Pass the training arguments to [`Trainer`] along with the model, dataset, image processor, and data collator.\n4. Call [`~Trainer.train`] to finetune your model.\n\nWhen loading the model from the same checkpoint that you used for the preprocessing, remember to pass the `label2id`\nand `id2label` maps that you created earlier from the dataset's metadata. Additionally, we specify `ignore_mismatched_sizes=True` to replace the existing classification head with a new one.\n\n```py\n>>> from transformers import AutoModelForObjectDetection\n\n>>> model = AutoModelForObjectDetection.from_pretrained(\n...     MODEL_NAME,\n...     id2label=id2label,\n...     label2id=label2id,\n...     ignore_mismatched_sizes=True,\n... )\n```\n\nIn the [`TrainingArguments`] use `output_dir` to specify where to save your model, then configure hyperparameters as you see fit. For `num_train_epochs=30` training will take about 35 minutes in Google Colab T4 GPU, increase the number of epoch to get better results.\n\nImportant notes:\n - Do not remove unused columns because this will drop the image column. Without the image column, you\ncan't create `pixel_values`. For this reason, set `remove_unused_columns` to `False`.\n - Set `eval_do_concat_batches=False` to get proper evaluation results. Images have different number of target boxes, if batches are concatenated we will not be able to determine which boxes belongs to particular image.\n\nIf you wish to share your model by pushing to the Hub, set `push_to_hub` to `True` (you must be signed in to Hugging\nFace to upload your model).\n\n```py\n>>> from transformers import TrainingArguments\n\n>>> training_args = TrainingArguments(\n.     output_dir=\"detr_finetuned_cppe5\",\n.     num_train_epochs=30,\n.     fp16=False,\n.     per_device_train_batch_size=8,\n.     dataloader_num_workers=4,\n.     learning_rate=5e-5,\n.     lr_scheduler_type=\"cosine\",\n.     weight_decay=1e-4,\n.     max_grad_norm=0.01,\n.     metric_for_best_model=\"eval_map\",\n.     greater_is_better=True,\n.     load_best_model_at_end=True,\n.     eval_strategy=\"epoch\",\n.     save_strategy=\"epoch\",\n.     save_total_limit=2,\n.     remove_unused_columns=False,\n.     eval_do_concat_batches=False,\n.     push_to_hub=True,\n. )\n```\n\nFinally, bring everything together, and call [`~transformers.Trainer.train`]:\n\n```py\n>>> from transformers import Trainer\n\n>>> trainer = Trainer(\n.     model=model,\n.     args=training_args,\n.     train_dataset=cppe5[\"train\"],\n.     eval_dataset=cppe5[\"validation\"],\n.     tokenizer=image_processor,\n.     data_collator=collate_fn,\n."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: e8280077-a9cb-4286-803d-5da03f0b140b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"e8280077-a9cb-4286-803d-5da03f0b140b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"85e20a81-9509-405c-b415-5c28a0245a33\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"81831528b8a21a2723008c21d15b4d039a0100f6a1874bd9ac01d5fbc40ff9c5\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d2f51363-de09-4fa9-8ea3-25353e0e2b0c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"373bc6419be3b6ecc65f9036aaa36cd93684e1f2bfa34c2b0a662440b84d8ffc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "Without the image column, you\ncan't create `pixel_values`. For this reason, set `remove_unused_columns` to `False`.\n - Set `eval_do_concat_batches=False` to get proper evaluation results. Images have different number of target boxes, if batches are concatenated we will not be able to determine which boxes belongs to particular image.\n\nIf you wish to share your model by pushing to the Hub, set `push_to_hub` to `True` (you must be signed in to Hugging\nFace to upload your model).\n\n```py\n>>> from transformers import TrainingArguments\n\n>>> training_args = TrainingArguments(\n.     output_dir=\"detr_finetuned_cppe5\",\n.     num_train_epochs=30,\n.     fp16=False,\n.     per_device_train_batch_size=8,\n.     dataloader_num_workers=4,\n.     learning_rate=5e-5,\n.     lr_scheduler_type=\"cosine\",\n.     weight_decay=1e-4,\n.     max_grad_norm=0.01,\n.     metric_for_best_model=\"eval_map\",\n.     greater_is_better=True,\n.     load_best_model_at_end=True,\n.     eval_strategy=\"epoch\",\n.     save_strategy=\"epoch\",\n.     save_total_limit=2,\n.     remove_unused_columns=False,\n.     eval_do_concat_batches=False,\n.     push_to_hub=True,\n. )\n```\n\nFinally, bring everything together, and call [`~transformers.Trainer.train`]:\n\n```py\n>>> from transformers import Trainer\n\n>>> trainer = Trainer(\n.     model=model,\n.     args=training_args,\n.     train_dataset=cppe5[\"train\"],\n.     eval_dataset=cppe5[\"validation\"],\n.     tokenizer=image_processor,\n.     data_collator=collate_fn,\n.     compute_metrics=eval_compute_metrics_fn,\n. )\n\n>>> trainer.train()\n```\n<div>\n\n  <progress value='3210' max='3210' style='width:300px; height:20px; vertical-align: middle;'></progress>\n  [3210/3210 26:07, Epoch 30/30]\n</div>\n\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: left;\">\n      <th>Epoch</th>\n      <th>Training Loss</th>\n      <th>Validation Loss</th>\n      <th>Map</th>\n      <th>Map 50</th>\n      <th>Map 75</th>\n      <th>Map Small</th>\n      <th>Map Medium</th>\n      <th>Map Large</th>\n      <th>Mar 1</th>\n      <th>Mar 10</th>\n      <th>Mar 100</th>\n      <th>Mar Small</th>\n      <th>Mar Medium</th>\n      <th>Mar Large</th>\n      <th>Map Coverall</th>\n      <th>Mar 100 Coverall</th>\n      <th>Map Face Shield</th>\n      <th>Mar 100 Face Shield</th>\n      <th>Map Gloves</th>\n      <th>Mar 100 Gloves</th>\n      <th>Map Goggles</th>\n      <th>Mar 100 Goggles</th>\n      <th>Map Mask</th>\n      <th>Mar 100 Mask</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td>1</td>\n      <td>No log</td>\n      <td>2."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d2f51363-de09-4fa9-8ea3-25353e0e2b0c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d2f51363-de09-4fa9-8ea3-25353e0e2b0c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e8280077-a9cb-4286-803d-5da03f0b140b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"0d629bb88d97aa32173202b6aa162f0a49e0f47e4983abd4a1d4cd368a0a08ae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e79ac3ab-71c8-47f1-86ac-df85ca41f566\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e3b8d70541fd887d0c347858c3265f25f555d6403e49d8081b16a8f7cb15ef44\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "tokenizer=image_processor,\n.     data_collator=collate_fn,\n.     compute_metrics=eval_compute_metrics_fn,\n. )\n\n>>> trainer.train()\n```\n<div>\n\n  <progress value='3210' max='3210' style='width:300px; height:20px; vertical-align: middle;'></progress>\n  [3210/3210 26:07, Epoch 30/30]\n</div>\n\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: left;\">\n      <th>Epoch</th>\n      <th>Training Loss</th>\n      <th>Validation Loss</th>\n      <th>Map</th>\n      <th>Map 50</th>\n      <th>Map 75</th>\n      <th>Map Small</th>\n      <th>Map Medium</th>\n      <th>Map Large</th>\n      <th>Mar 1</th>\n      <th>Mar 10</th>\n      <th>Mar 100</th>\n      <th>Mar Small</th>\n      <th>Mar Medium</th>\n      <th>Mar Large</th>\n      <th>Map Coverall</th>\n      <th>Mar 100 Coverall</th>\n      <th>Map Face Shield</th>\n      <th>Mar 100 Face Shield</th>\n      <th>Map Gloves</th>\n      <th>Mar 100 Gloves</th>\n      <th>Map Goggles</th>\n      <th>Mar 100 Goggles</th>\n      <th>Map Mask</th>\n      <th>Mar 100 Mask</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <td>1</td>\n      <td>No log</td>\n      <td>2.629903</td>\n      <td>0.008900</td>\n      <td>0.023200</td>\n      <td>0.006500</td>\n      <td>0.001300</td>\n      <td>0.002800</td>\n      <td>0.020500</td>\n      <td>0.021500</td>\n      <td>0.070400</td>\n      <td>0.101400</td>\n      <td>0.007600</td>\n      <td>0.106200</td>\n      <td>0.096100</td>\n      <td>0.036700</td>\n      <td>0.232000</td>\n      <td>0.000300</td>\n      <td>0.019000</td>\n      <td>0.003900</td>\n      <td>0.125400</td>\n      <td>0.000100</td>\n      <td>0.003100</td>\n      <td>0.003500</td>\n      <td>0.127600</td>\n    </tr>\n    <tr>\n      <td>2</td>\n      <td>No log</td>\n      <td>3.479864</td>\n      <td>0.014800</td>\n      <td>0.034600</td>\n      <td>0.010800</td>\n      <td>0.008600</td>\n      <td>0.011700</td>\n      <td>0.012500</td>\n      <td>0.041100</td>\n      <td>0.098700</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: e79ac3ab-71c8-47f1-86ac-df85ca41f566
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"e79ac3ab-71c8-47f1-86ac-df85ca41f566\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d2f51363-de09-4fa9-8ea3-25353e0e2b0c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"29bdec96642f6985411acfd94c6564a67995ae4a9ef1b4e6a735d0a0b9c15a1c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d6a9cb14-f154-45f9-85b5-262d30519655\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4ec71c4473b581d4c45912aa002df0f6d85e3e42ed94493ebe3878eb5a74afbe\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 25657, \"end_char_idx\": 27381, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "629903</td>\n      <td>0.008900</td>\n      <td>0.023200</td>\n      <td>0.006500</td>\n      <td>0.001300</td>\n      <td>0.002800</td>\n      <td>0.020500</td>\n      <td>0.021500</td>\n      <td>0.070400</td>\n      <td>0.101400</td>\n      <td>0.007600</td>\n      <td>0.106200</td>\n      <td>0.096100</td>\n      <td>0.036700</td>\n      <td>0.232000</td>\n      <td>0.000300</td>\n      <td>0.019000</td>\n      <td>0.003900</td>\n      <td>0.125400</td>\n      <td>0.000100</td>\n      <td>0.003100</td>\n      <td>0.003500</td>\n      <td>0.127600</td>\n    </tr>\n    <tr>\n      <td>2</td>\n      <td>No log</td>\n      <td>3.479864</td>\n      <td>0.014800</td>\n      <td>0.034600</td>\n      <td>0.010800</td>\n      <td>0.008600</td>\n      <td>0.011700</td>\n      <td>0.012500</td>\n      <td>0.041100</td>\n      <td>0.098700</td>\n      <td>0.130000</td>\n      <td>0.056000</td>\n      <td>0.062200</td>\n      <td>0.111900</td>\n      <td>0.053500</td>\n      <td>0.447300</td>\n      <td>0.010600</td>\n      <td>0.100000</td>\n      <td>0.000200</td>\n      <td>0.022800</td>\n      <td>0.000100</td>\n      <td>0.015400</td>\n      <td>0.009700</td>\n      <td>0.064400</td>\n    </tr>\n    <tr>\n      <td>3</td>\n      <td>No log</td>\n      <td>2.107622</td>\n      <td>0.041700</td>\n      <td>0.094000</td>\n      <td>0.034300</td>\n      <td>0.024100</td>\n      <td>0.026400</td>\n      <td>0.047400</td>\n      <td>0.091500</td>\n      <td>0.182800</td>\n      <td>0.225800</td>\n      <td>0.087200</td>\n      <td>0.199400</td>\n      <td>0.210600</td>\n      <td>0.150900</td>\n      <td>0.571200</td>\n      <td>0.017300</td>\n      <td>0.101300</td>\n      <td>0.007300</td>\n      <td>0.180400</td>\n      <td>0.002100</td>\n      <td>0.026200</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d6a9cb14-f154-45f9-85b5-262d30519655
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d6a9cb14-f154-45f9-85b5-262d30519655\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e79ac3ab-71c8-47f1-86ac-df85ca41f566\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"7ecc94fcd9460094a5c15af65a820f189f6e7749cb6255e3ee7144242cf1b8fb\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d4128a63-bf01-4576-b907-38549c5f2b90\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"fc4422e9985a27da2d43965fa2621a509b4205d33347f809f243518a1552106f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 26507, \"end_char_idx\": 28219, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "056000</td>\n      <td>0.062200</td>\n      <td>0.111900</td>\n      <td>0.053500</td>\n      <td>0.447300</td>\n      <td>0.010600</td>\n      <td>0.100000</td>\n      <td>0.000200</td>\n      <td>0.022800</td>\n      <td>0.000100</td>\n      <td>0.015400</td>\n      <td>0.009700</td>\n      <td>0.064400</td>\n    </tr>\n    <tr>\n      <td>3</td>\n      <td>No log</td>\n      <td>2.107622</td>\n      <td>0.041700</td>\n      <td>0.094000</td>\n      <td>0.034300</td>\n      <td>0.024100</td>\n      <td>0.026400</td>\n      <td>0.047400</td>\n      <td>0.091500</td>\n      <td>0.182800</td>\n      <td>0.225800</td>\n      <td>0.087200</td>\n      <td>0.199400</td>\n      <td>0.210600</td>\n      <td>0.150900</td>\n      <td>0.571200</td>\n      <td>0.017300</td>\n      <td>0.101300</td>\n      <td>0.007300</td>\n      <td>0.180400</td>\n      <td>0.002100</td>\n      <td>0.026200</td>\n      <td>0.031000</td>\n      <td>0.250200</td>\n    </tr>\n    <tr>\n      <td>4</td>\n      <td>No log</td>\n      <td>2.031242</td>\n      <td>0.055900</td>\n      <td>0.120600</td>\n      <td>0.046900</td>\n      <td>0.013800</td>\n      <td>0.038100</td>\n      <td>0.090300</td>\n      <td>0.105900</td>\n      <td>0.225600</td>\n      <td>0.266100</td>\n      <td>0.130200</td>\n      <td>0.228100</td>\n      <td>0.330000</td>\n      <td>0.191000</td>\n      <td>0.572100</td>\n      <td>0.010600</td>\n      <td>0.157000</td>\n      <td>0.014600</td>\n      <td>0.235300</td>\n      <td>0.001700</td>\n      <td>0.052300</td>\n      <td>0.061800</td>\n      <td>0.313800</td>\n    </tr>\n    <tr>\n      <td>5</td>\n      <td>3.889400</td>\n      <td>1.883433</td>\n      <td>0.089700</td>\n      <td>0.201800</td>\n      <td>0.067300</td>\n      <td>0.022800</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d4128a63-bf01-4576-b907-38549c5f2b90
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d4128a63-bf01-4576-b907-38549c5f2b90\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d6a9cb14-f154-45f9-85b5-262d30519655\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"f8a645dd9d35f31e40672c9f0ff20692d0fb2269dc047f8fd523fcae681f48ae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"81e2cd83-900b-404b-b791-7fd998609224\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"eb4318405ff3b14377badcb4ae9417f008f8e2b1828339a774c5351b74e5dc58\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 27357, \"end_char_idx\": 29071, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "026200</td>\n      <td>0.031000</td>\n      <td>0.250200</td>\n    </tr>\n    <tr>\n      <td>4</td>\n      <td>No log</td>\n      <td>2.031242</td>\n      <td>0.055900</td>\n      <td>0.120600</td>\n      <td>0.046900</td>\n      <td>0.013800</td>\n      <td>0.038100</td>\n      <td>0.090300</td>\n      <td>0.105900</td>\n      <td>0.225600</td>\n      <td>0.266100</td>\n      <td>0.130200</td>\n      <td>0.228100</td>\n      <td>0.330000</td>\n      <td>0.191000</td>\n      <td>0.572100</td>\n      <td>0.010600</td>\n      <td>0.157000</td>\n      <td>0.014600</td>\n      <td>0.235300</td>\n      <td>0.001700</td>\n      <td>0.052300</td>\n      <td>0.061800</td>\n      <td>0.313800</td>\n    </tr>\n    <tr>\n      <td>5</td>\n      <td>3.889400</td>\n      <td>1.883433</td>\n      <td>0.089700</td>\n      <td>0.201800</td>\n      <td>0.067300</td>\n      <td>0.022800</td>\n      <td>0.065300</td>\n      <td>0.129500</td>\n      <td>0.136000</td>\n      <td>0.272200</td>\n      <td>0.303700</td>\n      <td>0.112900</td>\n      <td>0.312500</td>\n      <td>0.424600</td>\n      <td>0.300200</td>\n      <td>0.585100</td>\n      <td>0.032700</td>\n      <td>0.202500</td>\n      <td>0.031300</td>\n      <td>0.271000</td>\n      <td>0.008700</td>\n      <td>0.126200</td>\n      <td>0.075500</td>\n      <td>0.333800</td>\n    </tr>\n    <tr>\n      <td>6</td>\n      <td>3.889400</td>\n      <td>1.807503</td>\n      <td>0.118500</td>\n      <td>0.270900</td>\n      <td>0.090200</td>\n      <td>0.034900</td>\n      <td>0.076700</td>\n      <td>0.152500</td>\n      <td>0.146100</td>\n      <td>0.297800</td>\n      <td>0.325400</td>\n      <td>0.171700</td>\n      <td>0.283700</td>\n      <td>0.545900</td>\n      <td>0.396900</td>\n      <td>0.554500</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 81e2cd83-900b-404b-b791-7fd998609224
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"81e2cd83-900b-404b-b791-7fd998609224\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d4128a63-bf01-4576-b907-38549c5f2b90\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"6586ea3cc7637c9ab7e2d02ff475a2d700cc946b3eb7378d3e283ef73d165b48\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"92afc537-3328-432c-ac7d-4be76ae576ad\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e7a715484b69a165f548bd7622a287d74ab1d22ef8ff711aba00d1860da097e2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 28219, \"end_char_idx\": 29935, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "065300</td>\n      <td>0.129500</td>\n      <td>0.136000</td>\n      <td>0.272200</td>\n      <td>0.303700</td>\n      <td>0.112900</td>\n      <td>0.312500</td>\n      <td>0.424600</td>\n      <td>0.300200</td>\n      <td>0.585100</td>\n      <td>0.032700</td>\n      <td>0.202500</td>\n      <td>0.031300</td>\n      <td>0.271000</td>\n      <td>0.008700</td>\n      <td>0.126200</td>\n      <td>0.075500</td>\n      <td>0.333800</td>\n    </tr>\n    <tr>\n      <td>6</td>\n      <td>3.889400</td>\n      <td>1.807503</td>\n      <td>0.118500</td>\n      <td>0.270900</td>\n      <td>0.090200</td>\n      <td>0.034900</td>\n      <td>0.076700</td>\n      <td>0.152500</td>\n      <td>0.146100</td>\n      <td>0.297800</td>\n      <td>0.325400</td>\n      <td>0.171700</td>\n      <td>0.283700</td>\n      <td>0.545900</td>\n      <td>0.396900</td>\n      <td>0.554500</td>\n      <td>0.043000</td>\n      <td>0.262000</td>\n      <td>0.054500</td>\n      <td>0.271900</td>\n      <td>0.020300</td>\n      <td>0.230800</td>\n      <td>0.077600</td>\n      <td>0.308000</td>\n    </tr>\n    <tr>\n      <td>7</td>\n      <td>3.889400</td>\n      <td>1.716169</td>\n      <td>0.143500</td>\n      <td>0.307700</td>\n      <td>0.123200</td>\n      <td>0.045800</td>\n      <td>0.097800</td>\n      <td>0.258300</td>\n      <td>0.165300</td>\n      <td>0.327700</td>\n      <td>0.352600</td>\n      <td>0.140900</td>\n      <td>0.336700</td>\n      <td>0.599400</td>\n      <td>0.442900</td>\n      <td>0.620700</td>\n      <td>0.069400</td>\n      <td>0.301300</td>\n      <td>0.081600</td>\n      <td>0.292000</td>\n      <td>0.011000</td>\n      <td>0.230800</td>\n      <td>0.112700</td>\n      <td>0.318200</td>\n    </tr>\n    <tr>\n      <td>8</td>\n      <td>3.889400</td>\n      <td>1."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 92afc537-3328-432c-ac7d-4be76ae576ad
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"92afc537-3328-432c-ac7d-4be76ae576ad\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"81e2cd83-900b-404b-b791-7fd998609224\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"425cad898c31c38f09ee2f2bb7c27bfa1403b5f4f27df15c126a209f92817fe9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a2e98398-e258-4bda-8693-f9b2758cbafa\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e32bf384f8ca4d2ff6407564725c87abc278c324dfdc34dd44ad639f3b7dba9a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 29071, \"end_char_idx\": 30787, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "043000</td>\n      <td>0.262000</td>\n      <td>0.054500</td>\n      <td>0.271900</td>\n      <td>0.020300</td>\n      <td>0.230800</td>\n      <td>0.077600</td>\n      <td>0.308000</td>\n    </tr>\n    <tr>\n      <td>7</td>\n      <td>3.889400</td>\n      <td>1.716169</td>\n      <td>0.143500</td>\n      <td>0.307700</td>\n      <td>0.123200</td>\n      <td>0.045800</td>\n      <td>0.097800</td>\n      <td>0.258300</td>\n      <td>0.165300</td>\n      <td>0.327700</td>\n      <td>0.352600</td>\n      <td>0.140900</td>\n      <td>0.336700</td>\n      <td>0.599400</td>\n      <td>0.442900</td>\n      <td>0.620700</td>\n      <td>0.069400</td>\n      <td>0.301300</td>\n      <td>0.081600</td>\n      <td>0.292000</td>\n      <td>0.011000</td>\n      <td>0.230800</td>\n      <td>0.112700</td>\n      <td>0.318200</td>\n    </tr>\n    <tr>\n      <td>8</td>\n      <td>3.889400</td>\n      <td>1.679014</td>\n      <td>0.153000</td>\n      <td>0.355800</td>\n      <td>0.127900</td>\n      <td>0.038700</td>\n      <td>0.115600</td>\n      <td>0.291600</td>\n      <td>0.176000</td>\n      <td>0.322500</td>\n      <td>0.349700</td>\n      <td>0.135600</td>\n      <td>0.326100</td>\n      <td>0.643700</td>\n      <td>0.431700</td>\n      <td>0.582900</td>\n      <td>0.069800</td>\n      <td>0.265800</td>\n      <td>0.088600</td>\n      <td>0.274600</td>\n      <td>0.028300</td>\n      <td>0.280000</td>\n      <td>0.146700</td>\n      <td>0.345300</td>\n    </tr>\n    <tr>\n      <td>9</td>\n      <td>3.889400</td>\n      <td>1.618239</td>\n      <td>0.172100</td>\n      <td>0.375300</td>\n      <td>0.137600</td>\n      <td>0.046100</td>\n      <td>0.141700</td>\n      <td>0.308500</td>\n      <td>0.194000</td>\n      <td>0.356200</td>\n      <td>0.386200</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a2e98398-e258-4bda-8693-f9b2758cbafa
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"a2e98398-e258-4bda-8693-f9b2758cbafa\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"92afc537-3328-432c-ac7d-4be76ae576ad\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"f8b5cdf6200e1e523386d947330720ddec93937dd77ec3c71ebf20abaac03d8d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c9f97e65-f590-4845-8ad6-8ece536d722b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"bbbcf3788fd8f7266a224ac316f989ff78479d775dbae48d943664fd1418b997\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 29935, \"end_char_idx\": 31640, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "679014</td>\n      <td>0.153000</td>\n      <td>0.355800</td>\n      <td>0.127900</td>\n      <td>0.038700</td>\n      <td>0.115600</td>\n      <td>0.291600</td>\n      <td>0.176000</td>\n      <td>0.322500</td>\n      <td>0.349700</td>\n      <td>0.135600</td>\n      <td>0.326100</td>\n      <td>0.643700</td>\n      <td>0.431700</td>\n      <td>0.582900</td>\n      <td>0.069800</td>\n      <td>0.265800</td>\n      <td>0.088600</td>\n      <td>0.274600</td>\n      <td>0.028300</td>\n      <td>0.280000</td>\n      <td>0.146700</td>\n      <td>0.345300</td>\n    </tr>\n    <tr>\n      <td>9</td>\n      <td>3.889400</td>\n      <td>1.618239</td>\n      <td>0.172100</td>\n      <td>0.375300</td>\n      <td>0.137600</td>\n      <td>0.046100</td>\n      <td>0.141700</td>\n      <td>0.308500</td>\n      <td>0.194000</td>\n      <td>0.356200</td>\n      <td>0.386200</td>\n      <td>0.162400</td>\n      <td>0.359200</td>\n      <td>0.677700</td>\n      <td>0.469800</td>\n      <td>0.623900</td>\n      <td>0.102100</td>\n      <td>0.317700</td>\n      <td>0.099100</td>\n      <td>0.290200</td>\n      <td>0.029300</td>\n      <td>0.335400</td>\n      <td>0.160200</td>\n      <td>0.364000</td>\n    </tr>\n    <tr>\n      <td>10</td>\n      <td>1.599700</td>\n      <td>1.572512</td>\n      <td>0.179500</td>\n      <td>0.400400</td>\n      <td>0.147200</td>\n      <td>0.056500</td>\n      <td>0.141700</td>\n      <td>0.316700</td>\n      <td>0.213100</td>\n      <td>0.357600</td>\n      <td>0.381300</td>\n      <td>0.197900</td>\n      <td>0.344300</td>\n      <td>0.638500</td>\n      <td>0.466900</td>\n      <td>0.623900</td>\n      <td>0.101300</td>\n      <td>0.311400</td>\n      <td>0.104700</td>\n      <td>0.279500</td>\n      <td>0.051600</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c9f97e65-f590-4845-8ad6-8ece536d722b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"c9f97e65-f590-4845-8ad6-8ece536d722b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a2e98398-e258-4bda-8693-f9b2758cbafa\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"5d7ff6def374f467033c6de4a09e366aeaf661e2fc6cc1abcbe6465696ffed1e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ed2d56ae-0f3a-4b73-9613-45ee0c0ac684\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"939d090f4a93794149c0061a726581a0762304442ea1b3e64071ab9ec56bb0ba\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 30787, \"end_char_idx\": 32506, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "162400</td>\n      <td>0.359200</td>\n      <td>0.677700</td>\n      <td>0.469800</td>\n      <td>0.623900</td>\n      <td>0.102100</td>\n      <td>0.317700</td>\n      <td>0.099100</td>\n      <td>0.290200</td>\n      <td>0.029300</td>\n      <td>0.335400</td>\n      <td>0.160200</td>\n      <td>0.364000</td>\n    </tr>\n    <tr>\n      <td>10</td>\n      <td>1.599700</td>\n      <td>1.572512</td>\n      <td>0.179500</td>\n      <td>0.400400</td>\n      <td>0.147200</td>\n      <td>0.056500</td>\n      <td>0.141700</td>\n      <td>0.316700</td>\n      <td>0.213100</td>\n      <td>0.357600</td>\n      <td>0.381300</td>\n      <td>0.197900</td>\n      <td>0.344300</td>\n      <td>0.638500</td>\n      <td>0.466900</td>\n      <td>0.623900</td>\n      <td>0.101300</td>\n      <td>0.311400</td>\n      <td>0.104700</td>\n      <td>0.279500</td>\n      <td>0.051600</td>\n      <td>0.338500</td>\n      <td>0.173000</td>\n      <td>0.353300</td>\n    </tr>\n    <tr>\n      <td>11</td>\n      <td>1.599700</td>\n      <td>1.528889</td>\n      <td>0.192200</td>\n      <td>0.415000</td>\n      <td>0.160800</td>\n      <td>0.053700</td>\n      <td>0.150500</td>\n      <td>0.378000</td>\n      <td>0.211500</td>\n      <td>0.371700</td>\n      <td>0.397800</td>\n      <td>0.204900</td>\n      <td>0.374600</td>\n      <td>0.684800</td>\n      <td>0.491900</td>\n      <td>0.632400</td>\n      <td>0.131200</td>\n      <td>0.346800</td>\n      <td>0.122000</td>\n      <td>0.300900</td>\n      <td>0.038400</td>\n      <td>0.344600</td>\n      <td>0.177500</td>\n      <td>0.364400</td>\n    </tr>\n    <tr>\n      <td>12</td>\n      <td>1.599700</td>\n      <td>1.517532</td>\n      <td>0.198300</td>\n      <td>0.429800</td>\n      <td>0.159800</td>\n      <td>0.066400</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: ed2d56ae-0f3a-4b73-9613-45ee0c0ac684
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"ed2d56ae-0f3a-4b73-9613-45ee0c0ac684\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c9f97e65-f590-4845-8ad6-8ece536d722b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"e12902a74f2ec1caecd348c00e69df5cab942ac197bae3967d92f2a91a494b3b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c20a6202-ea6f-4951-a4d0-d810fa4994a9\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"2b8b833c8606730771ccb0e973478e78a34ee844a105af9d063206d9bb2ff453\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 31640, \"end_char_idx\": 33359, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "338500</td>\n      <td>0.173000</td>\n      <td>0.353300</td>\n    </tr>\n    <tr>\n      <td>11</td>\n      <td>1.599700</td>\n      <td>1.528889</td>\n      <td>0.192200</td>\n      <td>0.415000</td>\n      <td>0.160800</td>\n      <td>0.053700</td>\n      <td>0.150500</td>\n      <td>0.378000</td>\n      <td>0.211500</td>\n      <td>0.371700</td>\n      <td>0.397800</td>\n      <td>0.204900</td>\n      <td>0.374600</td>\n      <td>0.684800</td>\n      <td>0.491900</td>\n      <td>0.632400</td>\n      <td>0.131200</td>\n      <td>0.346800</td>\n      <td>0.122000</td>\n      <td>0.300900</td>\n      <td>0.038400</td>\n      <td>0.344600</td>\n      <td>0.177500</td>\n      <td>0.364400</td>\n    </tr>\n    <tr>\n      <td>12</td>\n      <td>1.599700</td>\n      <td>1.517532</td>\n      <td>0.198300</td>\n      <td>0.429800</td>\n      <td>0.159800</td>\n      <td>0.066400</td>\n      <td>0.162900</td>\n      <td>0.383300</td>\n      <td>0.220700</td>\n      <td>0.382100</td>\n      <td>0.405400</td>\n      <td>0.214800</td>\n      <td>0.383200</td>\n      <td>0.672900</td>\n      <td>0.469000</td>\n      <td>0.610400</td>\n      <td>0.167800</td>\n      <td>0.379700</td>\n      <td>0.119700</td>\n      <td>0.307100</td>\n      <td>0.038100</td>\n      <td>0.335400</td>\n      <td>0.196800</td>\n      <td>0.394200</td>\n    </tr>\n    <tr>\n      <td>13</td>\n      <td>1.599700</td>\n      <td>1.488849</td>\n      <td>0.209800</td>\n      <td>0.452300</td>\n      <td>0.172300</td>\n      <td>0.094900</td>\n      <td>0.171100</td>\n      <td>0.437800</td>\n      <td>0.222000</td>\n      <td>0.379800</td>\n      <td>0.411500</td>\n      <td>0.203800</td>\n      <td>0.397300</td>\n      <td>0.707500</td>\n      <td>0.470700</td>\n      <td>0.620700</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c20a6202-ea6f-4951-a4d0-d810fa4994a9
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"c20a6202-ea6f-4951-a4d0-d810fa4994a9\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ed2d56ae-0f3a-4b73-9613-45ee0c0ac684\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d6dd8db7830b29a1c00df414b4142ea804e01820af5b1d78cf18c4ed3df79928\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c032ac6a-54e7-4e4b-94a0-f72a2f1e11f8\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"681df8a3bdec3ada79e24e702726a5f7b23eea3db9c4584c55c2c40b49957d62\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 32506, \"end_char_idx\": 34225, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "162900</td>\n      <td>0.383300</td>\n      <td>0.220700</td>\n      <td>0.382100</td>\n      <td>0.405400</td>\n      <td>0.214800</td>\n      <td>0.383200</td>\n      <td>0.672900</td>\n      <td>0.469000</td>\n      <td>0.610400</td>\n      <td>0.167800</td>\n      <td>0.379700</td>\n      <td>0.119700</td>\n      <td>0.307100</td>\n      <td>0.038100</td>\n      <td>0.335400</td>\n      <td>0.196800</td>\n      <td>0.394200</td>\n    </tr>\n    <tr>\n      <td>13</td>\n      <td>1.599700</td>\n      <td>1.488849</td>\n      <td>0.209800</td>\n      <td>0.452300</td>\n      <td>0.172300</td>\n      <td>0.094900</td>\n      <td>0.171100</td>\n      <td>0.437800</td>\n      <td>0.222000</td>\n      <td>0.379800</td>\n      <td>0.411500</td>\n      <td>0.203800</td>\n      <td>0.397300</td>\n      <td>0.707500</td>\n      <td>0.470700</td>\n      <td>0.620700</td>\n      <td>0.186900</td>\n      <td>0.407600</td>\n      <td>0.124200</td>\n      <td>0.306700</td>\n      <td>0.059300</td>\n      <td>0.355400</td>\n      <td>0.207700</td>\n      <td>0.367100</td>\n    </tr>\n    <tr>\n      <td>14</td>\n      <td>1.599700</td>\n      <td>1.482210</td>\n      <td>0.228900</td>\n      <td>0.482600</td>\n      <td>0.187800</td>\n      <td>0.083600</td>\n      <td>0.191800</td>\n      <td>0.444100</td>\n      <td>0.225900</td>\n      <td>0.376900</td>\n      <td>0.407400</td>\n      <td>0.182500</td>\n      <td>0.384800</td>\n      <td>0.700600</td>\n      <td>0.512100</td>\n      <td>0.640100</td>\n      <td>0.175000</td>\n      <td>0.363300</td>\n      <td>0.144300</td>\n      <td>0.300000</td>\n      <td>0.083100</td>\n      <td>0.363100</td>\n      <td>0.229900</td>\n      <td>0.370700</td>\n    </tr>\n    <tr>\n      <td>15</td>\n      <td>1.326800</td>\n      <td>1."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c032ac6a-54e7-4e4b-94a0-f72a2f1e11f8
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"c032ac6a-54e7-4e4b-94a0-f72a2f1e11f8\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c20a6202-ea6f-4951-a4d0-d810fa4994a9\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"7e76842df688ee38d98d8435e99e2edc7bb2ffe61a0d30e4a9268d5c9e44198e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"03ece2b7-baef-4af8-ad42-de69b32ceeb9\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"50120ce80d54abad8b1b009e80b1e2af3143afb5c900b75966952e9dd1a78067\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 33359, \"end_char_idx\": 35078, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "186900</td>\n      <td>0.407600</td>\n      <td>0.124200</td>\n      <td>0.306700</td>\n      <td>0.059300</td>\n      <td>0.355400</td>\n      <td>0.207700</td>\n      <td>0.367100</td>\n    </tr>\n    <tr>\n      <td>14</td>\n      <td>1.599700</td>\n      <td>1.482210</td>\n      <td>0.228900</td>\n      <td>0.482600</td>\n      <td>0.187800</td>\n      <td>0.083600</td>\n      <td>0.191800</td>\n      <td>0.444100</td>\n      <td>0.225900</td>\n      <td>0.376900</td>\n      <td>0.407400</td>\n      <td>0.182500</td>\n      <td>0.384800</td>\n      <td>0.700600</td>\n      <td>0.512100</td>\n      <td>0.640100</td>\n      <td>0.175000</td>\n      <td>0.363300</td>\n      <td>0.144300</td>\n      <td>0.300000</td>\n      <td>0.083100</td>\n      <td>0.363100</td>\n      <td>0.229900</td>\n      <td>0.370700</td>\n    </tr>\n    <tr>\n      <td>15</td>\n      <td>1.326800</td>\n      <td>1.475198</td>\n      <td>0.216300</td>\n      <td>0.455600</td>\n      <td>0.174900</td>\n      <td>0.088500</td>\n      <td>0.183500</td>\n      <td>0.424400</td>\n      <td>0.226900</td>\n      <td>0.373400</td>\n      <td>0.404300</td>\n      <td>0.199200</td>\n      <td>0.396400</td>\n      <td>0.677800</td>\n      <td>0.496300</td>\n      <td>0.633800</td>\n      <td>0.166300</td>\n      <td>0.392400</td>\n      <td>0.128900</td>\n      <td>0.312900</td>\n      <td>0.085200</td>\n      <td>0.312300</td>\n      <td>0.205000</td>\n      <td>0.370200</td>\n    </tr>\n    <tr>\n      <td>16</td>\n      <td>1.326800</td>\n      <td>1.459697</td>\n      <td>0.233200</td>\n      <td>0.504200</td>\n      <td>0.192200</td>\n      <td>0.096000</td>\n      <td>0.202000</td>\n      <td>0.430800</td>\n      <td>0.239100</td>\n      <td>0.382400</td>\n      <td>0.412600</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 03ece2b7-baef-4af8-ad42-de69b32ceeb9
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"03ece2b7-baef-4af8-ad42-de69b32ceeb9\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c032ac6a-54e7-4e4b-94a0-f72a2f1e11f8\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d9d9a93ee46df656774d68d037c47f113ce64c4da171378663fe30631847dd55\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b2edd459-60c4-4732-93ca-f395beb7301e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"884bb5e26c6e24aa4a56f35a3f1cdd9b23675715e65229abf6f9129d97122a8e\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 34225, \"end_char_idx\": 35931, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "475198</td>\n      <td>0.216300</td>\n      <td>0.455600</td>\n      <td>0.174900</td>\n      <td>0.088500</td>\n      <td>0.183500</td>\n      <td>0.424400</td>\n      <td>0.226900</td>\n      <td>0.373400</td>\n      <td>0.404300</td>\n      <td>0.199200</td>\n      <td>0.396400</td>\n      <td>0.677800</td>\n      <td>0.496300</td>\n      <td>0.633800</td>\n      <td>0.166300</td>\n      <td>0.392400</td>\n      <td>0.128900</td>\n      <td>0.312900</td>\n      <td>0.085200</td>\n      <td>0.312300</td>\n      <td>0.205000</td>\n      <td>0.370200</td>\n    </tr>\n    <tr>\n      <td>16</td>\n      <td>1.326800</td>\n      <td>1.459697</td>\n      <td>0.233200</td>\n      <td>0.504200</td>\n      <td>0.192200</td>\n      <td>0.096000</td>\n      <td>0.202000</td>\n      <td>0.430800</td>\n      <td>0.239100</td>\n      <td>0.382400</td>\n      <td>0.412600</td>\n      <td>0.219500</td>\n      <td>0.403100</td>\n      <td>0.670400</td>\n      <td>0.485200</td>\n      <td>0.625200</td>\n      <td>0.196500</td>\n      <td>0.410100</td>\n      <td>0.135700</td>\n      <td>0.299600</td>\n      <td>0.123100</td>\n      <td>0.356900</td>\n      <td>0.225300</td>\n      <td>0.371100</td>\n    </tr>\n    <tr>\n      <td>17</td>\n      <td>1.326800</td>\n      <td>1.407340</td>\n      <td>0.243400</td>\n      <td>0.511900</td>\n      <td>0.204500</td>\n      <td>0.121000</td>\n      <td>0.215700</td>\n      <td>0.468000</td>\n      <td>0.246200</td>\n      <td>0.394600</td>\n      <td>0.424200</td>\n      <td>0.225900</td>\n      <td>0.416100</td>\n      <td>0.705200</td>\n      <td>0.494900</td>\n      <td>0.638300</td>\n      <td>0.224900</td>\n      <td>0.430400</td>\n      <td>0.157200</td>\n      <td>0.317900</td>\n      <td>0.115700</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b2edd459-60c4-4732-93ca-f395beb7301e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"b2edd459-60c4-4732-93ca-f395beb7301e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"03ece2b7-baef-4af8-ad42-de69b32ceeb9\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"a7ba4a519846a974aaaf5ac4735e172f61ac665b886d0007c5b27635ff13ce3a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b058292f-d536-41cb-b30d-7ef09d9ac8d8\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f5e2ed09e692af5f24b4ab4ac43a5967511e83d14a34b0d9c5badb2179327433\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 35078, \"end_char_idx\": 36797, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "219500</td>\n      <td>0.403100</td>\n      <td>0.670400</td>\n      <td>0.485200</td>\n      <td>0.625200</td>\n      <td>0.196500</td>\n      <td>0.410100</td>\n      <td>0.135700</td>\n      <td>0.299600</td>\n      <td>0.123100</td>\n      <td>0.356900</td>\n      <td>0.225300</td>\n      <td>0.371100</td>\n    </tr>\n    <tr>\n      <td>17</td>\n      <td>1.326800</td>\n      <td>1.407340</td>\n      <td>0.243400</td>\n      <td>0.511900</td>\n      <td>0.204500</td>\n      <td>0.121000</td>\n      <td>0.215700</td>\n      <td>0.468000</td>\n      <td>0.246200</td>\n      <td>0.394600</td>\n      <td>0.424200</td>\n      <td>0.225900</td>\n      <td>0.416100</td>\n      <td>0.705200</td>\n      <td>0.494900</td>\n      <td>0.638300</td>\n      <td>0.224900</td>\n      <td>0.430400</td>\n      <td>0.157200</td>\n      <td>0.317900</td>\n      <td>0.115700</td>\n      <td>0.369200</td>\n      <td>0.224200</td>\n      <td>0.365300</td>\n    </tr>\n    <tr>\n      <td>18</td>\n      <td>1.326800</td>\n      <td>1.419522</td>\n      <td>0.245100</td>\n      <td>0.521500</td>\n      <td>0.210000</td>\n      <td>0.116100</td>\n      <td>0.211500</td>\n      <td>0.489900</td>\n      <td>0.255400</td>\n      <td>0.391600</td>\n      <td>0.419700</td>\n      <td>0.198800</td>\n      <td>0.421200</td>\n      <td>0.701400</td>\n      <td>0.501800</td>\n      <td>0.634200</td>\n      <td>0.226700</td>\n      <td>0.410100</td>\n      <td>0.154400</td>\n      <td>0.321400</td>\n      <td>0.105900</td>\n      <td>0.352300</td>\n      <td>0.236700</td>\n      <td>0.380400</td>\n    </tr>\n    <tr>\n      <td>19</td>\n      <td>1.158600</td>\n      <td>1.398764</td>\n      <td>0.253600</td>\n      <td>0.519200</td>\n      <td>0.213600</td>\n      <td>0.135200</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b058292f-d536-41cb-b30d-7ef09d9ac8d8
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"b058292f-d536-41cb-b30d-7ef09d9ac8d8\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b2edd459-60c4-4732-93ca-f395beb7301e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"98d410a89045fb493d4637349eaedc3477fb644feef5629736e3b1a5ef818b68\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"bdbf228a-ed07-4eef-b107-e531a2a61f95\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"092fa7f1d5b14d0cc6a0e83453021b2e9a7d89626975ee1f414c66738f01bcae\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 35931, \"end_char_idx\": 37650, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "369200</td>\n      <td>0.224200</td>\n      <td>0.365300</td>\n    </tr>\n    <tr>\n      <td>18</td>\n      <td>1.326800</td>\n      <td>1.419522</td>\n      <td>0.245100</td>\n      <td>0.521500</td>\n      <td>0.210000</td>\n      <td>0.116100</td>\n      <td>0.211500</td>\n      <td>0.489900</td>\n      <td>0.255400</td>\n      <td>0.391600</td>\n      <td>0.419700</td>\n      <td>0.198800</td>\n      <td>0.421200</td>\n      <td>0.701400</td>\n      <td>0.501800</td>\n      <td>0.634200</td>\n      <td>0.226700</td>\n      <td>0.410100</td>\n      <td>0.154400</td>\n      <td>0.321400</td>\n      <td>0.105900</td>\n      <td>0.352300</td>\n      <td>0.236700</td>\n      <td>0.380400</td>\n    </tr>\n    <tr>\n      <td>19</td>\n      <td>1.158600</td>\n      <td>1.398764</td>\n      <td>0.253600</td>\n      <td>0.519200</td>\n      <td>0.213600</td>\n      <td>0.135200</td>\n      <td>0.207700</td>\n      <td>0.491900</td>\n      <td>0.257300</td>\n      <td>0.397300</td>\n      <td>0.428000</td>\n      <td>0.241400</td>\n      <td>0.401800</td>\n      <td>0.703500</td>\n      <td>0.509700</td>\n      <td>0.631100</td>\n      <td>0.236700</td>\n      <td>0.441800</td>\n      <td>0.155900</td>\n      <td>0.330800</td>\n      <td>0.128100</td>\n      <td>0.352300</td>\n      <td>0.237500</td>\n      <td>0.384000</td>\n    </tr>\n    <tr>\n      <td>20</td>\n      <td>1.158600</td>\n      <td>1.390591</td>\n      <td>0.248800</td>\n      <td>0.520200</td>\n      <td>0.216600</td>\n      <td>0.127500</td>\n      <td>0.211400</td>\n      <td>0.471900</td>\n      <td>0.258300</td>\n      <td>0.407000</td>\n      <td>0.429100</td>\n      <td>0.240300</td>\n      <td>0.407600</td>\n      <td>0.708500</td>\n      <td>0.505800</td>\n      <td>0.623400</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: bdbf228a-ed07-4eef-b107-e531a2a61f95
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"bdbf228a-ed07-4eef-b107-e531a2a61f95\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b058292f-d536-41cb-b30d-7ef09d9ac8d8\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"b9806917656863919ffcb2502f15697d67ad987f093f5e502df2a609752a3c1d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"6933d054-5111-48e5-86e9-f71caa7196a0\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c206e2aba1c2e2db8e5ed8f34faceb2af3c9591ce2587612691b81e43049eeb6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 36797, \"end_char_idx\": 38516, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "207700</td>\n      <td>0.491900</td>\n      <td>0.257300</td>\n      <td>0.397300</td>\n      <td>0.428000</td>\n      <td>0.241400</td>\n      <td>0.401800</td>\n      <td>0.703500</td>\n      <td>0.509700</td>\n      <td>0.631100</td>\n      <td>0.236700</td>\n      <td>0.441800</td>\n      <td>0.155900</td>\n      <td>0.330800</td>\n      <td>0.128100</td>\n      <td>0.352300</td>\n      <td>0.237500</td>\n      <td>0.384000</td>\n    </tr>\n    <tr>\n      <td>20</td>\n      <td>1.158600</td>\n      <td>1.390591</td>\n      <td>0.248800</td>\n      <td>0.520200</td>\n      <td>0.216600</td>\n      <td>0.127500</td>\n      <td>0.211400</td>\n      <td>0.471900</td>\n      <td>0.258300</td>\n      <td>0.407000</td>\n      <td>0.429100</td>\n      <td>0.240300</td>\n      <td>0.407600</td>\n      <td>0.708500</td>\n      <td>0.505800</td>\n      <td>0.623400</td>\n      <td>0.235500</td>\n      <td>0.431600</td>\n      <td>0.150000</td>\n      <td>0.325000</td>\n      <td>0.125700</td>\n      <td>0.375400</td>\n      <td>0.227200</td>\n      <td>0.390200</td>\n    </tr>\n    <tr>\n      <td>21</td>\n      <td>1.158600</td>\n      <td>1.360608</td>\n      <td>0.262700</td>\n      <td>0.544800</td>\n      <td>0.222100</td>\n      <td>0.134700</td>\n      <td>0.230000</td>\n      <td>0.487500</td>\n      <td>0.269500</td>\n      <td>0.413300</td>\n      <td>0.436300</td>\n      <td>0.236200</td>\n      <td>0.419100</td>\n      <td>0.709300</td>\n      <td>0.514100</td>\n      <td>0.637400</td>\n      <td>0.257200</td>\n      <td>0.450600</td>\n      <td>0.165100</td>\n      <td>0.338400</td>\n      <td>0.139400</td>\n      <td>0.372300</td>\n      <td>0.237700</td>\n      <td>0.382700</td>\n    </tr>\n    <tr>\n      <td>22</td>\n      <td>1.158600</td>\n      <td>1."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 6933d054-5111-48e5-86e9-f71caa7196a0
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"6933d054-5111-48e5-86e9-f71caa7196a0\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"bdbf228a-ed07-4eef-b107-e531a2a61f95\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"75efab553a388d89495a58cfd25bc7a64ecdd9fba4e52092b5204803b1220211\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"dfc0c0f2-aaf5-4064-a007-d4de79db24c2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cb90b7d5f837b23c2bb070ff668090694bd9b4d301e1303d948b919f0020fe66\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 37650, \"end_char_idx\": 39369, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "235500</td>\n      <td>0.431600</td>\n      <td>0.150000</td>\n      <td>0.325000</td>\n      <td>0.125700</td>\n      <td>0.375400</td>\n      <td>0.227200</td>\n      <td>0.390200</td>\n    </tr>\n    <tr>\n      <td>21</td>\n      <td>1.158600</td>\n      <td>1.360608</td>\n      <td>0.262700</td>\n      <td>0.544800</td>\n      <td>0.222100</td>\n      <td>0.134700</td>\n      <td>0.230000</td>\n      <td>0.487500</td>\n      <td>0.269500</td>\n      <td>0.413300</td>\n      <td>0.436300</td>\n      <td>0.236200</td>\n      <td>0.419100</td>\n      <td>0.709300</td>\n      <td>0.514100</td>\n      <td>0.637400</td>\n      <td>0.257200</td>\n      <td>0.450600</td>\n      <td>0.165100</td>\n      <td>0.338400</td>\n      <td>0.139400</td>\n      <td>0.372300</td>\n      <td>0.237700</td>\n      <td>0.382700</td>\n    </tr>\n    <tr>\n      <td>22</td>\n      <td>1.158600</td>\n      <td>1.368296</td>\n      <td>0.262800</td>\n      <td>0.542400</td>\n      <td>0.236400</td>\n      <td>0.137400</td>\n      <td>0.228100</td>\n      <td>0.498500</td>\n      <td>0.266500</td>\n      <td>0.409000</td>\n      <td>0.433000</td>\n      <td>0.239900</td>\n      <td>0.418500</td>\n      <td>0.697500</td>\n      <td>0.520500</td>\n      <td>0.641000</td>\n      <td>0.257500</td>\n      <td>0.455700</td>\n      <td>0.162600</td>\n      <td>0.334800</td>\n      <td>0.140200</td>\n      <td>0.353800</td>\n      <td>0.233200</td>\n      <td>0.379600</td>\n    </tr>\n    <tr>\n      <td>23</td>\n      <td>1.158600</td>\n      <td>1.368176</td>\n      <td>0.264800</td>\n      <td>0.541100</td>\n      <td>0.233100</td>\n      <td>0.138200</td>\n      <td>0.223900</td>\n      <td>0.498700</td>\n      <td>0.272300</td>\n      <td>0.407400</td>\n      <td>0.434400</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: dfc0c0f2-aaf5-4064-a007-d4de79db24c2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"dfc0c0f2-aaf5-4064-a007-d4de79db24c2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"6933d054-5111-48e5-86e9-f71caa7196a0\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"a42eb577bc6d183070b970ab98fa77e015c695c7921dbcf8a3b59d47f2ba8f03\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"478214e9-409d-46c3-b510-e69082850998\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"108fb5b15066664482823b60bab61e4ec9e696ddca62cf0ada2baeb41f2d74d1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 38516, \"end_char_idx\": 40222, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "368296</td>\n      <td>0.262800</td>\n      <td>0.542400</td>\n      <td>0.236400</td>\n      <td>0.137400</td>\n      <td>0.228100</td>\n      <td>0.498500</td>\n      <td>0.266500</td>\n      <td>0.409000</td>\n      <td>0.433000</td>\n      <td>0.239900</td>\n      <td>0.418500</td>\n      <td>0.697500</td>\n      <td>0.520500</td>\n      <td>0.641000</td>\n      <td>0.257500</td>\n      <td>0.455700</td>\n      <td>0.162600</td>\n      <td>0.334800</td>\n      <td>0.140200</td>\n      <td>0.353800</td>\n      <td>0.233200</td>\n      <td>0.379600</td>\n    </tr>\n    <tr>\n      <td>23</td>\n      <td>1.158600</td>\n      <td>1.368176</td>\n      <td>0.264800</td>\n      <td>0.541100</td>\n      <td>0.233100</td>\n      <td>0.138200</td>\n      <td>0.223900</td>\n      <td>0.498700</td>\n      <td>0.272300</td>\n      <td>0.407400</td>\n      <td>0.434400</td>\n      <td>0.233100</td>\n      <td>0.418300</td>\n      <td>0.702000</td>\n      <td>0.524400</td>\n      <td>0.642300</td>\n      <td>0.262300</td>\n      <td>0.444300</td>\n      <td>0.159700</td>\n      <td>0.335300</td>\n      <td>0.140500</td>\n      <td>0.366200</td>\n      <td>0.236900</td>\n      <td>0.384000</td>\n    </tr>\n    <tr>\n      <td>24</td>\n      <td>1.049700</td>\n      <td>1.355271</td>\n      <td>0.269700</td>\n      <td>0.549200</td>\n      <td>0.239100</td>\n      <td>0.134700</td>\n      <td>0.229900</td>\n      <td>0.519200</td>\n      <td>0.274800</td>\n      <td>0.412700</td>\n      <td>0.437600</td>\n      <td>0.245400</td>\n      <td>0.417200</td>\n      <td>0.711200</td>\n      <td>0.523200</td>\n      <td>0.644100</td>\n      <td>0.272100</td>\n      <td>0.440500</td>\n      <td>0.166700</td>\n      <td>0.341500</td>\n      <td>0.137700</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 478214e9-409d-46c3-b510-e69082850998
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"478214e9-409d-46c3-b510-e69082850998\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"dfc0c0f2-aaf5-4064-a007-d4de79db24c2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"be68388591b61172fc6ecc2482a4b4f4c523047752a76d84ad7ad153acb94f97\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"24697754-e8d0-432b-8d7a-c347d9456bfa\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1ce097a499811ccd2fac259ac29813066670c0cb078fa4b1ea3c1ae03e3b3481\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 39369, \"end_char_idx\": 41088, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "233100</td>\n      <td>0.418300</td>\n      <td>0.702000</td>\n      <td>0.524400</td>\n      <td>0.642300</td>\n      <td>0.262300</td>\n      <td>0.444300</td>\n      <td>0.159700</td>\n      <td>0.335300</td>\n      <td>0.140500</td>\n      <td>0.366200</td>\n      <td>0.236900</td>\n      <td>0.384000</td>\n    </tr>\n    <tr>\n      <td>24</td>\n      <td>1.049700</td>\n      <td>1.355271</td>\n      <td>0.269700</td>\n      <td>0.549200</td>\n      <td>0.239100</td>\n      <td>0.134700</td>\n      <td>0.229900</td>\n      <td>0.519200</td>\n      <td>0.274800</td>\n      <td>0.412700</td>\n      <td>0.437600</td>\n      <td>0.245400</td>\n      <td>0.417200</td>\n      <td>0.711200</td>\n      <td>0.523200</td>\n      <td>0.644100</td>\n      <td>0.272100</td>\n      <td>0.440500</td>\n      <td>0.166700</td>\n      <td>0.341500</td>\n      <td>0.137700</td>\n      <td>0.373800</td>\n      <td>0.249000</td>\n      <td>0.388000</td>\n    </tr>\n    <tr>\n      <td>25</td>\n      <td>1.049700</td>\n      <td>1.355180</td>\n      <td>0.272500</td>\n      <td>0.547900</td>\n      <td>0.243800</td>\n      <td>0.149700</td>\n      <td>0.229900</td>\n      <td>0.523100</td>\n      <td>0.272500</td>\n      <td>0.415700</td>\n      <td>0.442200</td>\n      <td>0.256200</td>\n      <td>0.420200</td>\n      <td>0.705800</td>\n      <td>0.523900</td>\n      <td>0.639600</td>\n      <td>0.271700</td>\n      <td>0.451900</td>\n      <td>0.166300</td>\n      <td>0.346900</td>\n      <td>0.153700</td>\n      <td>0.383100</td>\n      <td>0.247000</td>\n      <td>0.389300</td>\n    </tr>\n    <tr>\n      <td>26</td>\n      <td>1.049700</td>\n      <td>1.349337</td>\n      <td>0.275600</td>\n      <td>0.556300</td>\n      <td>0.246400</td>\n      <td>0.146700</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 24697754-e8d0-432b-8d7a-c347d9456bfa
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"24697754-e8d0-432b-8d7a-c347d9456bfa\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"478214e9-409d-46c3-b510-e69082850998\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"b0329d896ab3f41a34e08c4b03f53cb615ca454e8be37b4d25933fdf47ba0dd7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d8decd53-0e06-4859-883c-f765fe967b58\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0ea5fcc2bf037a653b27fc45f254faee0670dedad78e4a966a3041d471cde9a4\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 40222, \"end_char_idx\": 41941, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "373800</td>\n      <td>0.249000</td>\n      <td>0.388000</td>\n    </tr>\n    <tr>\n      <td>25</td>\n      <td>1.049700</td>\n      <td>1.355180</td>\n      <td>0.272500</td>\n      <td>0.547900</td>\n      <td>0.243800</td>\n      <td>0.149700</td>\n      <td>0.229900</td>\n      <td>0.523100</td>\n      <td>0.272500</td>\n      <td>0.415700</td>\n      <td>0.442200</td>\n      <td>0.256200</td>\n      <td>0.420200</td>\n      <td>0.705800</td>\n      <td>0.523900</td>\n      <td>0.639600</td>\n      <td>0.271700</td>\n      <td>0.451900</td>\n      <td>0.166300</td>\n      <td>0.346900</td>\n      <td>0.153700</td>\n      <td>0.383100</td>\n      <td>0.247000</td>\n      <td>0.389300</td>\n    </tr>\n    <tr>\n      <td>26</td>\n      <td>1.049700</td>\n      <td>1.349337</td>\n      <td>0.275600</td>\n      <td>0.556300</td>\n      <td>0.246400</td>\n      <td>0.146700</td>\n      <td>0.234800</td>\n      <td>0.516300</td>\n      <td>0.274200</td>\n      <td>0.418300</td>\n      <td>0.440900</td>\n      <td>0.248700</td>\n      <td>0.418900</td>\n      <td>0.705800</td>\n      <td>0.523200</td>\n      <td>0.636500</td>\n      <td>0.274700</td>\n      <td>0.440500</td>\n      <td>0.172400</td>\n      <td>0.349100</td>\n      <td>0.155600</td>\n      <td>0.384600</td>\n      <td>0.252300</td>\n      <td>0.393800</td>\n    </tr>\n    <tr>\n      <td>27</td>\n      <td>1.049700</td>\n      <td>1.350782</td>\n      <td>0.275200</td>\n      <td>0.548700</td>\n      <td>0.246800</td>\n      <td>0.147300</td>\n      <td>0.236400</td>\n      <td>0.527200</td>\n      <td>0.280100</td>\n      <td>0.416200</td>\n      <td>0.442600</td>\n      <td>0.253400</td>\n      <td>0.424000</td>\n      <td>0.710300</td>\n      <td>0.526600</td>\n      <td>0.640100</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d8decd53-0e06-4859-883c-f765fe967b58
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"d8decd53-0e06-4859-883c-f765fe967b58\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"24697754-e8d0-432b-8d7a-c347d9456bfa\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"eea4576c6d6873529cc3bea66ea1778f283e2678101b921b93587b016d3cd8aa\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"7c79763e-b0af-4197-b09d-7e6b9096390e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b3ce0a443be5ab2466ca0c64fd00fabf3259ec1ecc365afdb2c7639f161401bf\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 41088, \"end_char_idx\": 42807, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "234800</td>\n      <td>0.516300</td>\n      <td>0.274200</td>\n      <td>0.418300</td>\n      <td>0.440900</td>\n      <td>0.248700</td>\n      <td>0.418900</td>\n      <td>0.705800</td>\n      <td>0.523200</td>\n      <td>0.636500</td>\n      <td>0.274700</td>\n      <td>0.440500</td>\n      <td>0.172400</td>\n      <td>0.349100</td>\n      <td>0.155600</td>\n      <td>0.384600</td>\n      <td>0.252300</td>\n      <td>0.393800</td>\n    </tr>\n    <tr>\n      <td>27</td>\n      <td>1.049700</td>\n      <td>1.350782</td>\n      <td>0.275200</td>\n      <td>0.548700</td>\n      <td>0.246800</td>\n      <td>0.147300</td>\n      <td>0.236400</td>\n      <td>0.527200</td>\n      <td>0.280100</td>\n      <td>0.416200</td>\n      <td>0.442600</td>\n      <td>0.253400</td>\n      <td>0.424000</td>\n      <td>0.710300</td>\n      <td>0.526600</td>\n      <td>0.640100</td>\n      <td>0.273200</td>\n      <td>0.445600</td>\n      <td>0.167000</td>\n      <td>0.346900</td>\n      <td>0.160100</td>\n      <td>0.387700</td>\n      <td>0.249200</td>\n      <td>0.392900</td>\n    </tr>\n    <tr>\n      <td>28</td>\n      <td>1.049700</td>\n      <td>1.346533</td>\n      <td>0.277000</td>\n      <td>0.552800</td>\n      <td>0.252900</td>\n      <td>0.147400</td>\n      <td>0.240000</td>\n      <td>0.527600</td>\n      <td>0.280900</td>\n      <td>0.420900</td>\n      <td>0.444100</td>\n      <td>0.255500</td>\n      <td>0.424500</td>\n      <td>0.711200</td>\n      <td>0.530200</td>\n      <td>0.646800</td>\n      <td>0.277400</td>\n      <td>0.441800</td>\n      <td>0.170900</td>\n      <td>0.346900</td>\n      <td>0.156600</td>\n      <td>0.389200</td>\n      <td>0.249600</td>\n      <td>0.396000</td>\n    </tr>\n    <tr>\n      <td>29</td>\n      <td>0.993700</td>\n      <td>1."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 7c79763e-b0af-4197-b09d-7e6b9096390e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"7c79763e-b0af-4197-b09d-7e6b9096390e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d8decd53-0e06-4859-883c-f765fe967b58\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"5b33e1d2e1d5d67dba5584bb5d11b6244c552e10c848f8be320d7b97b46991ff\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"023c0bb3-cd82-4f4d-97fc-6c27c50bab7e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"59982fcea0cccdb244a5f5733c581b0f3199be6ada77b32787c79b333e2706c8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 41941, \"end_char_idx\": 43660, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "273200</td>\n      <td>0.445600</td>\n      <td>0.167000</td>\n      <td>0.346900</td>\n      <td>0.160100</td>\n      <td>0.387700</td>\n      <td>0.249200</td>\n      <td>0.392900</td>\n    </tr>\n    <tr>\n      <td>28</td>\n      <td>1.049700</td>\n      <td>1.346533</td>\n      <td>0.277000</td>\n      <td>0.552800</td>\n      <td>0.252900</td>\n      <td>0.147400</td>\n      <td>0.240000</td>\n      <td>0.527600</td>\n      <td>0.280900</td>\n      <td>0.420900</td>\n      <td>0.444100</td>\n      <td>0.255500</td>\n      <td>0.424500</td>\n      <td>0.711200</td>\n      <td>0.530200</td>\n      <td>0.646800</td>\n      <td>0.277400</td>\n      <td>0.441800</td>\n      <td>0.170900</td>\n      <td>0.346900</td>\n      <td>0.156600</td>\n      <td>0.389200</td>\n      <td>0.249600</td>\n      <td>0.396000</td>\n    </tr>\n    <tr>\n      <td>29</td>\n      <td>0.993700</td>\n      <td>1.346575</td>\n      <td>0.277100</td>\n      <td>0.554800</td>\n      <td>0.252900</td>\n      <td>0.148400</td>\n      <td>0.239700</td>\n      <td>0.523600</td>\n      <td>0.278400</td>\n      <td>0.420000</td>\n      <td>0.443300</td>\n      <td>0.256300</td>\n      <td>0.424000</td>\n      <td>0.705600</td>\n      <td>0.529600</td>\n      <td>0.647300</td>\n      <td>0.273900</td>\n      <td>0.439200</td>\n      <td>0.174300</td>\n      <td>0.348700</td>\n      <td>0.157600</td>\n      <td>0.386200</td>\n      <td>0.250100</td>\n      <td>0.395100</td>\n    </tr>\n    <tr>\n      <td>30</td>\n      <td>0.993700</td>\n      <td>1.346446</td>\n      <td>0.277400</td>\n      <td>0.554700</td>\n      <td>0.252700</td>\n      <td>0.147900</td>\n      <td>0.240800</td>\n      <td>0.523600</td>\n      <td>0.278800</td>\n      <td>0.420400</td>\n      <td>0.443300</td>\n      <td>0."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 023c0bb3-cd82-4f4d-97fc-6c27c50bab7e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"023c0bb3-cd82-4f4d-97fc-6c27c50bab7e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"7c79763e-b0af-4197-b09d-7e6b9096390e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"75cf3b74e03b2f1bbc7c5ad2b5e71d5ffca6cc15caa636cdcaec6b267f83b0e3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"359312d0-84d0-4aa7-9c1a-4d9edcd48feb\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1ef026ef0d3a9ed9437da9b2d6b1a80a19cc4383f0da115e2f1842ad9d53c285\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 42807, \"end_char_idx\": 44245, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "346575</td>\n      <td>0.277100</td>\n      <td>0.554800</td>\n      <td>0.252900</td>\n      <td>0.148400</td>\n      <td>0.239700</td>\n      <td>0.523600</td>\n      <td>0.278400</td>\n      <td>0.420000</td>\n      <td>0.443300</td>\n      <td>0.256300</td>\n      <td>0.424000</td>\n      <td>0.705600</td>\n      <td>0.529600</td>\n      <td>0.647300</td>\n      <td>0.273900</td>\n      <td>0.439200</td>\n      <td>0.174300</td>\n      <td>0.348700</td>\n      <td>0.157600</td>\n      <td>0.386200</td>\n      <td>0.250100</td>\n      <td>0.395100</td>\n    </tr>\n    <tr>\n      <td>30</td>\n      <td>0.993700</td>\n      <td>1.346446</td>\n      <td>0.277400</td>\n      <td>0.554700</td>\n      <td>0.252700</td>\n      <td>0.147900</td>\n      <td>0.240800</td>\n      <td>0.523600</td>\n      <td>0.278800</td>\n      <td>0.420400</td>\n      <td>0.443300</td>\n      <td>0.256100</td>\n      <td>0.424200</td>\n      <td>0.705500</td>\n      <td>0.530100</td>\n      <td>0.646800</td>\n      <td>0.275600</td>\n      <td>0.440500</td>\n      <td>0.174500</td>\n      <td>0.348700</td>\n      <td>0.157300</td>\n      <td>0.386200</td>\n      <td>0.249200</td>\n      <td>0.394200</td>\n    </tr>\n  </tbody>\n</table><p>\n\nIf you have set `push_to_hub` to `True` in the `training_args`, the training checkpoints are pushed to the\nHugging Face Hub. Upon training completion, push the final model to the Hub as well by calling the [`~transformers.Trainer.push_to_hub`] method."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 359312d0-84d0-4aa7-9c1a-4d9edcd48feb
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"359312d0-84d0-4aa7-9c1a-4d9edcd48feb\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"023c0bb3-cd82-4f4d-97fc-6c27c50bab7e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"294c0852c60673f20a356f1c2b0c66012ff79f920b63c9bfdb007db94b486adb\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"aafa32be-6fbe-4dee-9490-75572aac1a0b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"fef76698782cd924dfa6d083953b9935fd0dc238a9ff8043046ec52647f20817\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 43263, \"end_char_idx\": 45324, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "157600</td>\n      <td>0.386200</td>\n      <td>0.250100</td>\n      <td>0.395100</td>\n    </tr>\n    <tr>\n      <td>30</td>\n      <td>0.993700</td>\n      <td>1.346446</td>\n      <td>0.277400</td>\n      <td>0.554700</td>\n      <td>0.252700</td>\n      <td>0.147900</td>\n      <td>0.240800</td>\n      <td>0.523600</td>\n      <td>0.278800</td>\n      <td>0.420400</td>\n      <td>0.443300</td>\n      <td>0.256100</td>\n      <td>0.424200</td>\n      <td>0.705500</td>\n      <td>0.530100</td>\n      <td>0.646800</td>\n      <td>0.275600</td>\n      <td>0.440500</td>\n      <td>0.174500</td>\n      <td>0.348700</td>\n      <td>0.157300</td>\n      <td>0.386200</td>\n      <td>0.249200</td>\n      <td>0.394200</td>\n    </tr>\n  </tbody>\n</table><p>\n\nIf you have set `push_to_hub` to `True` in the `training_args`, the training checkpoints are pushed to the\nHugging Face Hub. Upon training completion, push the final model to the Hub as well by calling the [`~transformers.Trainer.push_to_hub`] method.\n\n```py\n>>> trainer.push_to_hub()\n```\n\n## Evaluate\n\n```py\n>>> from pprint import pprint\n\n>>> metrics = trainer.evaluate(eval_dataset=cppe5[\"test\"], metric_key_prefix=\"test\")\n>>> pprint(metrics)\n{'epoch': 30.0,\n  'test_loss': 1.0877351760864258,\n  'test_map': 0.4116,\n  'test_map_50': 0.741,\n  'test_map_75': 0.3663,\n  'test_map_Coverall': 0.5937,\n  'test_map_Face_Shield': 0.5863,\n  'test_map_Gloves': 0.3416,\n  'test_map_Goggles': 0.1468,\n  'test_map_Mask': 0.3894,\n  'test_map_large': 0.5637,\n  'test_map_medium': 0.3257,\n  'test_map_small': 0.3589,\n  'test_mar_1': 0.323,\n  'test_mar_10': 0.5237,\n  'test_mar_100': 0.5587,\n  'test_mar_100_Coverall': 0.6756,\n  'test_mar_100_Face_Shield': 0.7294,\n  'test_mar_100_Gloves': 0.4721,\n  'test_mar_100_Goggles': 0.4125,\n  'test_mar_100_Mask': 0.5038,\n  'test_mar_large': 0.7283,\n  'test_mar_medium': 0.4901,\n  'test_mar_small': 0.4469,\n  'test_runtime': 1.6526,\n  'test_samples_per_second': 17.548,\n  'test_steps_per_second': 2.42}\n```\n\nThese results can be further improved by adjusting the hyperparameters in [`TrainingArguments`]."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: aafa32be-6fbe-4dee-9490-75572aac1a0b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/object_detection", "title": "Object detection", "tokens": 16536, "retrieve_doc": false, "source": "transformers", "_node_content": "{\"id_\": \"aafa32be-6fbe-4dee-9490-75572aac1a0b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"92f71fc2-01bd-41ac-a8c4-8549d870e56c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"d49b6ae8ff266cc10e090c1ef796434af536b844319b2f94a799eb913f650fc6\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"359312d0-84d0-4aa7-9c1a-4d9edcd48feb\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/object_detection\", \"title\": \"Object detection\", \"tokens\": 16536, \"retrieve_doc\": false, \"source\": \"transformers\"}, \"hash\": \"c01bfcedcc9708bb38549065936edfbcf3b16f93579ee0683ee90102e00dc845\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 45325, \"end_char_idx\": 47788, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "ref_doc_id": "92f71fc2-01bd-41ac-a8c4-8549d870e56c", "chroma:document": "Give it a go!\n\n## Inference\n\nNow that you have finetuned a model, evaluated it, and uploaded it to the Hugging Face Hub, you can use it for inference.\n\n```py\n>>> import torch\n>>> import requests\n\n>>> from PIL import Image, ImageDraw\n>>> from transformers import AutoImageProcessor, AutoModelForObjectDetection\n\n>>> url = \"https://images.pexels.com/photos/8413299/pexels-photo-8413299.jpeg?auto=compress&cs=tinysrgb&w=630&h=375&dpr=2\"\n>>> image = Image.open(requests.get(url, stream=True).raw)\n```\n\nLoad model and image processor from the Hugging Face Hub (skip to use already trained in this session):\n```py\n>>> device = \"cuda\"\n>>> model_repo = \"qubvel-hf/detr_finetuned_cppe5\"\n\n>>> image_processor = AutoImageProcessor.from_pretrained(model_repo)\n>>> model = AutoModelForObjectDetection.from_pretrained(model_repo)\n>>> model = model.to(device)\n```\n\nAnd detect bounding boxes:\n\n```py\n\n>>> with torch.no_grad():\n...     inputs = image_processor(images=[image], return_tensors=\"pt\")\n...     outputs = model(**inputs.to(device))\n...     target_sizes = torch.tensor([[image.size[1], image.size[0]]])\n...     results = image_processor.post_process_object_detection(outputs, threshold=0.3, target_sizes=target_sizes)[0]\n\n>>> for score, label, box in zip(results[\"scores\"], results[\"labels\"], results[\"boxes\"]):\n...     box = [round(i, 2) for i in box.tolist()]\n...     print(\n...         f\"Detected {model.config.id2label[label.item()]} with confidence \"\n...         f\"{round(score.item(), 3)} at location {box}\"\n...     )\nDetected Gloves with confidence 0.683 at location [244.58, 124.33, 300.35, 185.13]\nDetected Mask with confidence 0.517 at location [143.73, 64.58, 219.57, 125.89]\nDetected Gloves with confidence 0.425 at location [179.15, 155.57, 262.4, 226.35]\nDetected Coverall with confidence 0.407 at location [307.13, -1.18, 477.82, 318.06]\nDetected Coverall with confidence 0.391 at location [68.61, 126.66, 309.03, 318.89]\n```\n\nLet's plot the result:\n\n```py\n>>> draw = ImageDraw.Draw(image)\n\n>>> for score, label, box in zip(results[\"scores\"], results[\"labels\"], results[\"boxes\"]):\n...     box = [round(i, 2) for i in box.tolist()]\n...     x, y, x2, y2 = tuple(box)\n...     draw.rectangle((x, y, x2, y2), outline=\"red\", width=1)\n...     draw.text((x, y), model.config.id2label[label.item()], fill=\"white\")\n\n>>> image\n```\n\n<div class=\"flex justify-center\">\n    <img src=\"https://i.imgur.com/oDUqD0K.png\" alt=\"Object detection result on a new image\"/>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 95550600-1002-41c2-b9b2-8648a666f3c6
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/token_classification", "title": "Token classification", "tokens": 4795, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"95550600-1002-41c2-b9b2-8648a666f3c6\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c2948b67-4ffd-439c-ac4d-692dea7f982f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2e558a5bdc01661668c830be29b792690caf3fba9f44f2ddca71e69822f565f3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"9ec0a174-f639-401d-ae27-2299312fe23f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7fad541e01f49fb3c14c95628ab35eaf9915dee7237b3c7b3e8f2e6bf913debd\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2505, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "ref_doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "chroma:document": "# Token classification\n\n[[open-in-colab]]\n\n<Youtube id=\"wVHdVlPScxA\"/>\n\nToken classification assigns a label to individual tokens in a sentence. One of the most common token classification tasks is Named Entity Recognition (NER). NER attempts to find a label for each entity in a sentence, such as a person, location, or organization.\n\nThis guide will show you how to:\n\n1. Finetune [DistilBERT](https://huggingface.co/distilbert/distilbert-base-uncased) on the [WNUT 17](https://huggingface.co/datasets/wnut_17) dataset to detect new entities.\n2. Use your finetuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/token-classification).\n\n</Tip>\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install transformers datasets evaluate seqeval\n```\n\nWe encourage you to login to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to login:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\n## Load WNUT 17 dataset\n\nStart by loading the WNUT 17 dataset from the \ud83e\udd17 Datasets library:\n\n```py\n>>> from datasets import load_dataset\n\n>>> wnut = load_dataset(\"wnut_17\")\n```\n\nThen take a look at an example:\n\n```py\n>>> wnut[\"train\"][0]\n{'id': '0',\n 'ner_tags': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 8, 8, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0],\n 'tokens': ['@paulwalk', 'It', \"'s\", 'the', 'view', 'from', 'where', 'I', \"'m\", 'living', 'for', 'two', 'weeks', '.', 'Empire', 'State', 'Building', '=', 'ESB', '.', 'Pretty', 'bad', 'storm', 'here', 'last', 'evening', '.']\n}\n```\n\nEach number in `ner_tags` represents an entity. Convert the numbers to their label names to find out what the entities are:\n\n```py\n>>> label_list = wnut[\"train\"].features[f\"ner_tags\"].feature.names\n>>> label_list\n[\n    \"O\",\n    \"B-corporation\",\n    \"I-corporation\",\n    \"B-creative-work\",\n    \"I-creative-work\",\n    \"B-group\",\n    \"I-group\",\n    \"B-location\",\n    \"I-location\",\n    \"B-person\",\n    \"I-person\",\n    \"B-product\",\n    \"I-product\",\n]\n```\n\nThe letter that prefixes each `ner_tag` indicates the token position of the entity:\n\n- `B-` indicates the beginning of an entity.\n- `I-` indicates a token is contained inside the same entity (for example, the `State` token is a part of an entity like\n  `Empire State Building`).\n- `0` indicates the token doesn't correspond to any entity."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 9ec0a174-f639-401d-ae27-2299312fe23f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/token_classification", "title": "Token classification", "tokens": 4795, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"9ec0a174-f639-401d-ae27-2299312fe23f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c2948b67-4ffd-439c-ac4d-692dea7f982f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2e558a5bdc01661668c830be29b792690caf3fba9f44f2ddca71e69822f565f3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"95550600-1002-41c2-b9b2-8648a666f3c6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"4ce66b9f8a5303f783742c32261bd29bd6a7778e5aad92469187666fa05c7771\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"99bd8111-352b-4b38-937f-40dc370cd960\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e24a3259e936aa3c545d1ab2fdad9f602f66c385b824ec4ef84154749c1c20b4\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1600, \"end_char_idx\": 4255, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "ref_doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "chroma:document": "', 'Empire', 'State', 'Building', '=', 'ESB', '.', 'Pretty', 'bad', 'storm', 'here', 'last', 'evening', '.']\n}\n```\n\nEach number in `ner_tags` represents an entity. Convert the numbers to their label names to find out what the entities are:\n\n```py\n>>> label_list = wnut[\"train\"].features[f\"ner_tags\"].feature.names\n>>> label_list\n[\n    \"O\",\n    \"B-corporation\",\n    \"I-corporation\",\n    \"B-creative-work\",\n    \"I-creative-work\",\n    \"B-group\",\n    \"I-group\",\n    \"B-location\",\n    \"I-location\",\n    \"B-person\",\n    \"I-person\",\n    \"B-product\",\n    \"I-product\",\n]\n```\n\nThe letter that prefixes each `ner_tag` indicates the token position of the entity:\n\n- `B-` indicates the beginning of an entity.\n- `I-` indicates a token is contained inside the same entity (for example, the `State` token is a part of an entity like\n  `Empire State Building`).\n- `0` indicates the token doesn't correspond to any entity.\n\n## Preprocess\n\n<Youtube id=\"iY2AZYdZAr0\"/>\n\nThe next step is to load a DistilBERT tokenizer to preprocess the `tokens` field:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"distilbert/distilbert-base-uncased\")\n```\n\nAs you saw in the example `tokens` field above, it looks like the input has already been tokenized. But the input actually hasn't been tokenized yet and you'll need to set `is_split_into_words=True` to tokenize the words into subwords. For example:\n\n```py\n>>> example = wnut[\"train\"][0]\n>>> tokenized_input = tokenizer(example[\"tokens\"], is_split_into_words=True)\n>>> tokens = tokenizer.convert_ids_to_tokens(tokenized_input[\"input_ids\"])\n>>> tokens\n['[CLS]', '@', 'paul', '##walk', 'it', \"'\", 's', 'the', 'view', 'from', 'where', 'i', \"'\", 'm', 'living', 'for', 'two', 'weeks', '.', 'empire', 'state', 'building', '=', 'es', '##b', '.', 'pretty', 'bad', 'storm', 'here', 'last', 'evening', '.', '[SEP]']\n```\n\nHowever, this adds some special tokens `[CLS]` and `[SEP]` and the subword tokenization creates a mismatch between the input and labels. A single word corresponding to a single label may now be split into two subwords. You'll need to realign the tokens and labels by:\n\n1. Mapping all tokens to their corresponding word with the [`word_ids`](https://huggingface.co/docs/transformers/main_classes/tokenizer#transformers.BatchEncoding.word_ids) method.\n2. Assigning the label `-100` to the special tokens `[CLS]` and `[SEP]` so they're ignored by the PyTorch loss function (see [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)).\n3. Only labeling the first token of a given word. Assign `-100` to other subtokens from the same word."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 99bd8111-352b-4b38-937f-40dc370cd960
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/token_classification", "title": "Token classification", "tokens": 4795, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"99bd8111-352b-4b38-937f-40dc370cd960\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c2948b67-4ffd-439c-ac4d-692dea7f982f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2e558a5bdc01661668c830be29b792690caf3fba9f44f2ddca71e69822f565f3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9ec0a174-f639-401d-ae27-2299312fe23f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"27c61547823b6dade56a4b83b332b2e84f6e9837ecaa4036bc851501359a1b35\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ed57a8d2-145c-432f-8702-a8a2f36f4f80\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8b40dc6404aa43958b99d6dba236aab721b1780867b9fbaff0852f08e6ba03de\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2873, \"end_char_idx\": 5883, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "ref_doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "chroma:document": "But the input actually hasn't been tokenized yet and you'll need to set `is_split_into_words=True` to tokenize the words into subwords. For example:\n\n```py\n>>> example = wnut[\"train\"][0]\n>>> tokenized_input = tokenizer(example[\"tokens\"], is_split_into_words=True)\n>>> tokens = tokenizer.convert_ids_to_tokens(tokenized_input[\"input_ids\"])\n>>> tokens\n['[CLS]', '@', 'paul', '##walk', 'it', \"'\", 's', 'the', 'view', 'from', 'where', 'i', \"'\", 'm', 'living', 'for', 'two', 'weeks', '.', 'empire', 'state', 'building', '=', 'es', '##b', '.', 'pretty', 'bad', 'storm', 'here', 'last', 'evening', '.', '[SEP]']\n```\n\nHowever, this adds some special tokens `[CLS]` and `[SEP]` and the subword tokenization creates a mismatch between the input and labels. A single word corresponding to a single label may now be split into two subwords. You'll need to realign the tokens and labels by:\n\n1. Mapping all tokens to their corresponding word with the [`word_ids`](https://huggingface.co/docs/transformers/main_classes/tokenizer#transformers.BatchEncoding.word_ids) method.\n2. Assigning the label `-100` to the special tokens `[CLS]` and `[SEP]` so they're ignored by the PyTorch loss function (see [CrossEntropyLoss](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)).\n3. Only labeling the first token of a given word. Assign `-100` to other subtokens from the same word.\n\nHere is how you can create a function to realign the tokens and labels, and truncate sequences to be no longer than DistilBERT's maximum input length:\n\n```py\n>>> def tokenize_and_align_labels(examples):\n...     tokenized_inputs = tokenizer(examples[\"tokens\"], truncation=True, is_split_into_words=True)\n\n...     labels = []\n...     for i, label in enumerate(examples[f\"ner_tags\"]):\n...         word_ids = tokenized_inputs.word_ids(batch_index=i)  # Map tokens to their respective word.\n...         previous_word_idx = None\n...         label_ids = []\n...         for word_idx in word_ids:  # Set the special tokens to -100.\n...             if word_idx is None:\n...                 label_ids.append(-100)\n...             elif word_idx != previous_word_idx:  # Only label the first token of a given word.\n...                 label_ids.append(label[word_idx])\n...             else:\n...                 label_ids.append(-100)\n...             previous_word_idx = word_idx\n...         labels.append(label_ids)\n\n...     tokenized_inputs[\"labels\"] = labels\n...     return tokenized_inputs\n```\n\nTo apply the preprocessing function over the entire dataset, use \ud83e\udd17 Datasets [`~datasets.Dataset.map`] function. You can speed up the `map` function by setting `batched=True` to process multiple elements of the dataset at once:\n\n```py\n>>> tokenized_wnut = wnut.map(tokenize_and_align_labels, batched=True)\n```\n\nNow create a batch of examples using [`DataCollatorWithPadding`]. It's more efficient to *dynamically pad* the sentences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: ed57a8d2-145c-432f-8702-a8a2f36f4f80
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/token_classification", "title": "Token classification", "tokens": 4795, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"ed57a8d2-145c-432f-8702-a8a2f36f4f80\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c2948b67-4ffd-439c-ac4d-692dea7f982f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2e558a5bdc01661668c830be29b792690caf3fba9f44f2ddca71e69822f565f3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"99bd8111-352b-4b38-937f-40dc370cd960\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"ae7a759a0092c922744e5f5a3cd4cf4bb140daef03781c949842b6117c94eef4\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a667984a-dd94-4e7c-9297-3988057e3409\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c48f8e218afc02521019a541ca338e311e5413d985d6935bb41b53308401feca\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4153, \"end_char_idx\": 7140, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "ref_doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "chroma:document": "3. Only labeling the first token of a given word. Assign `-100` to other subtokens from the same word.\n\nHere is how you can create a function to realign the tokens and labels, and truncate sequences to be no longer than DistilBERT's maximum input length:\n\n```py\n>>> def tokenize_and_align_labels(examples):\n...     tokenized_inputs = tokenizer(examples[\"tokens\"], truncation=True, is_split_into_words=True)\n\n...     labels = []\n...     for i, label in enumerate(examples[f\"ner_tags\"]):\n...         word_ids = tokenized_inputs.word_ids(batch_index=i)  # Map tokens to their respective word.\n...         previous_word_idx = None\n...         label_ids = []\n...         for word_idx in word_ids:  # Set the special tokens to -100.\n...             if word_idx is None:\n...                 label_ids.append(-100)\n...             elif word_idx != previous_word_idx:  # Only label the first token of a given word.\n...                 label_ids.append(label[word_idx])\n...             else:\n...                 label_ids.append(-100)\n...             previous_word_idx = word_idx\n...         labels.append(label_ids)\n\n...     tokenized_inputs[\"labels\"] = labels\n...     return tokenized_inputs\n```\n\nTo apply the preprocessing function over the entire dataset, use \ud83e\udd17 Datasets [`~datasets.Dataset.map`] function. You can speed up the `map` function by setting `batched=True` to process multiple elements of the dataset at once:\n\n```py\n>>> tokenized_wnut = wnut.map(tokenize_and_align_labels, batched=True)\n```\n\nNow create a batch of examples using [`DataCollatorWithPadding`]. It's more efficient to *dynamically pad* the sentences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.\n\n<frameworkcontent>\n<pt>\n```py\n>>> from transformers import DataCollatorForTokenClassification\n\n>>> data_collator = DataCollatorForTokenClassification(tokenizer=tokenizer)\n```\n</pt>\n<tf>\n```py\n>>> from transformers import DataCollatorForTokenClassification\n\n>>> data_collator = DataCollatorForTokenClassification(tokenizer=tokenizer, return_tensors=\"tf\")\n```\n</tf>\n</frameworkcontent>\n\n## Evaluate\n\nIncluding a metric during training is often helpful for evaluating your model's performance. You can quickly load a evaluation method with the \ud83e\udd17 [Evaluate](https://huggingface.co/docs/evaluate/index) library. For this task, load the [seqeval](https://huggingface.co/spaces/evaluate-metric/seqeval) framework (see the \ud83e\udd17 Evaluate [quick tour](https://huggingface.co/docs/evaluate/a_quick_tour) to learn more about how to load and compute a metric). Seqeval actually produces several scores: precision, recall, F1, and accuracy.\n\n```py\n>>> import evaluate\n\n>>> seqeval = evaluate.load(\"seqeval\")\n```\n\nGet the NER labels first, and then create a function that passes your true predictions and true labels to [`~evaluate.EvaluationModule.compute`] to calculate the scores:\n\n```py\n>>> import numpy as np\n\n>>> labels = [label_list[i] for i in example[f\"ner_tags\"]]"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a667984a-dd94-4e7c-9297-3988057e3409
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/token_classification", "title": "Token classification", "tokens": 4795, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a667984a-dd94-4e7c-9297-3988057e3409\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c2948b67-4ffd-439c-ac4d-692dea7f982f\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2e558a5bdc01661668c830be29b792690caf3fba9f44f2ddca71e69822f565f3\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ed57a8d2-145c-432f-8702-a8a2f36f4f80\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/token_classification\", \"title\": \"Token classification\", \"tokens\": 4795, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"725395a4524e9155d3cd49543a87e29c340894bbf88935f95cc92383a96dca3a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"5972b63b-df02-4e10-b7fb-56bc5e764256\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"2785ed4cda887f25cbbfdbf7f11d1469fc8fb6cbc35dc7fd9c5fb56868f7f0ff\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5454, \"end_char_idx\": 8034, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "ref_doc_id": "c2948b67-4ffd-439c-ac4d-692dea7f982f", "chroma:document": "You can speed up the `map` function by setting `batched=True` to process multiple elements of the dataset at once:\n\n```py\n>>> tokenized_wnut = wnut.map(tokenize_and_align_labels, batched=True)\n```\n\nNow create a batch of examples using [`DataCollatorWithPadding`]. It's more efficient to *dynamically pad* the sentences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.\n\n<frameworkcontent>\n<pt>\n```py\n>>> from transformers import DataCollatorForTokenClassification\n\n>>> data_collator = DataCollatorForTokenClassification(tokenizer=tokenizer)\n```\n</pt>\n<tf>\n```py\n>>> from transformers import DataCollatorForTokenClassification\n\n>>> data_collator = DataCollatorForTokenClassification(tokenizer=tokenizer, return_tensors=\"tf\")\n```\n</tf>\n</frameworkcontent>\n\n## Evaluate\n\nIncluding a metric during training is often helpful for evaluating your model's performance. You can quickly load a evaluation method with the \ud83e\udd17 [Evaluate](https://huggingface.co/docs/evaluate/index) library. For this task, load the [seqeval](https://huggingface.co/spaces/evaluate-metric/seqeval) framework (see the \ud83e\udd17 Evaluate [quick tour](https://huggingface.co/docs/evaluate/a_quick_tour) to learn more about how to load and compute a metric). Seqeval actually produces several scores: precision, recall, F1, and accuracy.\n\n```py\n>>> import evaluate\n\n>>> seqeval = evaluate.load(\"seqeval\")\n```\n\nGet the NER labels first, and then create a function that passes your true predictions and true labels to [`~evaluate.EvaluationModule.compute`] to calculate the scores:\n\n```py\n>>> import numpy as np\n\n>>> labels = [label_list[i] for i in example[f\"ner_tags\"]]\n\n\n>>> def compute_metrics(p):\n...     predictions, labels = p\n...     predictions = np.argmax(predictions, axis=2)\n\n...     true_predictions = [\n...         [label_list[p] for (p, l) in zip(prediction, label) if l != -100]\n...         for prediction, label in zip(predictions, labels)\n...     ]\n...     true_labels = [\n...         [label_list[l] for (p, l) in zip(prediction, label) if l != -100]\n...         for prediction, label in zip(predictions, labels)\n...     ]\n\n...     results = seqeval.compute(predictions=true_predictions, references=true_labels)\n...     return {\n...         \"precision\": results[\"overall_precision\"],\n...         \"recall\": results[\"overall_recall\"],\n...         \"f1\": results[\"overall_f1\"],\n...         \"accuracy\": results[\"overall_accuracy\"],\n...     }\n```\n\nYour `compute_metrics` function is ready to go now, and you'll return to it when you setup your training."}


