# embeddings_queue

created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 0a1435b5-3b2b-4f13-95ec-97169865710c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_train_gpu_one", "title": "Methods and tools for efficient training on a single GPU", "tokens": 7716, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"0a1435b5-3b2b-4f13-95ec-97169865710c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7a488fe5-6358-4f8d-8e94-daa390cebae2\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"f4a1199f6734143a23fa61a74b0292b4dc919543135f61bdce1e193e695c4196\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b3d8bd32-c159-4016-b59b-1130b708a9e6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"65f9d35fca1713302e6bd6ddb84dba498398f6b6189b88891f1b507275c884bb\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"71804a69-acfe-4253-86fd-3e202e037705\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d1685d545f4b828a3ebabcad8413e0c9f89e6f169ba9d67a2e692d9127d8ef81\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 30525, \"end_char_idx\": 33971, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "doc_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "ref_doc_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "chroma:document": "Go into the release's notes for the desired release, check that the environment's \ncomponents are matching your needs (including NVIDIA Driver requirements!) and then at the very top of that document go \nto the corresponding NGC page. If for some reason you get lost, here is [the index of all PyTorch NGC images](https://ngc.nvidia.com/catalog/containers/nvidia:pytorch).\n\nNext follow the instructions to download and deploy the docker image.\n\n## Mixture of Experts\n\nSome recent papers reported a 4-5x training speedup and a faster inference by integrating\nMixture of Experts (MoE) into the Transformer models.\n\nSince it has been discovered that more parameters lead to better performance, this technique allows to increase the \nnumber of parameters by an order of magnitude without increasing training costs.\n\nIn this approach every other FFN layer is replaced with a MoE Layer which consists of many experts, with a gated function \nthat trains each expert in a balanced way depending on the input token's position in a sequence.\n\n![MoE Transformer 2x block](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/perf-moe-transformer.png)\n\n(source: [GLAM](https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html))\n\nYou can find exhaustive details and comparison tables in the papers listed at the end of this section.\n\nThe main drawback of this approach is that it requires staggering amounts of GPU memory - almost an order of magnitude \nlarger than its dense equivalent. Various distillation and approaches are proposed to how to overcome the much higher memory requirements.\n\nThere is direct trade-off though, you can use just a few experts with a 2-3x smaller base model instead of dozens or \nhundreds experts leading to a 5x smaller model and thus increase the training speed moderately while increasing the \nmemory requirements moderately as well.\n\nMost related papers and implementations are built around Tensorflow/TPUs:\n\n- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://arxiv.org/abs/2006.16668)\n- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://arxiv.org/abs/2101.03961)\n- [GLaM: Generalist Language Model (GLaM)](https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html)\n\nAnd for Pytorch DeepSpeed has built one as well: [DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://arxiv.org/abs/2201.05596), [Mixture of Experts](https://www.deepspeed.ai/tutorials/mixture-of-experts/) - blog posts:  [1](https://www.microsoft.com/en-us/research/blog/deepspeed-powers-8x-larger-moe-model-training-with-high-performance/), [2](https://www.microsoft.com/en-us/research/publication/scalable-and-efficient-moe-training-for-multitask-multilingual-models/) and specific deployment with large transformer-based natural language generation models: [blog post](https://www.deepspeed.ai/2021/12/09/deepspeed-moe-nlg.html), [Megatron-Deepspeed branch](https://github.com/microsoft/Megatron-DeepSpeed/tree/moe-training).\n\n## Using PyTorch native attention and Flash Attention\n\nPyTorch's [`torch.nn.functional.scaled_dot_product_attention`](https://pytorch.org/docs/master/generated/torch.nn.functional.scaled_dot_product_attention.html) (SDPA) can also call FlashAttention and memory-efficient attention kernels under the hood."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 71804a69-acfe-4253-86fd-3e202e037705
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_train_gpu_one", "title": "Methods and tools for efficient training on a single GPU", "tokens": 7716, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"71804a69-acfe-4253-86fd-3e202e037705\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7a488fe5-6358-4f8d-8e94-daa390cebae2\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"f4a1199f6734143a23fa61a74b0292b4dc919543135f61bdce1e193e695c4196\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"0a1435b5-3b2b-4f13-95ec-97169865710c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_gpu_one\", \"title\": \"Methods and tools for efficient training on a single GPU\", \"tokens\": 7716, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e23b78d849f4c72b46dde6c31648b5a4d0dcba81f6cd4a57a715c238a255f785\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 32429, \"end_char_idx\": 34460, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "doc_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "ref_doc_id": "7a488fe5-6358-4f8d-8e94-daa390cebae2", "chroma:document": "Most related papers and implementations are built around Tensorflow/TPUs:\n\n- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://arxiv.org/abs/2006.16668)\n- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://arxiv.org/abs/2101.03961)\n- [GLaM: Generalist Language Model (GLaM)](https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html)\n\nAnd for Pytorch DeepSpeed has built one as well: [DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://arxiv.org/abs/2201.05596), [Mixture of Experts](https://www.deepspeed.ai/tutorials/mixture-of-experts/) - blog posts:  [1](https://www.microsoft.com/en-us/research/blog/deepspeed-powers-8x-larger-moe-model-training-with-high-performance/), [2](https://www.microsoft.com/en-us/research/publication/scalable-and-efficient-moe-training-for-multitask-multilingual-models/) and specific deployment with large transformer-based natural language generation models: [blog post](https://www.deepspeed.ai/2021/12/09/deepspeed-moe-nlg.html), [Megatron-Deepspeed branch](https://github.com/microsoft/Megatron-DeepSpeed/tree/moe-training).\n\n## Using PyTorch native attention and Flash Attention\n\nPyTorch's [`torch.nn.functional.scaled_dot_product_attention`](https://pytorch.org/docs/master/generated/torch.nn.functional.scaled_dot_product_attention.html) (SDPA) can also call FlashAttention and memory-efficient attention kernels under the hood. SDPA support is currently being added natively in Transformers and is used by default for `torch>=2.1.1` when an implementation is available. Please refer to [PyTorch scaled dot product attention](https://huggingface.co/docs/transformers/perf_infer_gpu_one#pytorch-scaled-dot-product-attention) for a list of supported models and more details.\n\nCheck out this [blogpost](https://pytorch.org/blog/out-of-the-box-acceleration/) to learn more about acceleration and memory-savings with SDPA."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4c487acb-43b7-474e-b583-f4df5272a70b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4c487acb-43b7-474e-b583-f4df5272a70b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"41a65d5d-221a-4bc0-afd1-e7d9beb06214\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"eaf39928c104f616403f50a4ec03f81bd2d1ff8b54de892267a1798b2f08099e\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3287, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "<!---\nCopyright 2022 The HuggingFace Team. All rights reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n\u26a0\ufe0f Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be\nrendered properly in your Markdown viewer.\n\n-->\n\n# Installation\n\nInstall \ud83e\udd17 Transformers for whichever deep learning library you're working with, setup your cache, and optionally configure \ud83e\udd17 Transformers to run offline.\n\n\ud83e\udd17 Transformers is tested on Python 3.6+, PyTorch 1.1.0+, TensorFlow 2.0+, and Flax. Follow the installation instructions below for the deep learning library you are using:\n\n* [PyTorch](https://pytorch.org/get-started/locally/) installation instructions.\n* [TensorFlow 2.0](https://www.tensorflow.org/install/pip) installation instructions.\n* [Flax](https://flax.readthedocs.io/en/latest/) installation instructions.\n\n## Install with pip\n\nYou should install \ud83e\udd17 Transformers in a [virtual environment](https://docs.python.org/3/library/venv.html). If you're unfamiliar with Python virtual environments, take a look at this [guide](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/). A virtual environment makes it easier to manage different projects, and avoid compatibility issues between dependencies.\n\nStart by creating a virtual environment in your project directory:\n\n```bash\npython -m venv .env\n```\n\nActivate the virtual environment. On Linux and MacOs:\n\n```bash\nsource .env/bin/activate\n```\nActivate Virtual environment on Windows\n\n```bash\n.env/Scripts/activate\n```\n\nNow you're ready to install \ud83e\udd17 Transformers with the following command:\n\n```bash\npip install transformers\n```\n\nFor CPU-support only, you can conveniently install \ud83e\udd17 Transformers and a deep learning library in one line. For example, install \ud83e\udd17 Transformers and PyTorch with:\n\n```bash\npip install 'transformers[torch]'\n```\n\n\ud83e\udd17 Transformers and TensorFlow 2.0:\n\n```bash\npip install 'transformers[tf-cpu]'\n```\n\n<Tip warning={true}>\n\nM1 / ARM Users\n\nYou will need to install the following before installing TensorFLow 2.0\n```bash\nbrew install cmake\nbrew install pkg-config\n```\n\n</Tip>\n\n\ud83e\udd17 Transformers and Flax:\n\n```bash\npip install 'transformers[flax]'\n```\n\nFinally, check if \ud83e\udd17 Transformers has been properly installed by running the following command. It will download a pretrained model:\n\n```bash\npython -c \"from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))\"\n```\n\nThen print out the label and score:\n\n```bash\n[{'label': 'POSITIVE', 'score': 0.9998704791069031}]\n```\n\n## Install from source\n\nInstall \ud83e\udd17 Transformers from source with the following command:\n\n```bash\npip install git+https://github.com/huggingface/transformers\n```\n\nThis command installs the bleeding edge `main` version rather than the latest `stable` version."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 41a65d5d-221a-4bc0-afd1-e7d9beb06214
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"41a65d5d-221a-4bc0-afd1-e7d9beb06214\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4c487acb-43b7-474e-b583-f4df5272a70b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"edbeed3a858acf3816698e4bcbd4bbc950fcef1a4e31cc378113d8a2b0516739\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"5f8fbe85-37b0-499d-ac72-70c26a109a4b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b5d0d21984db14aa1802940e6d744bd6ef0f340de8005559300114e9e2b4ccb3\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1642, \"end_char_idx\": 4848, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "A virtual environment makes it easier to manage different projects, and avoid compatibility issues between dependencies.\n\nStart by creating a virtual environment in your project directory:\n\n```bash\npython -m venv .env\n```\n\nActivate the virtual environment. On Linux and MacOs:\n\n```bash\nsource .env/bin/activate\n```\nActivate Virtual environment on Windows\n\n```bash\n.env/Scripts/activate\n```\n\nNow you're ready to install \ud83e\udd17 Transformers with the following command:\n\n```bash\npip install transformers\n```\n\nFor CPU-support only, you can conveniently install \ud83e\udd17 Transformers and a deep learning library in one line. For example, install \ud83e\udd17 Transformers and PyTorch with:\n\n```bash\npip install 'transformers[torch]'\n```\n\n\ud83e\udd17 Transformers and TensorFlow 2.0:\n\n```bash\npip install 'transformers[tf-cpu]'\n```\n\n<Tip warning={true}>\n\nM1 / ARM Users\n\nYou will need to install the following before installing TensorFLow 2.0\n```bash\nbrew install cmake\nbrew install pkg-config\n```\n\n</Tip>\n\n\ud83e\udd17 Transformers and Flax:\n\n```bash\npip install 'transformers[flax]'\n```\n\nFinally, check if \ud83e\udd17 Transformers has been properly installed by running the following command. It will download a pretrained model:\n\n```bash\npython -c \"from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))\"\n```\n\nThen print out the label and score:\n\n```bash\n[{'label': 'POSITIVE', 'score': 0.9998704791069031}]\n```\n\n## Install from source\n\nInstall \ud83e\udd17 Transformers from source with the following command:\n\n```bash\npip install git+https://github.com/huggingface/transformers\n```\n\nThis command installs the bleeding edge `main` version rather than the latest `stable` version. The `main` version is useful for staying up-to-date with the latest developments. For instance, if a bug has been fixed since the last official release but a new release hasn't been rolled out yet. However, this means the `main` version may not always be stable. We strive to keep the `main` version operational, and most issues are usually resolved within a few hours or a day. If you run into a problem, please open an [Issue](https://github.com/huggingface/transformers/issues) so we can fix it even sooner!\n\nCheck if \ud83e\udd17 Transformers has been properly installed by running the following command:\n\n```bash\npython -c \"from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))\"\n```\n\n## Editable install\n\nYou will need an editable install if you'd like to:\n\n* Use the `main` version of the source code.\n* Contribute to \ud83e\udd17 Transformers and need to test changes in the code.\n\nClone the repository and install \ud83e\udd17 Transformers with the following commands:\n\n```bash\ngit clone https://github.com/huggingface/transformers.git\ncd transformers\npip install -e .\n```\n\nThese commands will link the folder you cloned the repository to and your Python library paths. Python will now look inside the folder you cloned to in addition to the normal library paths. For example, if your Python packages are typically installed in `~/anaconda3/envs/main/lib/python3.7/site-packages/`, Python will also search the folder you cloned to: `~/transformers/`.\n\n<Tip warning={true}>\n\nYou must keep the `transformers` folder if you want to keep using the library."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 5f8fbe85-37b0-499d-ac72-70c26a109a4b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"5f8fbe85-37b0-499d-ac72-70c26a109a4b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"41a65d5d-221a-4bc0-afd1-e7d9beb06214\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"49cd58cce254bbf930fc90689541a1fdc14bafdb9586051469ab4a2e7bf81bd5\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"385e018f-b641-4a2c-912a-50487c63344b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cbb13bdfa3d6f1604e84f17fb180aa9742a49094c04025c6eae711f93f8d4164\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3288, \"end_char_idx\": 6442, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "The `main` version is useful for staying up-to-date with the latest developments. For instance, if a bug has been fixed since the last official release but a new release hasn't been rolled out yet. However, this means the `main` version may not always be stable. We strive to keep the `main` version operational, and most issues are usually resolved within a few hours or a day. If you run into a problem, please open an [Issue](https://github.com/huggingface/transformers/issues) so we can fix it even sooner!\n\nCheck if \ud83e\udd17 Transformers has been properly installed by running the following command:\n\n```bash\npython -c \"from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))\"\n```\n\n## Editable install\n\nYou will need an editable install if you'd like to:\n\n* Use the `main` version of the source code.\n* Contribute to \ud83e\udd17 Transformers and need to test changes in the code.\n\nClone the repository and install \ud83e\udd17 Transformers with the following commands:\n\n```bash\ngit clone https://github.com/huggingface/transformers.git\ncd transformers\npip install -e .\n```\n\nThese commands will link the folder you cloned the repository to and your Python library paths. Python will now look inside the folder you cloned to in addition to the normal library paths. For example, if your Python packages are typically installed in `~/anaconda3/envs/main/lib/python3.7/site-packages/`, Python will also search the folder you cloned to: `~/transformers/`.\n\n<Tip warning={true}>\n\nYou must keep the `transformers` folder if you want to keep using the library.\n\n</Tip>\n\nNow you can easily update your clone to the latest version of \ud83e\udd17 Transformers with the following command:\n\n```bash\ncd ~/transformers/\ngit pull\n```\n\nYour Python environment will find the `main` version of \ud83e\udd17 Transformers on the next run.\n\n## Install with conda\n\nInstall from the conda channel `conda-forge`:\n\n```bash\nconda install conda-forge::transformers\n```\n\n## Cache setup\n\nPretrained models are downloaded and locally cached at: `~/.cache/huggingface/hub`. This is the default directory given by the shell environment variable `TRANSFORMERS_CACHE`. On Windows, the default directory is given by `C:\\Users\\username\\.cache\\huggingface\\hub`. You can change the shell environment variables shown below - in order of priority - to specify a different cache directory:\n\n1. Shell environment variable (default): `HUGGINGFACE_HUB_CACHE` or `TRANSFORMERS_CACHE`.\n2. Shell environment variable: `HF_HOME`.\n3. Shell environment variable: `XDG_CACHE_HOME` + `/huggingface`.\n\n<Tip>\n\n\ud83e\udd17 Transformers will use the shell environment variables `PYTORCH_TRANSFORMERS_CACHE` or `PYTORCH_PRETRAINED_BERT_CACHE` if you are coming from an earlier iteration of this library and have set those environment variables, unless you specify the shell environment variable `TRANSFORMERS_CACHE`.\n\n</Tip>\n\n## Offline mode\n\nRun \ud83e\udd17 Transformers in a firewalled or offline environment with locally cached files by setting the environment variable `HF_HUB_OFFLINE=1`.\n\n<Tip>\n\nAdd [\ud83e\udd17 Datasets](https://huggingface.co/docs/datasets/) to your offline training workflow with the environment variable `HF_DATASETS_OFFLINE=1`."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 385e018f-b641-4a2c-912a-50487c63344b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"385e018f-b641-4a2c-912a-50487c63344b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"5f8fbe85-37b0-499d-ac72-70c26a109a4b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e86ca2a92d70d424ce72416abdcee943a039b0f26dc74b4f8593a7c184fe53b9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3595dd2e-1321-40ab-b197-57e0ce445693\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3a564744caca7f9bdad5df12c335ba36f51fa3013d4c26b05c2167990d7e2dc5\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4850, \"end_char_idx\": 7996, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "</Tip>\n\nNow you can easily update your clone to the latest version of \ud83e\udd17 Transformers with the following command:\n\n```bash\ncd ~/transformers/\ngit pull\n```\n\nYour Python environment will find the `main` version of \ud83e\udd17 Transformers on the next run.\n\n## Install with conda\n\nInstall from the conda channel `conda-forge`:\n\n```bash\nconda install conda-forge::transformers\n```\n\n## Cache setup\n\nPretrained models are downloaded and locally cached at: `~/.cache/huggingface/hub`. This is the default directory given by the shell environment variable `TRANSFORMERS_CACHE`. On Windows, the default directory is given by `C:\\Users\\username\\.cache\\huggingface\\hub`. You can change the shell environment variables shown below - in order of priority - to specify a different cache directory:\n\n1. Shell environment variable (default): `HUGGINGFACE_HUB_CACHE` or `TRANSFORMERS_CACHE`.\n2. Shell environment variable: `HF_HOME`.\n3. Shell environment variable: `XDG_CACHE_HOME` + `/huggingface`.\n\n<Tip>\n\n\ud83e\udd17 Transformers will use the shell environment variables `PYTORCH_TRANSFORMERS_CACHE` or `PYTORCH_PRETRAINED_BERT_CACHE` if you are coming from an earlier iteration of this library and have set those environment variables, unless you specify the shell environment variable `TRANSFORMERS_CACHE`.\n\n</Tip>\n\n## Offline mode\n\nRun \ud83e\udd17 Transformers in a firewalled or offline environment with locally cached files by setting the environment variable `HF_HUB_OFFLINE=1`.\n\n<Tip>\n\nAdd [\ud83e\udd17 Datasets](https://huggingface.co/docs/datasets/) to your offline training workflow with the environment variable `HF_DATASETS_OFFLINE=1`.\n\n</Tip>\n\n```bash\nHF_DATASETS_OFFLINE=1 HF_HUB_OFFLINE=1 \\\npython examples/pytorch/translation/run_translation.py --model_name_or_path google-t5/t5-small --dataset_name wmt16 --dataset_config ro-en ...\n```\n\nThis script should run without hanging or waiting to timeout because it won't attempt to download the model from the Hub.\n\nYou can also bypass loading a model from the Hub from each [`~PreTrainedModel.from_pretrained`] call with the [`local_files_only`] parameter. When set to `True`, only local files are loaded:\n\n```py\nfrom transformers import T5Model\n\nmodel = T5Model.from_pretrained(\"./path/to/local/directory\", local_files_only=True)\n```\n\n### Fetch models and tokenizers to use offline\n\nAnother option for using \ud83e\udd17 Transformers offline is to download the files ahead of time, and then point to their local path when you need to use them offline. There are three ways to do this:\n\n* Download a file through the user interface on the [Model Hub](https://huggingface.co/models) by clicking on the \u2193 icon.\n\n    ![download-icon](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/download-icon.png)\n\n* Use the [`PreTrainedModel.from_pretrained`] and [`PreTrainedModel.save_pretrained`] workflow:\n\n    1. Download your files ahead of time with [`PreTrainedModel.from_pretrained`]:\n\n    ```py\n    >>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n\n    >>> tokenizer = AutoTokenizer.from_pretrained(\"bigscience/T0_3B\")\n    >>> model = AutoModelForSeq2SeqLM.from_pretrained(\"bigscience/T0_3B\")\n    ```\n\n    2."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 3595dd2e-1321-40ab-b197-57e0ce445693
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"3595dd2e-1321-40ab-b197-57e0ce445693\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"385e018f-b641-4a2c-912a-50487c63344b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"2c921bc83867404130a6f4a1d72c6144c5310c9e81ba762ae24fc13b42c79202\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"295a2226-a041-4587-a946-1c2847340965\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6777ec7304a04bb36cc1a727105977ae48e32e9dc0a90fa456c3835e1f28a445\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6444, \"end_char_idx\": 8960, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "</Tip>\n\n```bash\nHF_DATASETS_OFFLINE=1 HF_HUB_OFFLINE=1 \\\npython examples/pytorch/translation/run_translation.py --model_name_or_path google-t5/t5-small --dataset_name wmt16 --dataset_config ro-en ...\n```\n\nThis script should run without hanging or waiting to timeout because it won't attempt to download the model from the Hub.\n\nYou can also bypass loading a model from the Hub from each [`~PreTrainedModel.from_pretrained`] call with the [`local_files_only`] parameter. When set to `True`, only local files are loaded:\n\n```py\nfrom transformers import T5Model\n\nmodel = T5Model.from_pretrained(\"./path/to/local/directory\", local_files_only=True)\n```\n\n### Fetch models and tokenizers to use offline\n\nAnother option for using \ud83e\udd17 Transformers offline is to download the files ahead of time, and then point to their local path when you need to use them offline. There are three ways to do this:\n\n* Download a file through the user interface on the [Model Hub](https://huggingface.co/models) by clicking on the \u2193 icon.\n\n    ![download-icon](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/download-icon.png)\n\n* Use the [`PreTrainedModel.from_pretrained`] and [`PreTrainedModel.save_pretrained`] workflow:\n\n    1. Download your files ahead of time with [`PreTrainedModel.from_pretrained`]:\n\n    ```py\n    >>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n\n    >>> tokenizer = AutoTokenizer.from_pretrained(\"bigscience/T0_3B\")\n    >>> model = AutoModelForSeq2SeqLM.from_pretrained(\"bigscience/T0_3B\")\n    ```\n\n    2. Save your files to a specified directory with [`PreTrainedModel.save_pretrained`]:\n\n    ```py\n    >>> tokenizer.save_pretrained(\"./your/path/bigscience_t0\")\n    >>> model.save_pretrained(\"./your/path/bigscience_t0\")\n    ```\n\n    3. Now when you're offline, reload your files with [`PreTrainedModel.from_pretrained`] from the specified directory:\n\n    ```py\n    >>> tokenizer = AutoTokenizer.from_pretrained(\"./your/path/bigscience_t0\")\n    >>> model = AutoModel.from_pretrained(\"./your/path/bigscience_t0\")\n    ```\n\n* Programmatically download files with the [huggingface_hub](https://github.com/huggingface/huggingface_hub/tree/main/src/huggingface_hub) library:\n\n    1. Install the `huggingface_hub` library in your virtual environment:\n\n    ```bash\n    python -m pip install huggingface_hub\n    ```\n\n    2. Use the [`hf_hub_download`](https://huggingface.co/docs/hub/adding-a-library#download-files-from-the-hub) function to download a file to a specific path."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 295a2226-a041-4587-a946-1c2847340965
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/installation", "title": "Installation", "tokens": 2277, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"295a2226-a041-4587-a946-1c2847340965\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"29527b16-a87f-45dd-a0b9-19d294e6d997\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfef8db584ba695f4cebba362b20f6cab428ef5e064641c2aa9d0695f768953\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3595dd2e-1321-40ab-b197-57e0ce445693\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/installation\", \"title\": \"Installation\", \"tokens\": 2277, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"a56826f2469bf7b74763375b6a766647a543c36607c0c7ba7b353a508e039c43\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7460, \"end_char_idx\": 9702, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "ref_doc_id": "29527b16-a87f-45dd-a0b9-19d294e6d997", "chroma:document": "![download-icon](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/download-icon.png)\n\n* Use the [`PreTrainedModel.from_pretrained`] and [`PreTrainedModel.save_pretrained`] workflow:\n\n    1. Download your files ahead of time with [`PreTrainedModel.from_pretrained`]:\n\n    ```py\n    >>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM\n\n    >>> tokenizer = AutoTokenizer.from_pretrained(\"bigscience/T0_3B\")\n    >>> model = AutoModelForSeq2SeqLM.from_pretrained(\"bigscience/T0_3B\")\n    ```\n\n    2. Save your files to a specified directory with [`PreTrainedModel.save_pretrained`]:\n\n    ```py\n    >>> tokenizer.save_pretrained(\"./your/path/bigscience_t0\")\n    >>> model.save_pretrained(\"./your/path/bigscience_t0\")\n    ```\n\n    3. Now when you're offline, reload your files with [`PreTrainedModel.from_pretrained`] from the specified directory:\n\n    ```py\n    >>> tokenizer = AutoTokenizer.from_pretrained(\"./your/path/bigscience_t0\")\n    >>> model = AutoModel.from_pretrained(\"./your/path/bigscience_t0\")\n    ```\n\n* Programmatically download files with the [huggingface_hub](https://github.com/huggingface/huggingface_hub/tree/main/src/huggingface_hub) library:\n\n    1. Install the `huggingface_hub` library in your virtual environment:\n\n    ```bash\n    python -m pip install huggingface_hub\n    ```\n\n    2. Use the [`hf_hub_download`](https://huggingface.co/docs/hub/adding-a-library#download-files-from-the-hub) function to download a file to a specific path. For example, the following command downloads the `config.json` file from the [T0](https://huggingface.co/bigscience/T0_3B) model to your desired path:\n\n    ```py\n    >>> from huggingface_hub import hf_hub_download\n\n    >>> hf_hub_download(repo_id=\"bigscience/T0_3B\", filename=\"config.json\", cache_dir=\"./your/path/bigscience_t0\")\n    ```\n\nOnce your file is downloaded and locally cached, specify it's local path to load and use it:\n\n```py\n>>> from transformers import AutoConfig\n\n>>> config = AutoConfig.from_pretrained(\"./your/path/bigscience_t0/config.json\")\n```\n\n<Tip>\n\nSee the [How to download files from the Hub](https://huggingface.co/docs/hub/how-to-downstream) section for more details on downloading files stored on the Hub.\n\n</Tip>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a016a651-cda5-45fb-8aa1-7a74e073c37a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a016a651-cda5-45fb-8aa1-7a74e073c37a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"debcb94a-d336-47e3-8e83-d45923ffbecd\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cae151ad16374d59543be6e232b84e8c332430214e522f972eef428170d42e81\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3417, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "<!---\nCopyright 2020 The HuggingFace Team. All rights reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n\u26a0\ufe0f Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be\nrendered properly in your Markdown viewer.\n\n-->\n\n# Checks on a Pull Request\n\nWhen you open a pull request on \ud83e\udd17 Transformers, a fair number of checks will be run to make sure the patch you are adding is not breaking anything existing. Those checks are of four types:\n- regular tests\n- documentation build\n- code and documentation style\n- general repository consistency\n\nIn this document, we will take a stab at explaining what those various checks are and the reason behind them, as well as how to debug them locally if one of them fails on your PR.\n\nNote that, ideally, they require you to have a dev install:\n\n```bash\npip install transformers[dev]\n```\n\nor for an editable install:\n\n```bash\npip install -e .[dev]\n```\n\ninside the Transformers repo. Since the number of optional dependencies of Transformers has grown a lot, it's possible you don't manage to get all of them. If the dev install fails, make sure to install the Deep Learning framework you are working with (PyTorch, TensorFlow and/or Flax) then do\n\n```bash\npip install transformers[quality]\n```\n\nor for an editable install:\n\n```bash\npip install -e .[quality]\n```\n\n\n## Tests\n\nAll the jobs that begin with `ci/circleci: run_tests_` run parts of the Transformers testing suite. Each of those jobs focuses on a part of the library in a certain environment: for instance `ci/circleci: run_tests_pipelines_tf` runs the pipelines test in an environment where TensorFlow only is installed.\n\nNote that to avoid running tests when there is no real change in the modules they are testing, only part of the test suite is run each time: a utility is run to determine the differences in the library between before and after the PR (what GitHub shows you in the \"Files changes\" tab) and picks the tests impacted by that diff. That utility can be run locally with:\n\n```bash\npython utils/tests_fetcher.py\n```\n\nfrom the root of the Transformers repo. It will:\n\n1. Check for each file in the diff if the changes are in the code or only in comments or docstrings. Only the files with real code changes are kept.\n2. Build an internal map that gives for each file of the source code of the library all the files it recursively impacts. Module A is said to impact module B if module B imports module A. For the recursive impact, we need a chain of modules going from module A to module B in which each module imports the previous one.\n3. Apply this map on the files gathered in step 1, which  gives us the list of model files impacted by the PR.\n4. Map each of those files to their corresponding test file(s) and get the list of tests to run.\n\nWhen executing the script locally, you should get the results of step 1, 3 and 4 printed and thus know which tests are run."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: debcb94a-d336-47e3-8e83-d45923ffbecd
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"debcb94a-d336-47e3-8e83-d45923ffbecd\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a016a651-cda5-45fb-8aa1-7a74e073c37a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"f138789cd51e560264017a8f806beed1d2f3f2a36f388b03ab28636e9bfb6b68\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a90648a8-976e-4762-aad3-dc8bffe1e27f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"16a310c00d2fef1fe9a44664b28695bae9b1556ce3af39ee3954139a115f73b2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1838, \"end_char_idx\": 5082, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "## Tests\n\nAll the jobs that begin with `ci/circleci: run_tests_` run parts of the Transformers testing suite. Each of those jobs focuses on a part of the library in a certain environment: for instance `ci/circleci: run_tests_pipelines_tf` runs the pipelines test in an environment where TensorFlow only is installed.\n\nNote that to avoid running tests when there is no real change in the modules they are testing, only part of the test suite is run each time: a utility is run to determine the differences in the library between before and after the PR (what GitHub shows you in the \"Files changes\" tab) and picks the tests impacted by that diff. That utility can be run locally with:\n\n```bash\npython utils/tests_fetcher.py\n```\n\nfrom the root of the Transformers repo. It will:\n\n1. Check for each file in the diff if the changes are in the code or only in comments or docstrings. Only the files with real code changes are kept.\n2. Build an internal map that gives for each file of the source code of the library all the files it recursively impacts. Module A is said to impact module B if module B imports module A. For the recursive impact, we need a chain of modules going from module A to module B in which each module imports the previous one.\n3. Apply this map on the files gathered in step 1, which  gives us the list of model files impacted by the PR.\n4. Map each of those files to their corresponding test file(s) and get the list of tests to run.\n\nWhen executing the script locally, you should get the results of step 1, 3 and 4 printed and thus know which tests are run. The script will also create a file named `test_list.txt` which contains the list of tests to run, and you can run them locally with the following command:\n\n```bash\npython -m pytest -n 8 --dist=loadfile -rA -s $(cat test_list.txt)\n```\n\nJust in case anything slipped through the cracks, the full test suite is also run daily.\n\n## Documentation build\n\nThe `build_pr_documentation` job builds and generates a preview of the documentation to make sure everything looks okay once your PR is merged. A bot will add a link to preview the documentation in your PR. Any changes you make to the PR are automatically updated in the preview. If the documentation fails to build, click on **Details** next to the failed job to see where things went wrong. Often, the error is as simple as a missing file in the `toctree`.\n\nIf you're interested in building or previewing the documentation locally, take a look at the [`README.md`](https://github.com/huggingface/transformers/tree/main/docs) in the docs folder.\n\n## Code and documentation style\n\nCode formatting is applied to all the source files, the examples and the tests using `black` and `ruff`. We also have a custom tool taking care of the formatting of docstrings and `rst` files (`utils/style_doc.py`), as well as the order of the lazy imports performed in the Transformers `__init__.py` files (`utils/custom_init_isort.py`). All of this can be launched by executing\n\n```bash\nmake style\n```\n\nThe CI checks those have been applied inside the `ci/circleci: check_code_quality` check. It also runs `ruff`, that will have a basic look at your code and will complain if it finds an undefined variable, or one that is not used."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a90648a8-976e-4762-aad3-dc8bffe1e27f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a90648a8-976e-4762-aad3-dc8bffe1e27f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"debcb94a-d336-47e3-8e83-d45923ffbecd\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7c4263641f20018567083d22e48b4d3e836cac8cf7dabf8917f46a3f2d228784\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4c5a5c52-0dba-401c-8a1a-e59c5bfd8a63\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"de20bfcf29b1ccbde76088ab3b3afc431cf2d082830fb92581db07adf31ce4ac\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3418, \"end_char_idx\": 5585, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "The script will also create a file named `test_list.txt` which contains the list of tests to run, and you can run them locally with the following command:\n\n```bash\npython -m pytest -n 8 --dist=loadfile -rA -s $(cat test_list.txt)\n```\n\nJust in case anything slipped through the cracks, the full test suite is also run daily.\n\n## Documentation build\n\nThe `build_pr_documentation` job builds and generates a preview of the documentation to make sure everything looks okay once your PR is merged. A bot will add a link to preview the documentation in your PR. Any changes you make to the PR are automatically updated in the preview. If the documentation fails to build, click on **Details** next to the failed job to see where things went wrong. Often, the error is as simple as a missing file in the `toctree`.\n\nIf you're interested in building or previewing the documentation locally, take a look at the [`README.md`](https://github.com/huggingface/transformers/tree/main/docs) in the docs folder.\n\n## Code and documentation style\n\nCode formatting is applied to all the source files, the examples and the tests using `black` and `ruff`. We also have a custom tool taking care of the formatting of docstrings and `rst` files (`utils/style_doc.py`), as well as the order of the lazy imports performed in the Transformers `__init__.py` files (`utils/custom_init_isort.py`). All of this can be launched by executing\n\n```bash\nmake style\n```\n\nThe CI checks those have been applied inside the `ci/circleci: check_code_quality` check. It also runs `ruff`, that will have a basic look at your code and will complain if it finds an undefined variable, or one that is not used. To run that check locally, use\n\n```bash\nmake quality\n```\n\nThis can take a lot of time, so to run the same thing on only the files you modified in the current branch, run\n\n```bash\nmake fixup\n```\n\nThis last command will also run all the additional checks for the repository consistency. Let's have a look at them.\n\n## Repository consistency\n\nThis regroups all the tests to make sure your PR leaves the repository in a good state, and is performed by the `ci/circleci: check_repository_consistency` check."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4c5a5c52-0dba-401c-8a1a-e59c5bfd8a63
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4c5a5c52-0dba-401c-8a1a-e59c5bfd8a63\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a90648a8-976e-4762-aad3-dc8bffe1e27f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b45b903e5ed4df7e2cc133e89661d3c260dfba8545e838b382fb947309dfe679\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b13ab574-596e-4220-9a39-06acb5cb795f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c2305b09c0845a1cd234086ef51fd1574f2924d76b81af7b56eed2572f606667\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3911, \"end_char_idx\": 7747, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "A bot will add a link to preview the documentation in your PR. Any changes you make to the PR are automatically updated in the preview. If the documentation fails to build, click on **Details** next to the failed job to see where things went wrong. Often, the error is as simple as a missing file in the `toctree`.\n\nIf you're interested in building or previewing the documentation locally, take a look at the [`README.md`](https://github.com/huggingface/transformers/tree/main/docs) in the docs folder.\n\n## Code and documentation style\n\nCode formatting is applied to all the source files, the examples and the tests using `black` and `ruff`. We also have a custom tool taking care of the formatting of docstrings and `rst` files (`utils/style_doc.py`), as well as the order of the lazy imports performed in the Transformers `__init__.py` files (`utils/custom_init_isort.py`). All of this can be launched by executing\n\n```bash\nmake style\n```\n\nThe CI checks those have been applied inside the `ci/circleci: check_code_quality` check. It also runs `ruff`, that will have a basic look at your code and will complain if it finds an undefined variable, or one that is not used. To run that check locally, use\n\n```bash\nmake quality\n```\n\nThis can take a lot of time, so to run the same thing on only the files you modified in the current branch, run\n\n```bash\nmake fixup\n```\n\nThis last command will also run all the additional checks for the repository consistency. Let's have a look at them.\n\n## Repository consistency\n\nThis regroups all the tests to make sure your PR leaves the repository in a good state, and is performed by the `ci/circleci: check_repository_consistency` check. You can locally run that check by executing the following:\n\n```bash\nmake repo-consistency\n```\n\nThis checks that:\n\n- All objects added to the init are documented (performed by `utils/check_repo.py`)\n- All `__init__.py` files have the same content in their two sections (performed by `utils/check_inits.py`)\n- All code identified as a copy from another module is consistent with the original (performed by `utils/check_copies.py`)\n- All configuration classes have at least one valid checkpoint mentioned in their docstrings (performed by `utils/check_config_docstrings.py`)\n- All configuration classes only contain attributes that are used in corresponding modeling files (performed by `utils/check_config_attributes.py`)\n- The translations of the READMEs and the index of the doc have the same model list as the main README (performed by `utils/check_copies.py`)\n- The auto-generated tables in the documentation are up to date (performed by `utils/check_table.py`)\n- The library has all objects available even if not all optional dependencies are installed (performed by `utils/check_dummies.py`)\n- All docstrings properly document the arguments in the signature of the object (performed by `utils/check_docstrings.py`)\n\nShould this check fail, the first two items require manual fixing, the last four can be fixed automatically for you by running the command\n\n```bash\nmake fix-copies\n```\n\nAdditional checks concern PRs that add new models, mainly that:\n\n- All models added are in an Auto-mapping (performed by `utils/check_repo.py`)\n<!-- TODO Sylvain, add a check that makes sure the common tests are implemented.-->\n- All models are properly tested (performed by `utils/check_repo.py`)\n\n<!-- TODO Sylvain, add the following\n- All models are added to the main README, inside the main doc\n- All checkpoints used actually exist on the Hub\n\n-->\n\n### Check copies\n\nSince the Transformers library is very opinionated with respect to model code, and each model should fully be implemented in a single file without relying on other models, we have added a mechanism that checks whether a copy of the code of a layer of a given model stays consistent with the original."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b13ab574-596e-4220-9a39-06acb5cb795f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"b13ab574-596e-4220-9a39-06acb5cb795f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4c5a5c52-0dba-401c-8a1a-e59c5bfd8a63\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"0a046faad9bc1a3e3e960d35da49e86ada96a632fcd555e1da78feb6f6563544\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ade268f6-b879-4225-b253-a40314600d89\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f94a0cf9b286bf13b56756048eb29e027b83b10654d866954da7a4f982981675\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7748, \"end_char_idx\": 10116, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "This way, when there is a bug fix, we can see all other impacted models and choose to trickle down the modification or break the copy.\n\n<Tip>\n\nIf a file is a full copy of another file, you should register it in the constant `FULL_COPIES` of `utils/check_copies.py`.\n\n</Tip>\n\nThis mechanism relies on comments of the form `# Copied from xxx`. The `xxx` should contain the whole path to the class of function which is being copied below. For instance, `RobertaSelfOutput` is a direct copy of the `BertSelfOutput` class, so you can see [here](https://github.com/huggingface/transformers/blob/2bd7a27a671fd1d98059124024f580f8f5c0f3b5/src/transformers/models/roberta/modeling_roberta.py#L289) it has a comment:\n\n```py\n# Copied from transformers.models.bert.modeling_bert.BertSelfOutput\n```\n\nNote that instead of applying this to a whole class, you can apply it to the relevant methods that are copied from. For instance [here](https://github.com/huggingface/transformers/blob/2bd7a27a671fd1d98059124024f580f8f5c0f3b5/src/transformers/models/roberta/modeling_roberta.py#L598) you can see how `RobertaPreTrainedModel._init_weights` is copied from the same method in `BertPreTrainedModel` with the comment:\n\n```py\n# Copied from transformers.models.bert.modeling_bert.BertPreTrainedModel._init_weights\n```\n\nSometimes the copy is exactly the same except for names: for instance in `RobertaAttention`, we use `RobertaSelfAttention` insted of `BertSelfAttention` but other than that, the code is exactly the same. This is why `# Copied from` supports simple string replacements with the following syntax: `Copied from xxx with foo->bar`. This means the code is copied with all instances of `foo` being replaced by `bar`. You can see how it used [here](https://github.com/huggingface/transformers/blob/2bd7a27a671fd1d98059124024f580f8f5c0f3b5/src/transformers/models/roberta/modeling_roberta.py#L304C1-L304C86) in `RobertaAttention` with the comment:\n\n```py\n# Copied from transformers.models.bert.modeling_bert.BertAttention with Bert->Roberta\n```\n\nNote that there shouldn't be any spaces around the arrow (unless that space is part of the pattern to replace of course).\n\nYou can add several patterns separated by a comma. For instance here `CamemberForMaskedLM` is a direct copy of `RobertaForMaskedLM` with two replacements: `Roberta` to `Camembert` and `ROBERTA` to `CAMEMBERT`."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: ade268f6-b879-4225-b253-a40314600d89
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/pr_checks", "title": "Checks on a Pull Request", "tokens": 2764, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"ade268f6-b879-4225-b253-a40314600d89\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"aca48b8e-ca54-47c9-9c64-9d021702ffa7\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b214e07ef3ecae09c5461993de9fccbc5447a3c7ea0d44c0ffacb76327cf9123\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b13ab574-596e-4220-9a39-06acb5cb795f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/pr_checks\", \"title\": \"Checks on a Pull Request\", \"tokens\": 2764, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"f9cde03352f5688a4271baa6cf177ee13193507e88d141ccd356cb4ee4c20ed6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9250, \"end_char_idx\": 11612, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "ref_doc_id": "aca48b8e-ca54-47c9-9c64-9d021702ffa7", "chroma:document": "This is why `# Copied from` supports simple string replacements with the following syntax: `Copied from xxx with foo->bar`. This means the code is copied with all instances of `foo` being replaced by `bar`. You can see how it used [here](https://github.com/huggingface/transformers/blob/2bd7a27a671fd1d98059124024f580f8f5c0f3b5/src/transformers/models/roberta/modeling_roberta.py#L304C1-L304C86) in `RobertaAttention` with the comment:\n\n```py\n# Copied from transformers.models.bert.modeling_bert.BertAttention with Bert->Roberta\n```\n\nNote that there shouldn't be any spaces around the arrow (unless that space is part of the pattern to replace of course).\n\nYou can add several patterns separated by a comma. For instance here `CamemberForMaskedLM` is a direct copy of `RobertaForMaskedLM` with two replacements: `Roberta` to `Camembert` and `ROBERTA` to `CAMEMBERT`. You can see [here](https://github.com/huggingface/transformers/blob/15082a9dc6950ecae63a0d3e5060b2fc7f15050a/src/transformers/models/camembert/modeling_camembert.py#L929) this is done with the comment:\n\n```py\n# Copied from transformers.models.roberta.modeling_roberta.RobertaForMaskedLM with Roberta->Camembert, ROBERTA->CAMEMBERT\n```\n\nIf the order matters (because one of the replacements might conflict with a previous one), the replacements are executed from left to right.\n\n<Tip>\n\nIf the replacements change the formatting (if you replace a short name by a very long name for instance), the copy is checked after applying the auto-formatter.\n\n</Tip>\n\nAnother way when the patterns are just different casings of the same replacement (with an uppercased and a lowercased variants) is just to add the option `all-casing`. [Here](https://github.com/huggingface/transformers/blob/15082a9dc6950ecae63a0d3e5060b2fc7f15050a/src/transformers/models/mobilebert/modeling_mobilebert.py#L1237) is an example in `MobileBertForSequenceClassification` with the comment:\n\n```py\n# Copied from transformers.models.bert.modeling_bert.BertForSequenceClassification with Bert->MobileBert all-casing\n```\n\nIn this case, the code is copied from `BertForSequenceClassification` by replacing:\n- `Bert` by `MobileBert` (for instance when using `MobileBertModel` in the init)\n- `bert` by `mobilebert` (for instance when defining `self.mobilebert`)\n- `BERT` by `MOBILEBERT` (in the constant `MOBILEBERT_INPUTS_DOCSTRING`)"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d4360482-dbc1-4daa-96d0-73f4e16be64e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tflite", "title": "Export to TFLite", "tokens": 763, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"d4360482-dbc1-4daa-96d0-73f4e16be64e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tflite\", \"title\": \"Export to TFLite\", \"tokens\": 763, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"0fe7386a-782c-40a2-90eb-f615aba0b9e4\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tflite\", \"title\": \"Export to TFLite\", \"tokens\": 763, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"0130f2681f69d6ce6f652cbe10af6485aa43a241bb98b616df984578248e01fd\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2358, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "0fe7386a-782c-40a2-90eb-f615aba0b9e4", "doc_id": "0fe7386a-782c-40a2-90eb-f615aba0b9e4", "ref_doc_id": "0fe7386a-782c-40a2-90eb-f615aba0b9e4", "chroma:document": "# Export to TFLite\n\n[TensorFlow Lite](https://www.tensorflow.org/lite/guide) is a lightweight framework for deploying machine learning models \non resource-constrained devices, such as mobile phones, embedded systems, and Internet of Things (IoT) devices. \nTFLite is designed to optimize and run models efficiently on these devices with limited computational power, memory, and \npower consumption.\nA TensorFlow Lite model is represented in a special efficient portable format identified by the `.tflite` file extension. \n\n\ud83e\udd17 Optimum offers functionality to export \ud83e\udd17 Transformers models to TFLite through the `exporters.tflite` module. \nFor the list of supported model architectures, please refer to [\ud83e\udd17 Optimum documentation](https://huggingface.co/docs/optimum/exporters/tflite/overview).\n\nTo export a model to TFLite, install the required dependencies:\n \n```bash\npip install optimum[exporters-tf]\n```\n\nTo check out all available arguments, refer to the [\ud83e\udd17 Optimum docs](https://huggingface.co/docs/optimum/main/en/exporters/tflite/usage_guides/export_a_model), \nor view help in command line:\n\n```bash\noptimum-cli export tflite --help\n```\n\nTo export a model's checkpoint from the \ud83e\udd17 Hub, for example, `google-bert/bert-base-uncased`, run the following command:\n\n```bash\noptimum-cli export tflite --model google-bert/bert-base-uncased --sequence_length 128 bert_tflite/\n```\n\nYou should see the logs indicating progress and showing where the resulting `model.tflite` is saved, like this:\n\n```bash\nValidating TFLite model...\n\t-[\u2713] TFLite model output names match reference model (logits)\n\t- Validating TFLite Model output \"logits\":\n\t\t-[\u2713] (1, 128, 30522) matches (1, 128, 30522)\n\t\t-[x] values not close enough, max diff: 5.817413330078125e-05 (atol: 1e-05)\nThe TensorFlow Lite export succeeded with the warning: The maximum absolute difference between the output of the reference model and the TFLite exported model is not within the set tolerance 1e-05:\n- logits: max diff = 5.817413330078125e-05.\n The exported model was saved at: bert_tflite\n ```\n\nThe example above illustrates exporting a checkpoint from \ud83e\udd17 Hub. When exporting a local model, first make sure that you \nsaved both the model's weights and tokenizer files in the same directory (`local_path`). When using CLI, pass the \n`local_path` to the `model` argument instead of the checkpoint name on \ud83e\udd17 Hub."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a386d6fd-ddbd-4198-84eb-3b9524118e13
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a386d6fd-ddbd-4198-84eb-3b9524118e13\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a7f7f9cd-b7cb-4838-be24-ed6642209d51\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"aa61503410636d6cdc5e54d4f157ad12b345e897dc6150a6fb5978cb9be0970c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 1766, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "# Optimize inference using torch.compile()\n\nThis guide aims to provide a benchmark on the inference speed-ups introduced with [`torch.compile()`](https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html)\u00a0for [computer vision models in \ud83e\udd17 Transformers](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers&sort=trending).\n\n## Benefits of torch.compile\n   \nDepending on the model and the GPU, `torch.compile()` yields up to 30% speed-up during inference. To use `torch.compile()`, simply install any version of `torch` above 2.0. \n\nCompiling a model takes time, so it's useful if you are compiling the model only once instead of every time you infer.\nTo compile any computer vision model of your choice, call `torch.compile()` on the model as shown below:\n\n```diff\nfrom transformers import AutoModelForImageClassification\n\nmodel = AutoModelForImageClassification.from_pretrained(MODEL_ID).to(\"cuda\")\n+ model = torch.compile(model)\n```\n\n`compile()`\u00a0comes with multiple modes for compiling, which essentially differ in compilation time and inference overhead. `max-autotune`\u00a0takes longer than `reduce-overhead`\u00a0but results in faster inference. Default mode is fastest for compilation but is not as efficient compared to `reduce-overhead` for inference time. In this guide, we used the default mode. You can learn more about it [here](https://pytorch.org/get-started/pytorch-2.0/#user-experience).\n\nWe benchmarked `torch.compile` with different computer vision models, tasks, types of hardware, and batch sizes on `torch`\u00a0version 2.0.1.\n\n## Benchmarking code \n\nBelow you can find the benchmarking code for each task. We warm up the GPU before inference and take the mean time of 300 inferences, using the same image each time."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a7f7f9cd-b7cb-4838-be24-ed6642209d51
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a7f7f9cd-b7cb-4838-be24-ed6642209d51\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a386d6fd-ddbd-4198-84eb-3b9524118e13\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"6b2bfce5800241e07ffca2be95ced5487ac286f129e0a891bbe7105cd797842c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"fc75c8d3-efea-4c75-a389-e4305c0c0698\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"40e9ed06e3df9126df16dc9108f1d49f0c958e748594f1fc91e8e7a63d802171\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 367, \"end_char_idx\": 3469, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "## Benefits of torch.compile\n   \nDepending on the model and the GPU, `torch.compile()` yields up to 30% speed-up during inference. To use `torch.compile()`, simply install any version of `torch` above 2.0. \n\nCompiling a model takes time, so it's useful if you are compiling the model only once instead of every time you infer.\nTo compile any computer vision model of your choice, call `torch.compile()` on the model as shown below:\n\n```diff\nfrom transformers import AutoModelForImageClassification\n\nmodel = AutoModelForImageClassification.from_pretrained(MODEL_ID).to(\"cuda\")\n+ model = torch.compile(model)\n```\n\n`compile()`\u00a0comes with multiple modes for compiling, which essentially differ in compilation time and inference overhead. `max-autotune`\u00a0takes longer than `reduce-overhead`\u00a0but results in faster inference. Default mode is fastest for compilation but is not as efficient compared to `reduce-overhead` for inference time. In this guide, we used the default mode. You can learn more about it [here](https://pytorch.org/get-started/pytorch-2.0/#user-experience).\n\nWe benchmarked `torch.compile` with different computer vision models, tasks, types of hardware, and batch sizes on `torch`\u00a0version 2.0.1.\n\n## Benchmarking code \n\nBelow you can find the benchmarking code for each task. We warm up the GPU before inference and take the mean time of 300 inferences, using the same image each time.\n\n### Image Classification with ViT\n\n```python \nimport torch\nfrom PIL import Image\nimport requests\nimport numpy as np\nfrom transformers import AutoImageProcessor, AutoModelForImageClassification\n\nurl = 'http://images.cocodataset.org/val2017/000000039769.jpg'\nimage = Image.open(requests.get(url, stream=True).raw)\n\nprocessor = AutoImageProcessor.from_pretrained(\"google/vit-base-patch16-224\")\nmodel = AutoModelForImageClassification.from_pretrained(\"google/vit-base-patch16-224\").to(\"cuda\")\nmodel = torch.compile(model)\n\nprocessed_input = processor(image, return_tensors='pt').to(device=\"cuda\")\n\nwith torch.no_grad():\n    _ = model(**processed_input)\n\n```\n\n#### Object Detection with DETR\n\n```python \nfrom transformers import AutoImageProcessor, AutoModelForObjectDetection\n\nprocessor = AutoImageProcessor.from_pretrained(\"facebook/detr-resnet-50\")\nmodel = AutoModelForObjectDetection.from_pretrained(\"facebook/detr-resnet-50\").to(\"cuda\")\nmodel = torch.compile(model)\n\ntexts = [\"a photo of a cat\", \"a photo of a dog\"]\ninputs = processor(text=texts, images=image, return_tensors=\"pt\").to(\"cuda\")\n\nwith torch.no_grad():\n    _ = model(**inputs)\n```\n\n#### Image Segmentation with Segformer\n\n```python \nfrom transformers import SegformerImageProcessor, SegformerForSemanticSegmentation\n\nprocessor = SegformerImageProcessor.from_pretrained(\"nvidia/segformer-b0-finetuned-ade-512-512\")\nmodel = SegformerForSemanticSegmentation.from_pretrained(\"nvidia/segformer-b0-finetuned-ade-512-512\").to(\"cuda\")\nmodel = torch.compile(model)\nseg_inputs = processor(images=image, return_tensors=\"pt\").to(\"cuda\")\n\nwith torch.no_grad():\n    _ = model(**seg_inputs)\n```\n\nBelow you can find the list of the models we benchmarked."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: fc75c8d3-efea-4c75-a389-e4305c0c0698
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"fc75c8d3-efea-4c75-a389-e4305c0c0698\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a7f7f9cd-b7cb-4838-be24-ed6642209d51\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9b60c8f8b5140fb195dd09c5e2b0d5c823d1153760991b1f7e05fce2229f6685\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"1f325634-3def-462a-9e8d-b15cbbee2bec\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"5325ed3cddc9941ad793c36ebc7e01c0bb28b7a5a8c55c7040a129ef35e1f35f\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3471, \"end_char_idx\": 5599, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "**Image Classification** \n- [google/vit-base-patch16-224](https://huggingface.co/google/vit-base-patch16-224)\n- [microsoft/beit-base-patch16-224-pt22k-ft22k](https://huggingface.co/microsoft/beit-base-patch16-224-pt22k-ft22k)\n- [facebook/convnext-large-224](https://huggingface.co/facebook/convnext-large-224)\n- [microsoft/resnet-50](https://huggingface.co/)\n\n**Image Segmentation** \n- [nvidia/segformer-b0-finetuned-ade-512-512](https://huggingface.co/nvidia/segformer-b0-finetuned-ade-512-512)\n- [facebook/mask2former-swin-tiny-coco-panoptic](https://huggingface.co/facebook/mask2former-swin-tiny-coco-panoptic)\n- [facebook/maskformer-swin-base-ade](https://huggingface.co/facebook/maskformer-swin-base-ade)\n- [google/deeplabv3_mobilenet_v2_1.0_513](https://huggingface.co/google/deeplabv3_mobilenet_v2_1.0_513)\n\n**Object Detection** \n- [google/owlvit-base-patch32](https://huggingface.co/google/owlvit-base-patch32)\n- [facebook/detr-resnet-101](https://huggingface.co/facebook/detr-resnet-101)\n- [microsoft/conditional-detr-resnet-50](https://huggingface.co/microsoft/conditional-detr-resnet-50)\n\nBelow you can find visualization of inference durations with and without `torch.compile()`\u00a0and percentage improvements for each model in different hardware and batch sizes. \n\n<div class=\"flex\">\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/a100_batch_comp.png\" />\n  </div>\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/v100_batch_comp.png\" />\n  </div>\n   <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/t4_batch_comp.png\" />\n  </div>\n</div>\n\n<div class=\"flex\">\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/A100_1_duration.png\" />\n  </div>\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/A100_1_percentage.png\" />\n  </div>\n</div>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 1f325634-3def-462a-9e8d-b15cbbee2bec
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"1f325634-3def-462a-9e8d-b15cbbee2bec\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"fc75c8d3-efea-4c75-a389-e4305c0c0698\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c5c54806d8e679166b77d6a17d11a93742dc445d33e8e340f0e38c5740439090\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"eaf60e67-7777-4ce6-96cb-9871c4284983\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1ee584ad0f7e7e9e75ae09c5d5c4c2876552e2eef499c4dac771ead7c1bd8ed0\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4746, \"end_char_idx\": 7258, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "<div class=\"flex\">\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/a100_batch_comp.png\" />\n  </div>\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/v100_batch_comp.png\" />\n  </div>\n   <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/t4_batch_comp.png\" />\n  </div>\n</div>\n\n<div class=\"flex\">\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/A100_1_duration.png\" />\n  </div>\n  <div>\n    <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/A100_1_percentage.png\" />\n  </div>\n</div>\n\n\n![Duration Comparison on V100 with Batch Size of 1](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/v100_1_duration.png)\n\n![Percentage Improvement on T4 with Batch Size of 4](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/torch_compile/T4_4_percentage.png)\n\nBelow you can find inference durations in milliseconds for each model with and without `compile()`. Note that OwlViT results in OOM in larger batch sizes.\n\n### A100 (batch size: 1)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 9.325 | 7.584 | \n| Image Segmentation/Segformer | 11.759 | 10.500 |\n| Object Detection/OwlViT | 24.978 | 18.420 |\n| Image Classification/BeiT | 11.282 | 8.448 | \n| Object Detection/DETR | 34.619 | 19.040 |\n| Image Classification/ConvNeXT | 10.410 | 10.208 | \n| Image Classification/ResNet | 6.531 | 4.124 |\n| Image Segmentation/Mask2former | 60.188 | 49.117 |\n| Image Segmentation/Maskformer | 75.764 | 59.487 | \n| Image Segmentation/MobileNet | 8.583 | 3.974 |\n| Object Detection/Resnet-101 | 36.276 | 18.197 |\n| Object Detection/Conditional-DETR | 31.219 | 17.993 |\n\n\n### A100 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 14.832 | 14.499 | \n| Image Segmentation/Segformer | 18.838 | 16.476 |\n| Image Classification/BeiT | 13.205 | 13.048 | \n| Object Detection/DETR | 48.657 | 32.418|\n| Image Classification/ConvNeXT | 22.940 | 21.631 | \n| Image Classification/ResNet | 6.657 | 4."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: eaf60e67-7777-4ce6-96cb-9871c4284983
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"eaf60e67-7777-4ce6-96cb-9871c4284983\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"1f325634-3def-462a-9e8d-b15cbbee2bec\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9cb60cf8b653fd8d1c8761733383a314cfbebd9572337cdef9ad0238b6d1ef0e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"06fcc1d2-d2ce-4f40-93fe-2036f45321e2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d6b7857f59fc78b1b22c907248693b118eb20152ccf5e14f5bcd2463ebb449eb\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6844, \"end_char_idx\": 8889, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "### A100 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 14.832 | 14.499 | \n| Image Segmentation/Segformer | 18.838 | 16.476 |\n| Image Classification/BeiT | 13.205 | 13.048 | \n| Object Detection/DETR | 48.657 | 32.418|\n| Image Classification/ConvNeXT | 22.940 | 21.631 | \n| Image Classification/ResNet | 6.657 | 4.268 |\n| Image Segmentation/Mask2former | 74.277 | 61.781 |\n| Image Segmentation/Maskformer | 180.700 | 159.116 | \n| Image Segmentation/MobileNet | 14.174 | 8.515 |\n| Object Detection/Resnet-101 | 68.101 | 44.998 |\n| Object Detection/Conditional-DETR | 56.470 | 35.552 |\n\n### A100 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 40.944 | 40.010 | \n| Image Segmentation/Segformer | 37.005 | 31.144 |\n| Image Classification/BeiT | 41.854 | 41.048 | \n| Object Detection/DETR | 164.382 | 161.902 |\n| Image Classification/ConvNeXT | 82.258 | 75.561 | \n| Image Classification/ResNet | 7.018 | 5.024 |\n| Image Segmentation/Mask2former | 178.945 | 154.814 |\n| Image Segmentation/Maskformer | 638.570 | 579.826 | \n| Image Segmentation/MobileNet | 51.693 | 30.310 |\n| Object Detection/Resnet-101 | 232.887 | 155.021 |\n| Object Detection/Conditional-DETR | 180.491 | 124.032 |\n\n### V100 (batch size: 1)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 10.495 | 6.00 | \n| Image Segmentation/Segformer | 13.321 | 5.862 | \n| Object Detection/OwlViT | 25.769 | 22.395 | \n| Image Classification/BeiT | 11.347 | 7.234 | \n| Object Detection/DETR | 33.951 | 19.388 |\n| Image Classification/ConvNeXT | 11.623 | 10.412 | \n| Image Classification/ResNet | 6.484 | 3.820 |\n| Image Segmentation/Mask2former | 64.640 | 49.873 |\n| Image Segmentation/Maskformer | 95.532 | 72.207 | \n| Image Segmentation/MobileNet | 9.217 | 4.753 |\n| Object Detection/Resnet-101 | 52.818 | 28."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 06fcc1d2-d2ce-4f40-93fe-2036f45321e2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"06fcc1d2-d2ce-4f40-93fe-2036f45321e2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"eaf60e67-7777-4ce6-96cb-9871c4284983\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9bc75b87fba0fc87ebebe95ca8dad67c352c2f88be3e41e9c99c2ab1296b91b6\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"5cb3651b-b206-48b7-95f2-ff388662168f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"351268f5efc603dca17cf46b34f34824d971c976bcc080d2a098e9d731fe7789\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7836, \"end_char_idx\": 9911, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "382 | 161.902 |\n| Image Classification/ConvNeXT | 82.258 | 75.561 | \n| Image Classification/ResNet | 7.018 | 5.024 |\n| Image Segmentation/Mask2former | 178.945 | 154.814 |\n| Image Segmentation/Maskformer | 638.570 | 579.826 | \n| Image Segmentation/MobileNet | 51.693 | 30.310 |\n| Object Detection/Resnet-101 | 232.887 | 155.021 |\n| Object Detection/Conditional-DETR | 180.491 | 124.032 |\n\n### V100 (batch size: 1)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 10.495 | 6.00 | \n| Image Segmentation/Segformer | 13.321 | 5.862 | \n| Object Detection/OwlViT | 25.769 | 22.395 | \n| Image Classification/BeiT | 11.347 | 7.234 | \n| Object Detection/DETR | 33.951 | 19.388 |\n| Image Classification/ConvNeXT | 11.623 | 10.412 | \n| Image Classification/ResNet | 6.484 | 3.820 |\n| Image Segmentation/Mask2former | 64.640 | 49.873 |\n| Image Segmentation/Maskformer | 95.532 | 72.207 | \n| Image Segmentation/MobileNet | 9.217 | 4.753 |\n| Object Detection/Resnet-101 | 52.818 | 28.367 |\n| Object Detection/Conditional-DETR | 39.512 | 20.816 |\n\n### V100 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 15.181 | 14.501 | \n| Image Segmentation/Segformer | 16.787 | 16.188 |\n| Image Classification/BeiT | 15.171 | 14.753 | \n| Object Detection/DETR | 88.529 | 64.195 |\n| Image Classification/ConvNeXT | 29.574 | 27.085 | \n| Image Classification/ResNet | 6.109 | 4.731 |\n| Image Segmentation/Mask2former | 90.402 | 76.926 |\n| Image Segmentation/Maskformer | 234.261 | 205.456 | \n| Image Segmentation/MobileNet | 24.623 | 14.816 |\n| Object Detection/Resnet-101 | 134.672 | 101.304 |\n| Object Detection/Conditional-DETR | 97.464 | 69.739 |\n\n### V100 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 52.209 | 51.633 | \n| Image Segmentation/Segformer | 61.013 | 55.499 |\n| Image Classification/BeiT | 53.938 | 53."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 5cb3651b-b206-48b7-95f2-ff388662168f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"5cb3651b-b206-48b7-95f2-ff388662168f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"06fcc1d2-d2ce-4f40-93fe-2036f45321e2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e44a924cfbb9cb312a48b2333debaa0fbcfe9771080282d3e103c54d8a347d02\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a963ce14-40c5-4663-b388-5e75607d30de\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4f31c039172791eb9fb95cd3aeb5850d0eaeafde8a5db06dcac79c14a7b7b692\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8880, \"end_char_idx\": 10950, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "818 | 28.367 |\n| Object Detection/Conditional-DETR | 39.512 | 20.816 |\n\n### V100 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 15.181 | 14.501 | \n| Image Segmentation/Segformer | 16.787 | 16.188 |\n| Image Classification/BeiT | 15.171 | 14.753 | \n| Object Detection/DETR | 88.529 | 64.195 |\n| Image Classification/ConvNeXT | 29.574 | 27.085 | \n| Image Classification/ResNet | 6.109 | 4.731 |\n| Image Segmentation/Mask2former | 90.402 | 76.926 |\n| Image Segmentation/Maskformer | 234.261 | 205.456 | \n| Image Segmentation/MobileNet | 24.623 | 14.816 |\n| Object Detection/Resnet-101 | 134.672 | 101.304 |\n| Object Detection/Conditional-DETR | 97.464 | 69.739 |\n\n### V100 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 52.209 | 51.633 | \n| Image Segmentation/Segformer | 61.013 | 55.499 |\n| Image Classification/BeiT | 53.938 | 53.581  |\n| Object Detection/DETR | OOM | OOM |\n| Image Classification/ConvNeXT | 109.682 | 100.771 | \n| Image Classification/ResNet | 14.857 | 12.089 |\n| Image Segmentation/Mask2former | 249.605 | 222.801 |\n| Image Segmentation/Maskformer | 831.142 | 743.645 | \n| Image Segmentation/MobileNet | 93.129 | 55.365 |\n| Object Detection/Resnet-101 | 482.425 | 361.843 |\n| Object Detection/Conditional-DETR | 344.661 | 255.298 |\n\n### T4 (batch size: 1)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 16.520 | 15.786 | \n| Image Segmentation/Segformer | 16.116 | 14.205 |\n| Object Detection/OwlViT | 53.634 | 51.105 |\n| Image Classification/BeiT | 16.464 | 15.710 | \n| Object Detection/DETR | 73.100 | 53.99 |\n| Image Classification/ConvNeXT | 32.932 | 30.845 | \n| Image Classification/ResNet | 6.031 | 4.321 |\n| Image Segmentation/Mask2former | 79.192 | 66.815 |\n| Image Segmentation/Maskformer | 200.026 | 188.268 | \n| Image Segmentation/MobileNet | 18.908 | 11."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a963ce14-40c5-4663-b388-5e75607d30de
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a963ce14-40c5-4663-b388-5e75607d30de\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"5cb3651b-b206-48b7-95f2-ff388662168f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"788024aaef7037ccee1627bc56ec2a632baac90c4a0d0463db3061fd3223e9c9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d836023f-6a1e-4dc9-a5f3-eac07ce3149f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"a5e248a607ec8791392e67a56ac6aa4d5ecd033ff6543db0de1bdaa177e84087\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9863, \"end_char_idx\": 11975, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "499 |\n| Image Classification/BeiT | 53.938 | 53.581  |\n| Object Detection/DETR | OOM | OOM |\n| Image Classification/ConvNeXT | 109.682 | 100.771 | \n| Image Classification/ResNet | 14.857 | 12.089 |\n| Image Segmentation/Mask2former | 249.605 | 222.801 |\n| Image Segmentation/Maskformer | 831.142 | 743.645 | \n| Image Segmentation/MobileNet | 93.129 | 55.365 |\n| Object Detection/Resnet-101 | 482.425 | 361.843 |\n| Object Detection/Conditional-DETR | 344.661 | 255.298 |\n\n### T4 (batch size: 1)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 16.520 | 15.786 | \n| Image Segmentation/Segformer | 16.116 | 14.205 |\n| Object Detection/OwlViT | 53.634 | 51.105 |\n| Image Classification/BeiT | 16.464 | 15.710 | \n| Object Detection/DETR | 73.100 | 53.99 |\n| Image Classification/ConvNeXT | 32.932 | 30.845 | \n| Image Classification/ResNet | 6.031 | 4.321 |\n| Image Segmentation/Mask2former | 79.192 | 66.815 |\n| Image Segmentation/Maskformer | 200.026 | 188.268 | \n| Image Segmentation/MobileNet | 18.908 | 11.997 |\n| Object Detection/Resnet-101 | 106.622 | 82.566 |\n| Object Detection/Conditional-DETR | 77.594 | 56.984 |\n\n### T4 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 43.653 | 43.626 | \n| Image Segmentation/Segformer | 45.327 | 42.445 |\n| Image Classification/BeiT | 52.007 | 51.354 | \n| Object Detection/DETR | 277.850 | 268.003 |\n| Image Classification/ConvNeXT | 119.259 | 105.580 | \n| Image Classification/ResNet | 13.039 | 11.388 |\n| Image Segmentation/Mask2former | 201.540 | 184.670 |\n| Image Segmentation/Maskformer | 764.052 | 711.280 | \n| Image Segmentation/MobileNet | 74.289 | 48.677 |\n| Object Detection/Resnet-101 | 421.859 | 357.614 |\n| Object Detection/Conditional-DETR | 289.002 | 226.945 |\n\n### T4 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 163.914 | 160.907 | \n| Image Segmentation/Segformer | 192."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d836023f-6a1e-4dc9-a5f3-eac07ce3149f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"d836023f-6a1e-4dc9-a5f3-eac07ce3149f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a963ce14-40c5-4663-b388-5e75607d30de\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"648535502b4e4a41af4f333d11020cd7920919e68abceef632c4e5232fa6cf70\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"cf137b00-7fff-43b0-9e01-aeae3b706453\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"a9f9b97c53781c583bd40baa27483b6c8dd5a0cbb4cbb2e48db9ee7bacd832a8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 10941, \"end_char_idx\": 12680, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "908 | 11.997 |\n| Object Detection/Resnet-101 | 106.622 | 82.566 |\n| Object Detection/Conditional-DETR | 77.594 | 56.984 |\n\n### T4 (batch size: 4)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 43.653 | 43.626 | \n| Image Segmentation/Segformer | 45.327 | 42.445 |\n| Image Classification/BeiT | 52.007 | 51.354 | \n| Object Detection/DETR | 277.850 | 268.003 |\n| Image Classification/ConvNeXT | 119.259 | 105.580 | \n| Image Classification/ResNet | 13.039 | 11.388 |\n| Image Segmentation/Mask2former | 201.540 | 184.670 |\n| Image Segmentation/Maskformer | 764.052 | 711.280 | \n| Image Segmentation/MobileNet | 74.289 | 48.677 |\n| Object Detection/Resnet-101 | 421.859 | 357.614 |\n| Object Detection/Conditional-DETR | 289.002 | 226.945 |\n\n### T4 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 163.914 | 160.907 | \n| Image Segmentation/Segformer | 192.412 | 163.620 |\n| Image Classification/BeiT | 188.978 | 187.976 | \n| Object Detection/DETR | OOM | OOM |\n| Image Classification/ConvNeXT | 422.886 | 388.078 | \n| Image Classification/ResNet | 44.114 | 37.604 |\n| Image Segmentation/Mask2former | 756.337 | 695.291 |\n| Image Segmentation/Maskformer | 2842.940 | 2656.88 | \n| Image Segmentation/MobileNet | 299.003 | 201.942 |\n| Object Detection/Resnet-101 |  1619.505 | 1262.758 | \n| Object Detection/Conditional-DETR | 1137.513 | 897.390|\n\n## PyTorch Nightly\nWe also benchmarked on PyTorch nightly (2.1.0dev, find the wheel [here](https://download.pytorch.org/whl/nightly/cu118)) and observed improvement in latency both for uncompiled and compiled models."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: cf137b00-7fff-43b0-9e01-aeae3b706453
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"cf137b00-7fff-43b0-9e01-aeae3b706453\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d836023f-6a1e-4dc9-a5f3-eac07ce3149f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9d75abc02a6184be3ab3a84338d1fde757c1e93b7b2a21d37f19cad309f3f9de\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"ba07f115-32db-4517-9316-a4cc5e724dbc\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"9192f96ee7c0c80ad4248351255391c240126ba3a4eda7f3d7e4a8853ce4b890\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11538, \"end_char_idx\": 14041, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "670 |\n| Image Segmentation/Maskformer | 764.052 | 711.280 | \n| Image Segmentation/MobileNet | 74.289 | 48.677 |\n| Object Detection/Resnet-101 | 421.859 | 357.614 |\n| Object Detection/Conditional-DETR | 289.002 | 226.945 |\n\n### T4 (batch size: 16)\n\n| **Task/Model** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|\n| Image Classification/ViT | 163.914 | 160.907 | \n| Image Segmentation/Segformer | 192.412 | 163.620 |\n| Image Classification/BeiT | 188.978 | 187.976 | \n| Object Detection/DETR | OOM | OOM |\n| Image Classification/ConvNeXT | 422.886 | 388.078 | \n| Image Classification/ResNet | 44.114 | 37.604 |\n| Image Segmentation/Mask2former | 756.337 | 695.291 |\n| Image Segmentation/Maskformer | 2842.940 | 2656.88 | \n| Image Segmentation/MobileNet | 299.003 | 201.942 |\n| Object Detection/Resnet-101 |  1619.505 | 1262.758 | \n| Object Detection/Conditional-DETR | 1137.513 | 897.390|\n\n## PyTorch Nightly\nWe also benchmarked on PyTorch nightly (2.1.0dev, find the wheel [here](https://download.pytorch.org/whl/nightly/cu118)) and observed improvement in latency both for uncompiled and compiled models. \n\n### A100\n\n| **Task/Model** | **Batch Size** | **torch 2.0 - no compile** | **torch 2.0 -<br> compile** |\n|:---:|:---:|:---:|:---:|\n| Image Classification/BeiT | Unbatched | 12.462 | 6.954 | \n| Image Classification/BeiT | 4 | 14.109 | 12.851 | \n| Image Classification/BeiT | 16 | 42.179 | 42.147 | \n| Object Detection/DETR | Unbatched | 30.484 | 15.221 |\n| Object Detection/DETR | 4 | 46.816 | 30.942 |\n| Object Detection/DETR | 16 | 163.749 | 163.706  |\n\n### T4\n\n| **Task/Model** | **Batch Size** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|:---:|\n| Image Classification/BeiT | Unbatched | 14.408 | 14.052 | \n| Image Classification/BeiT | 4 | 47.381 | 46.604 | \n| Image Classification/BeiT | 16 | 42.179 | 42.147  | \n| Object Detection/DETR | Unbatched | 68.382 | 53.481 |\n| Object Detection/DETR | 4 | 269.615 | 204.785 |\n| Object Detection/DETR | 16 | OOM | OOM   |\n\n### V100\n\n| **Task/Model** | **Batch Size** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|:---:|\n| Image Classification/BeiT | Unbatched | 13.477 | 7.926 | \n| Image Classification/BeiT | 4 | 15.103 | 14.378 | \n| Image Classification/BeiT | 16 | 52.517 | 51.691  | \n| Object Detection/DETR | Unbatched | 28.706 | 19.077 |\n| Object Detection/DETR | 4 | 88.402 | 62.949|\n| Object Detection/DETR | 16 | OOM | OOM  |"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: ba07f115-32db-4517-9316-a4cc5e724dbc
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_torch_compile", "title": "Optimize inference using torch.compile()", "tokens": 5179, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"ba07f115-32db-4517-9316-a4cc5e724dbc\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"c83b11b5-0d57-40f0-91d5-7bd05ab5721c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fb25ed2f15cb56104265b8944af262358c36e4e9545c1acf8156ae74f65d2adc\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"cf137b00-7fff-43b0-9e01-aeae3b706453\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_torch_compile\", \"title\": \"Optimize inference using torch.compile()\", \"tokens\": 5179, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3b678e23ba3b1fbb2fdbbdffe0663eabddb3e85a45644b4a722e96c338f4e373\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 14044, \"end_char_idx\": 15386, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "ref_doc_id": "c83b11b5-0d57-40f0-91d5-7bd05ab5721c", "chroma:document": "## Reduce Overhead\nWe benchmarked `reduce-overhead` compilation mode for A100 and T4 in Nightly.\n\n### A100\n\n| **Task/Model** | **Batch Size** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** |\n|:---:|:---:|:---:|:---:|\n| Image Classification/ConvNeXT | Unbatched | 11.758 | 7.335 | \n| Image Classification/ConvNeXT | 4 | 23.171 | 21.490 | \n| Image Classification/ResNet | Unbatched | 7.435 | 3.801 | \n| Image Classification/ResNet | 4 | 7.261 | 2.187 | \n| Object Detection/Conditional-DETR | Unbatched | 32.823 | 11.627  | \n| Object Detection/Conditional-DETR | 4 | 50.622 | 33.831  | \n| Image Segmentation/MobileNet | Unbatched | 9.869 | 4.244 |\n| Image Segmentation/MobileNet | 4 | 14.385 | 7.946 |\n\n\n### T4\n\n| **Task/Model** | **Batch Size** | **torch 2.0 - <br>no compile** | **torch 2.0 - <br>compile** | \n|:---:|:---:|:---:|:---:|\n| Image Classification/ConvNeXT | Unbatched | 32.137 | 31.84 | \n| Image Classification/ConvNeXT | 4 | 120.944 | 110.209 | \n| Image Classification/ResNet | Unbatched | 9.761 | 7.698 | \n| Image Classification/ResNet | 4 | 15.215 | 13.871 | \n| Object Detection/Conditional-DETR | Unbatched | 72.150 | 57.660  | \n| Object Detection/Conditional-DETR | 4 | 301.494 | 247.543  | \n| Image Segmentation/MobileNet | Unbatched | 22.266 | 19.339  |\n| Image Segmentation/MobileNet | 4 | 78.311 | 50.983 |"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: fe51fd80-a45e-4557-a45c-043c13571b06
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/perf_train_special", "title": "PyTorch training on Apple silicon", "tokens": 771, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"fe51fd80-a45e-4557-a45c-043c13571b06\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_special\", \"title\": \"PyTorch training on Apple silicon\", \"tokens\": 771, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"106cc734-7145-45db-b473-86ad47564484\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/perf_train_special\", \"title\": \"PyTorch training on Apple silicon\", \"tokens\": 771, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3b92385ad59a6c38344c47086105ae06547252dae5793516cdd7089ed1d393e7\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2592, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "106cc734-7145-45db-b473-86ad47564484", "doc_id": "106cc734-7145-45db-b473-86ad47564484", "ref_doc_id": "106cc734-7145-45db-b473-86ad47564484", "chroma:document": "# PyTorch training on Apple silicon\n\nPreviously, training models on a Mac was limited to the CPU only. With the release of PyTorch v1.12, you can take advantage of training models with Apple's silicon GPUs for significantly faster performance and training. This is powered in PyTorch by integrating Apple's Metal Performance Shaders (MPS) as a backend. The [MPS backend](https://pytorch.org/docs/stable/notes/mps.html) implements PyTorch operations as custom Metal shaders and places these modules on a `mps` device.\n\n<Tip warning={true}>\n\nSome PyTorch operations are not implemented in MPS yet and will throw an error. To avoid this, you should set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU kernels instead (you'll still see a `UserWarning`).\n\n<br>\n\nIf you run into any other errors, please open an issue in the [PyTorch](https://github.com/pytorch/pytorch/issues) repository because the [`Trainer`] only integrates the MPS backend.\n\n</Tip>\n\nWith the `mps` device set, you can:\n\n* train larger networks or batch sizes locally\n* reduce data retrieval latency because the GPU's unified memory architecture allows direct access to the full memory store\n* reduce costs because you don't need to train on cloud-based GPUs or add additional local GPUs\n\nGet started by making sure you have PyTorch installed. MPS acceleration is supported on macOS 12.3+.\n\n```bash\npip install torch torchvision torchaudio\n```\n\n[`TrainingArguments`] uses the `mps` device by default if it's available which means you don't need to explicitly set the device. For example, you can run the [run_glue.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/text-classification/run_glue.py) script with the MPS backend automatically enabled without making any changes.\n\n```diff\nexport TASK_NAME=mrpc\n\npython examples/pytorch/text-classification/run_glue.py \\\n  --model_name_or_path google-bert/bert-base-cased \\\n  --task_name $TASK_NAME \\\n- --use_mps_device \\\n  --do_train \\\n  --do_eval \\\n  --max_seq_length 128 \\\n  --per_device_train_batch_size 32 \\\n  --learning_rate 2e-5 \\\n  --num_train_epochs 3 \\\n  --output_dir /tmp/$TASK_NAME/ \\\n  --overwrite_output_dir\n```\n\nBackends for [distributed setups](https://pytorch.org/docs/stable/distributed.html#backends) like `gloo` and `nccl` are not supported by the `mps` device which means you can only train on a single GPU with the MPS backend.\n\nYou can learn more about the MPS backend in the [Introducing Accelerated PyTorch Training on Mac](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/) blog post."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 820cf4ee-92d0-44e9-b5c6-3284fa959989
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/peft", "title": "Load adapters with \ud83e\udd17 PEFT", "tokens": 2169, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"820cf4ee-92d0-44e9-b5c6-3284fa959989\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ea86996-cd10-42c5-8d74-f2ec0f00791c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"be7098b1712d5597c51cf0dd6286d1c7d52c17f66a2874c74268ab4e4502eb30\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"5a7bf680-4466-4e03-95ec-21f0c3c8eb4f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e27405c2aa0699f084d144afa325ddd58ce5fd51e8c95ec25e52f539539dd320\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2778, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "ref_doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "chroma:document": "# Load adapters with \ud83e\udd17 PEFT\n\n[[open-in-colab]]\n\n[Parameter-Efficient Fine Tuning (PEFT)](https://huggingface.co/blog/peft) methods freeze the pretrained model parameters during fine-tuning and add a small number of trainable parameters (the adapters) on top of it. The adapters are trained to learn task-specific information. This approach has been shown to be very memory-efficient with lower compute usage while producing results comparable to a fully fine-tuned model. \n\nAdapters trained with PEFT are also usually an order of magnitude smaller than the full model, making it convenient to share, store, and load them.\n\n<div class=\"flex flex-col justify-center\">\n  <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/peft/PEFT-hub-screenshot.png\"/>\n  <figcaption class=\"text-center\">The adapter weights for a OPTForCausalLM model stored on the Hub are only ~6MB compared to the full size of the model weights, which can be ~700MB.</figcaption>\n</div>\n\nIf you're interested in learning more about the \ud83e\udd17 PEFT library, check out the [documentation](https://huggingface.co/docs/peft/index).\n\n## Setup\n\nGet started by installing \ud83e\udd17 PEFT:\n\n```bash\npip install peft\n```\n\nIf you want to try out the brand new features, you might be interested in installing the library from source:\n\n```bash\npip install git+https://github.com/huggingface/peft.git\n```\n\n## Supported PEFT models\n\n\ud83e\udd17 Transformers natively supports some PEFT methods, meaning you can load adapter weights stored locally or on the Hub and easily run or train them with a few lines of code. The following methods are supported:\n\n- [Low Rank Adapters](https://huggingface.co/docs/peft/conceptual_guides/lora)\n- [IA3](https://huggingface.co/docs/peft/conceptual_guides/ia3)\n- [AdaLoRA](https://arxiv.org/abs/2303.10512)\n\nIf you want to use other PEFT methods, such as prompt learning or prompt tuning, or about the \ud83e\udd17 PEFT library in general, please refer to the [documentation](https://huggingface.co/docs/peft/index).\n\n\n## Load a PEFT adapter\n\nTo load and use a PEFT adapter model from \ud83e\udd17 Transformers, make sure the Hub repository or local directory contains an `adapter_config.json` file and the adapter weights, as shown in the example image above. Then you can load the PEFT adapter model using the `AutoModelFor` class. For example, to load a PEFT adapter model for causal language modeling:\n\n1. specify the PEFT model id\n2. pass it to the [`AutoModelForCausalLM`] class\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\npeft_model_id = \"ybelkada/opt-350m-lora\"\nmodel = AutoModelForCausalLM.from_pretrained(peft_model_id)\n```\n\n<Tip>\n\nYou can load a PEFT adapter with either an `AutoModelFor` class or the base model class like `OPTForCausalLM` or `LlamaForCausalLM`."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 5a7bf680-4466-4e03-95ec-21f0c3c8eb4f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/peft", "title": "Load adapters with \ud83e\udd17 PEFT", "tokens": 2169, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"5a7bf680-4466-4e03-95ec-21f0c3c8eb4f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ea86996-cd10-42c5-8d74-f2ec0f00791c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"be7098b1712d5597c51cf0dd6286d1c7d52c17f66a2874c74268ab4e4502eb30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"820cf4ee-92d0-44e9-b5c6-3284fa959989\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"6d0d9305367ab16700b70ee912952630caad4fd79ce75febe6526fbd63bcb5ab\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"3d208f01-4ffb-486e-a927-e1ca79fa5a56\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"4439f1bbbb111b77b87e2ed770a99c44b135919652977be5dc179b7c81383eb8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2009, \"end_char_idx\": 4089, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "ref_doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "chroma:document": "## Load a PEFT adapter\n\nTo load and use a PEFT adapter model from \ud83e\udd17 Transformers, make sure the Hub repository or local directory contains an `adapter_config.json` file and the adapter weights, as shown in the example image above. Then you can load the PEFT adapter model using the `AutoModelFor` class. For example, to load a PEFT adapter model for causal language modeling:\n\n1. specify the PEFT model id\n2. pass it to the [`AutoModelForCausalLM`] class\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\npeft_model_id = \"ybelkada/opt-350m-lora\"\nmodel = AutoModelForCausalLM.from_pretrained(peft_model_id)\n```\n\n<Tip>\n\nYou can load a PEFT adapter with either an `AutoModelFor` class or the base model class like `OPTForCausalLM` or `LlamaForCausalLM`.\n\n</Tip>\n\nYou can also load a PEFT adapter by calling the `load_adapter` method:\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_id = \"facebook/opt-350m\"\npeft_model_id = \"ybelkada/opt-350m-lora\"\n\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\nmodel.load_adapter(peft_model_id)\n```\n\nCheck out the [API documentation](#transformers.integrations.PeftAdapterMixin) section below for more details.\n\n## Load in 8bit or 4bit\n\nThe `bitsandbytes` integration supports 8bit and 4bit precision data types, which are useful for loading large models because it saves memory (see the `bitsandbytes` integration [guide](./quantization#bitsandbytes-integration) to learn more). Add the `load_in_8bit` or `load_in_4bit` parameters to [`~PreTrainedModel.from_pretrained`] and set `device_map=\"auto\"` to effectively distribute the model to your hardware:\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n\npeft_model_id = \"ybelkada/opt-350m-lora\"\nmodel = AutoModelForCausalLM.from_pretrained(peft_model_id, quantization_config=BitsAndBytesConfig(load_in_8bit=True))\n```\n\n## Add a new adapter\n\nYou can use [`~peft.PeftModel.add_adapter`] to add a new adapter to a model with an existing adapter as long as the new adapter is the same type as the current one."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 3d208f01-4ffb-486e-a927-e1ca79fa5a56
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/peft", "title": "Load adapters with \ud83e\udd17 PEFT", "tokens": 2169, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"3d208f01-4ffb-486e-a927-e1ca79fa5a56\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ea86996-cd10-42c5-8d74-f2ec0f00791c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"be7098b1712d5597c51cf0dd6286d1c7d52c17f66a2874c74268ab4e4502eb30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"5a7bf680-4466-4e03-95ec-21f0c3c8eb4f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"921b7e54569fdd0057ffeb4f21bd1d970f070bd33c62bb24ac04f5c18d85d582\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4686a0e6-d2bd-4d5d-a527-e496d156508c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e02d31294110cd2094935b3338e93851a0aa7fbf820db5e4b4ba6744047aae34\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2780, \"end_char_idx\": 5167, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "ref_doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "chroma:document": "</Tip>\n\nYou can also load a PEFT adapter by calling the `load_adapter` method:\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_id = \"facebook/opt-350m\"\npeft_model_id = \"ybelkada/opt-350m-lora\"\n\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\nmodel.load_adapter(peft_model_id)\n```\n\nCheck out the [API documentation](#transformers.integrations.PeftAdapterMixin) section below for more details.\n\n## Load in 8bit or 4bit\n\nThe `bitsandbytes` integration supports 8bit and 4bit precision data types, which are useful for loading large models because it saves memory (see the `bitsandbytes` integration [guide](./quantization#bitsandbytes-integration) to learn more). Add the `load_in_8bit` or `load_in_4bit` parameters to [`~PreTrainedModel.from_pretrained`] and set `device_map=\"auto\"` to effectively distribute the model to your hardware:\n\n```py\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n\npeft_model_id = \"ybelkada/opt-350m-lora\"\nmodel = AutoModelForCausalLM.from_pretrained(peft_model_id, quantization_config=BitsAndBytesConfig(load_in_8bit=True))\n```\n\n## Add a new adapter\n\nYou can use [`~peft.PeftModel.add_adapter`] to add a new adapter to a model with an existing adapter as long as the new adapter is the same type as the current one. For example, if you have an existing LoRA adapter attached to a model:\n\n```py\nfrom transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\n\nmodel_id = \"facebook/opt-350m\"\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\n\nlora_config = LoraConfig(\n    target_modules=[\"q_proj\", \"k_proj\"],\n    init_lora_weights=False\n)\n\nmodel.add_adapter(lora_config, adapter_name=\"adapter_1\")\n```\n\nTo add a new adapter:\n\n```py\n# attach new adapter with same config\nmodel.add_adapter(lora_config, adapter_name=\"adapter_2\")\n```\n\nNow you can use [`~peft.PeftModel.set_adapter`] to set which adapter to use:\n\n```py\n# use adapter_1\nmodel.set_adapter(\"adapter_1\")\noutput = model.generate(**inputs)\nprint(tokenizer.decode(output_disabled[0], skip_special_tokens=True))\n\n# use adapter_2\nmodel.set_adapter(\"adapter_2\")\noutput_enabled = model.generate(**inputs)\nprint(tokenizer.decode(output_enabled[0], skip_special_tokens=True))\n```\n\n## Enable and disable adapters\n\nOnce you've added an adapter to a model, you can enable or disable the adapter module."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4686a0e6-d2bd-4d5d-a527-e496d156508c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/peft", "title": "Load adapters with \ud83e\udd17 PEFT", "tokens": 2169, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4686a0e6-d2bd-4d5d-a527-e496d156508c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ea86996-cd10-42c5-8d74-f2ec0f00791c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"be7098b1712d5597c51cf0dd6286d1c7d52c17f66a2874c74268ab4e4502eb30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"3d208f01-4ffb-486e-a927-e1ca79fa5a56\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"38e6e2f9e0c6e85aa43a5e6082994beadd0b09fd9b900bb2d7439075ac100da8\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"0cef7e49-421d-40f8-a4a3-d8dc4757b29a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8bc89707a9d994367f824d0c23884bd9c696dc335ef2a4491d941c57e222aafa\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4090, \"end_char_idx\": 7112, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "ref_doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "chroma:document": "For example, if you have an existing LoRA adapter attached to a model:\n\n```py\nfrom transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\n\nmodel_id = \"facebook/opt-350m\"\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\n\nlora_config = LoraConfig(\n    target_modules=[\"q_proj\", \"k_proj\"],\n    init_lora_weights=False\n)\n\nmodel.add_adapter(lora_config, adapter_name=\"adapter_1\")\n```\n\nTo add a new adapter:\n\n```py\n# attach new adapter with same config\nmodel.add_adapter(lora_config, adapter_name=\"adapter_2\")\n```\n\nNow you can use [`~peft.PeftModel.set_adapter`] to set which adapter to use:\n\n```py\n# use adapter_1\nmodel.set_adapter(\"adapter_1\")\noutput = model.generate(**inputs)\nprint(tokenizer.decode(output_disabled[0], skip_special_tokens=True))\n\n# use adapter_2\nmodel.set_adapter(\"adapter_2\")\noutput_enabled = model.generate(**inputs)\nprint(tokenizer.decode(output_enabled[0], skip_special_tokens=True))\n```\n\n## Enable and disable adapters\n\nOnce you've added an adapter to a model, you can enable or disable the adapter module. To enable the adapter module:\n\n```py\nfrom transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\nfrom peft import PeftConfig\n\nmodel_id = \"facebook/opt-350m\"\nadapter_model_id = \"ybelkada/opt-350m-lora\"\ntokenizer = AutoTokenizer.from_pretrained(model_id)\ntext = \"Hello\"\ninputs = tokenizer(text, return_tensors=\"pt\")\n\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\npeft_config = PeftConfig.from_pretrained(adapter_model_id)\n\n# to initiate with random weights\npeft_config.init_lora_weights = False\n\nmodel.add_adapter(peft_config)\nmodel.enable_adapters()\noutput = model.generate(**inputs)\n```\n\nTo disable the adapter module:\n\n```py\nmodel.disable_adapters()\noutput = model.generate(**inputs)\n```\n\n## Train a PEFT adapter\n\nPEFT adapters are supported by the [`Trainer`] class so that you can train an adapter for your specific use case. It only requires adding a few more lines of code. For example, to train a LoRA adapter:\n\n<Tip>\n\nIf you aren't familiar with fine-tuning a model with [`Trainer`], take a look at the [Fine-tune a pretrained model](training) tutorial.\n\n</Tip>\n\n1. Define your adapter configuration with the task type and hyperparameters (see [`~peft.LoraConfig`] for more details about what the hyperparameters do).\n\n```py\nfrom peft import LoraConfig\n\npeft_config = LoraConfig(\n    lora_alpha=16,\n    lora_dropout=0.1,\n    r=64,\n    bias=\"none\",\n    task_type=\"CAUSAL_LM\",\n)\n```\n\n2. Add adapter to the model.\n\n```py\nmodel.add_adapter(peft_config)\n```\n\n3. Now you can pass the model to [`Trainer`]!\n\n```py\ntrainer = Trainer(model=model, ...)\ntrainer.train()\n```\n\nTo save your trained adapter and load it back:\n\n```py\nmodel.save_pretrained(save_dir)\nmodel = AutoModelForCausalLM.from_pretrained(save_dir)\n```\n\n## Add additional trainable layers to a PEFT adapter\n\nYou can also fine-tune additional trainable adapters on top of a model that has adapters attached by passing `modules_to_save` in your PEFT config."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 0cef7e49-421d-40f8-a4a3-d8dc4757b29a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/peft", "title": "Load adapters with \ud83e\udd17 PEFT", "tokens": 2169, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"0cef7e49-421d-40f8-a4a3-d8dc4757b29a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ea86996-cd10-42c5-8d74-f2ec0f00791c\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"be7098b1712d5597c51cf0dd6286d1c7d52c17f66a2874c74268ab4e4502eb30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4686a0e6-d2bd-4d5d-a527-e496d156508c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/peft\", \"title\": \"Load adapters with \\ud83e\\udd17 PEFT\", \"tokens\": 2169, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"0888577565f0dff656e787a803fb0695de17fbfd15d1f8cf7210a5f89b7c21ad\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6023, \"end_char_idx\": 7882, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "ref_doc_id": "2ea86996-cd10-42c5-8d74-f2ec0f00791c", "chroma:document": "It only requires adding a few more lines of code. For example, to train a LoRA adapter:\n\n<Tip>\n\nIf you aren't familiar with fine-tuning a model with [`Trainer`], take a look at the [Fine-tune a pretrained model](training) tutorial.\n\n</Tip>\n\n1. Define your adapter configuration with the task type and hyperparameters (see [`~peft.LoraConfig`] for more details about what the hyperparameters do).\n\n```py\nfrom peft import LoraConfig\n\npeft_config = LoraConfig(\n    lora_alpha=16,\n    lora_dropout=0.1,\n    r=64,\n    bias=\"none\",\n    task_type=\"CAUSAL_LM\",\n)\n```\n\n2. Add adapter to the model.\n\n```py\nmodel.add_adapter(peft_config)\n```\n\n3. Now you can pass the model to [`Trainer`]!\n\n```py\ntrainer = Trainer(model=model, ...)\ntrainer.train()\n```\n\nTo save your trained adapter and load it back:\n\n```py\nmodel.save_pretrained(save_dir)\nmodel = AutoModelForCausalLM.from_pretrained(save_dir)\n```\n\n## Add additional trainable layers to a PEFT adapter\n\nYou can also fine-tune additional trainable adapters on top of a model that has adapters attached by passing `modules_to_save` in your PEFT config. For example, if you want to also fine-tune the lm_head on top of a model with a LoRA adapter:\n\n```py\nfrom transformers import AutoModelForCausalLM, OPTForCausalLM, AutoTokenizer\nfrom peft import LoraConfig\n\nmodel_id = \"facebook/opt-350m\"\nmodel = AutoModelForCausalLM.from_pretrained(model_id)\n\nlora_config = LoraConfig(\n    target_modules=[\"q_proj\", \"k_proj\"],\n    modules_to_save=[\"lm_head\"],\n)\n\nmodel.add_adapter(lora_config)\n```\n\n## API docs\n\n[[autodoc]] integrations.PeftAdapterMixin\n    - load_adapter\n    - add_adapter\n    - set_adapter\n    - disable_adapters\n    - enable_adapters\n    - active_adapters\n    - get_adapter_state_dict\n\n\n\n\n<!--\nTODO: (@younesbelkada @stevhliu)\n-   Link to PEFT docs for further details\n-   Trainer  \n-   8-bit / 4-bit examples ?\n-->"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 292813d8-6a88-45bf-9cb8-693927673485
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"292813d8-6a88-45bf-9cb8-693927673485\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f4491ae1-afc5-4aaf-9ba4-af0bc8e7aeb2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"36c01562d2d54924e66a1d73f7069c5b734782af77a28d1c786cef6dc8296979\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 2985, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "# Translation\n\n[[open-in-colab]]\n\n<Youtube id=\"1JvfrvZgi6c\"/>\n\nTranslation converts a sequence of text from one language to another. It is one of several tasks you can formulate as a sequence-to-sequence problem, a powerful framework for returning some output from an input, like translation or summarization. Translation systems are commonly used for translation between different language texts, but it can also be used for speech or some combination in between like text-to-speech or speech-to-text.\n\nThis guide will show you how to:\n\n1. Finetune [T5](https://huggingface.co/google-t5/t5-small) on the English-French subset of the [OPUS Books](https://huggingface.co/datasets/opus_books) dataset to translate English text to French.\n2. Use your finetuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/translation).\n\n</Tip>\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install transformers datasets evaluate sacrebleu\n```\n\nWe encourage you to login to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to login:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\n## Load OPUS Books dataset\n\nStart by loading the English-French subset of the [OPUS Books](https://huggingface.co/datasets/opus_books) dataset from the \ud83e\udd17 Datasets library:\n\n```py\n>>> from datasets import load_dataset\n\n>>> books = load_dataset(\"opus_books\", \"en-fr\")\n```\n\nSplit the dataset into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n```py\n>>> books = books[\"train\"].train_test_split(test_size=0.2)\n```\n\nThen take a look at an example:\n\n```py\n>>> books[\"train\"][0]\n{'id': '90560',\n 'translation': {'en': 'But this lofty plateau measured only a few fathoms, and soon we reentered Our Element.',\n  'fr': 'Mais ce plateau \u00e9lev\u00e9 ne mesurait que quelques toises, et bient\u00f4t nous f\u00fbmes rentr\u00e9s dans notre \u00e9l\u00e9ment.'}}\n```\n\n`translation`: an English and French translation of the text.\n\n## Preprocess\n\n<Youtube id=\"XAR8jnZZuUs\"/>\n\nThe next step is to load a T5 tokenizer to process the English-French language pairs:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> checkpoint = \"google-t5/t5-small\"\n>>> tokenizer = AutoTokenizer.from_pretrained(checkpoint)\n```\n\nThe preprocessing function you want to create needs to:\n\n1. Prefix the input with a prompt so T5 knows this is a translation task. Some models capable of multiple NLP tasks require prompting for specific tasks.\n2. Tokenize the input (English) and target (French) separately because you can't tokenize French text with a tokenizer pretrained on an English vocabulary.\n3. Truncate sequences to be no longer than the maximum length set by the `max_length` parameter.\n\n```py\n>>> source_lang = \"en\"\n>>> target_lang = \"fr\"\n>>> prefix = \"translate English to French: \""}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: f4491ae1-afc5-4aaf-9ba4-af0bc8e7aeb2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"f4491ae1-afc5-4aaf-9ba4-af0bc8e7aeb2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"292813d8-6a88-45bf-9cb8-693927673485\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"61811aab7a0fc36f1b986c8b38849d82b870004657c427b02e2885f166b88e62\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"62070509-de14-430e-af9d-d273b5010955\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"f424f901e7a4e1f207adbbec5fcc63901bddeae43953d7efa8b2d8c73cd3a6aa\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 2988, \"end_char_idx\": 6244, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": ">>> def preprocess_function(examples):\n...     inputs = [prefix + example[source_lang] for example in examples[\"translation\"]]\n...     targets = [example[target_lang] for example in examples[\"translation\"]]\n...     model_inputs = tokenizer(inputs, text_target=targets, max_length=128, truncation=True)\n...     return model_inputs\n```\n\nTo apply the preprocessing function over the entire dataset, use \ud83e\udd17 Datasets [`~datasets.Dataset.map`] method. You can speed up the `map` function by setting `batched=True` to process multiple elements of the dataset at once:\n\n```py\n>>> tokenized_books = books.map(preprocess_function, batched=True)\n```\n\nNow create a batch of examples using [`DataCollatorForSeq2Seq`]. It's more efficient to *dynamically pad* the sentences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.\n\n<frameworkcontent>\n<pt>\n```py\n>>> from transformers import DataCollatorForSeq2Seq\n\n>>> data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint)\n```\n</pt>\n<tf>\n\n```py\n>>> from transformers import DataCollatorForSeq2Seq\n\n>>> data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint, return_tensors=\"tf\")\n```\n</tf>\n</frameworkcontent>\n\n## Evaluate\n\nIncluding a metric during training is often helpful for evaluating your model's performance. You can quickly load a evaluation method with the \ud83e\udd17 [Evaluate](https://huggingface.co/docs/evaluate/index) library. For this task, load the [SacreBLEU](https://huggingface.co/spaces/evaluate-metric/sacrebleu) metric (see the \ud83e\udd17 Evaluate [quick tour](https://huggingface.co/docs/evaluate/a_quick_tour) to learn more about how to load and compute a metric):\n\n```py\n>>> import evaluate\n\n>>> metric = evaluate.load(\"sacrebleu\")\n```\n\nThen create a function that passes your predictions and labels to [`~evaluate.EvaluationModule.compute`] to calculate the SacreBLEU score:\n\n```py\n>>> import numpy as np\n\n\n>>> def postprocess_text(preds, labels):\n...     preds = [pred.strip() for pred in preds]\n...     labels = [[label.strip()] for label in labels]\n\n...     return preds, labels\n\n\n>>> def compute_metrics(eval_preds):\n...     preds, labels = eval_preds\n...     if isinstance(preds, tuple):\n...         preds = preds[0]\n...     decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)\n\n...     labels = np.where(labels != -100, labels, tokenizer.pad_token_id)\n...     decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n\n...     decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)\n\n...     result = metric.compute(predictions=decoded_preds, references=decoded_labels)\n...     result = {\"bleu\": result[\"score\"]}\n\n...     prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]\n...     result[\"gen_len\"] = np.mean(prediction_lens)\n...     result = {k: round(v, 4) for k, v in result.items()}\n...     return result\n```\n\nYour `compute_metrics` function is ready to go now, and you'll return to it when you setup your training.\n\n## Train\n\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#train-with-pytorch-trainer)!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 62070509-de14-430e-af9d-d273b5010955
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"62070509-de14-430e-af9d-d273b5010955\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f4491ae1-afc5-4aaf-9ba4-af0bc8e7aeb2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"54847dbf9c72805bc137e054b4d192cf430b2dfd50a65ee3257766defef57cb9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"86a5b665-cee1-4700-af2e-29bdde6db4b6\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"b908af4f159c3e551b2b0af5e0c38554a20b4d7b9cf208c0c62e066f5fd9ce94\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4938, \"end_char_idx\": 7141, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": ">>> def postprocess_text(preds, labels):\n...     preds = [pred.strip() for pred in preds]\n...     labels = [[label.strip()] for label in labels]\n\n...     return preds, labels\n\n\n>>> def compute_metrics(eval_preds):\n...     preds, labels = eval_preds\n...     if isinstance(preds, tuple):\n...         preds = preds[0]\n...     decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)\n\n...     labels = np.where(labels != -100, labels, tokenizer.pad_token_id)\n...     decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n\n...     decoded_preds, decoded_labels = postprocess_text(decoded_preds, decoded_labels)\n\n...     result = metric.compute(predictions=decoded_preds, references=decoded_labels)\n...     result = {\"bleu\": result[\"score\"]}\n\n...     prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in preds]\n...     result[\"gen_len\"] = np.mean(prediction_lens)\n...     result = {k: round(v, 4) for k, v in result.items()}\n...     return result\n```\n\nYour `compute_metrics` function is ready to go now, and you'll return to it when you setup your training.\n\n## Train\n\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#train-with-pytorch-trainer)!\n\n</Tip>\n\nYou're ready to start training your model now! Load T5 with [`AutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nAt this point, only three steps remain:\n\n1. Define your training hyperparameters in [`Seq2SeqTrainingArguments`]. The only required parameter is `output_dir` which specifies where to save your model. You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the SacreBLEU metric and save the training checkpoint.\n2. Pass the training arguments to [`Seq2SeqTrainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 86a5b665-cee1-4700-af2e-29bdde6db4b6
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"86a5b665-cee1-4700-af2e-29bdde6db4b6\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"62070509-de14-430e-af9d-d273b5010955\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"babad5fabf6a20052bb91d5feaf72b9b5ae52e3179c3a19433b3e01072bc0308\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"724bf708-613e-41e1-870e-ffd3c6b4ddd1\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"dfa70486787d27016373dbaf027cd410cb9abbf2a3f7aaf98388324fb417132b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6053, \"end_char_idx\": 8212, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "## Train\n\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#train-with-pytorch-trainer)!\n\n</Tip>\n\nYou're ready to start training your model now! Load T5 with [`AutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nAt this point, only three steps remain:\n\n1. Define your training hyperparameters in [`Seq2SeqTrainingArguments`]. The only required parameter is `output_dir` which specifies where to save your model. You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the SacreBLEU metric and save the training checkpoint.\n2. Pass the training arguments to [`Seq2SeqTrainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = Seq2SeqTrainingArguments(\n...     output_dir=\"my_awesome_opus_books_model\",\n...     eval_strategy=\"epoch\",\n...     learning_rate=2e-5,\n...     per_device_train_batch_size=16,\n...     per_device_eval_batch_size=16,\n...     weight_decay=0.01,\n...     save_total_limit=3,\n...     num_train_epochs=2,\n...     predict_with_generate=True,\n...     fp16=True,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Seq2SeqTrainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=tokenized_books[\"train\"],\n...     eval_dataset=tokenized_books[\"test\"],\n...     tokenizer=tokenizer,\n...     data_collator=data_collator,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n<tf>\n<Tip>\n\nIf you aren't familiar with finetuning a model with Keras, take a look at the basic tutorial [here](../training#train-a-tensorflow-model-with-keras)!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 724bf708-613e-41e1-870e-ffd3c6b4ddd1
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"724bf708-613e-41e1-870e-ffd3c6b4ddd1\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"86a5b665-cee1-4700-af2e-29bdde6db4b6\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9500bb3461f415cba49ecc55c85a1d1e1f4b483ef1e8fbe7899eea6da45b31f7\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"35d59f16-37ff-4964-ba25-2dd59912e8c0\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"60de0cecc43834dc4ca4d83458ffac3949d79a8ae69d8a6ff74334dc3e2572b7\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6709, \"end_char_idx\": 9715, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the SacreBLEU metric and save the training checkpoint.\n2. Pass the training arguments to [`Seq2SeqTrainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = Seq2SeqTrainingArguments(\n...     output_dir=\"my_awesome_opus_books_model\",\n...     eval_strategy=\"epoch\",\n...     learning_rate=2e-5,\n...     per_device_train_batch_size=16,\n...     per_device_eval_batch_size=16,\n...     weight_decay=0.01,\n...     save_total_limit=3,\n...     num_train_epochs=2,\n...     predict_with_generate=True,\n...     fp16=True,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Seq2SeqTrainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=tokenized_books[\"train\"],\n...     eval_dataset=tokenized_books[\"test\"],\n...     tokenizer=tokenizer,\n...     data_collator=data_collator,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n<tf>\n<Tip>\n\nIf you aren't familiar with finetuning a model with Keras, take a look at the basic tutorial [here](../training#train-a-tensorflow-model-with-keras)!\n\n</Tip>\nTo finetune a model in TensorFlow, start by setting up an optimizer function, learning rate schedule, and some training hyperparameters:\n\n```py\n>>> from transformers import AdamWeightDecay\n\n>>> optimizer = AdamWeightDecay(learning_rate=2e-5, weight_decay_rate=0.01)\n```\n\nThen you can load T5 with [`TFAutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nConvert your datasets to the `tf.data.Dataset` format with [`~transformers.TFPreTrainedModel.prepare_tf_dataset`]:\n\n```py\n>>> tf_train_set = model.prepare_tf_dataset(\n...     tokenized_books[\"train\"],\n...     shuffle=True,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n\n>>> tf_test_set = model.prepare_tf_dataset(\n...     tokenized_books[\"test\"],\n...     shuffle=False,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n```\n\nConfigure the model for training with [`compile`](https://keras.io/api/models/model_training_apis/#compile-method). Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> import tensorflow as tf\n\n>>> model.compile(optimizer=optimizer)  # No loss argument!\n```\n\nThe last two things to setup before you start training is to compute the SacreBLEU metric from the predictions, and provide a way to push your model to the Hub. Both are done by using [Keras callbacks](../main_classes/keras_callbacks)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 35d59f16-37ff-4964-ba25-2dd59912e8c0
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"35d59f16-37ff-4964-ba25-2dd59912e8c0\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"724bf708-613e-41e1-870e-ffd3c6b4ddd1\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e97e047bd74674e6eed157e7d178950291baddff8ba133828c27e50cfa12f268\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"592c4bee-2a58-44da-858d-f94ca974c123\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8fce709c167c86bdb02defdbfc2e7ccd937efd2908b41d93774123f4918b6eb1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8214, \"end_char_idx\": 11445, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "</Tip>\nTo finetune a model in TensorFlow, start by setting up an optimizer function, learning rate schedule, and some training hyperparameters:\n\n```py\n>>> from transformers import AdamWeightDecay\n\n>>> optimizer = AdamWeightDecay(learning_rate=2e-5, weight_decay_rate=0.01)\n```\n\nThen you can load T5 with [`TFAutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nConvert your datasets to the `tf.data.Dataset` format with [`~transformers.TFPreTrainedModel.prepare_tf_dataset`]:\n\n```py\n>>> tf_train_set = model.prepare_tf_dataset(\n...     tokenized_books[\"train\"],\n...     shuffle=True,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n\n>>> tf_test_set = model.prepare_tf_dataset(\n...     tokenized_books[\"test\"],\n...     shuffle=False,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n```\n\nConfigure the model for training with [`compile`](https://keras.io/api/models/model_training_apis/#compile-method). Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> import tensorflow as tf\n\n>>> model.compile(optimizer=optimizer)  # No loss argument!\n```\n\nThe last two things to setup before you start training is to compute the SacreBLEU metric from the predictions, and provide a way to push your model to the Hub. Both are done by using [Keras callbacks](../main_classes/keras_callbacks).\n\nPass your `compute_metrics` function to [`~transformers.KerasMetricCallback`]:\n\n```py\n>>> from transformers.keras_callbacks import KerasMetricCallback\n\n>>> metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_validation_set)\n```\n\nSpecify where to push your model and tokenizer in the [`~transformers.PushToHubCallback`]:\n\n```py\n>>> from transformers.keras_callbacks import PushToHubCallback\n\n>>> push_to_hub_callback = PushToHubCallback(\n...     output_dir=\"my_awesome_opus_books_model\",\n...     tokenizer=tokenizer,\n... )\n```\n\nThen bundle your callbacks together:\n\n```py\n>>> callbacks = [metric_callback, push_to_hub_callback]\n```\n\nFinally, you're ready to start training your model! Call [`fit`](https://keras.io/api/models/model_training_apis/#fit-method) with your training and validation datasets, the number of epochs, and your callbacks to finetune the model:\n\n```py\n>>> model.fit(x=tf_train_set, validation_data=tf_test_set, epochs=3, callbacks=callbacks)\n```\n\nOnce training is completed, your model is automatically uploaded to the Hub so everyone can use it!\n</tf>\n</frameworkcontent>\n\n<Tip>\n\nFor a more in-depth example of how to finetune a model for translation, take a look at the corresponding\n[PyTorch notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation.ipynb)\nor [TensorFlow notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation-tf.ipynb).\n\n</Tip>\n\n## Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nCome up with some text you'd like to translate to another language. For T5, you need to prefix your input depending on the task you're working on."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 592c4bee-2a58-44da-858d-f94ca974c123
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"592c4bee-2a58-44da-858d-f94ca974c123\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"35d59f16-37ff-4964-ba25-2dd59912e8c0\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"687df7087225af0e3cc7fda0ce8ec43824600a02a4226bfd5164775e2c63d7ff\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"1d9ec70a-1482-4e6a-8faf-a32ef5becb27\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"cadd133cb6e157b37e75e40f88d9a0f8eba34b1dbf7342ad37992d5fb7483578\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 10430, \"end_char_idx\": 13442, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "Call [`fit`](https://keras.io/api/models/model_training_apis/#fit-method) with your training and validation datasets, the number of epochs, and your callbacks to finetune the model:\n\n```py\n>>> model.fit(x=tf_train_set, validation_data=tf_test_set, epochs=3, callbacks=callbacks)\n```\n\nOnce training is completed, your model is automatically uploaded to the Hub so everyone can use it!\n</tf>\n</frameworkcontent>\n\n<Tip>\n\nFor a more in-depth example of how to finetune a model for translation, take a look at the corresponding\n[PyTorch notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation.ipynb)\nor [TensorFlow notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/translation-tf.ipynb).\n\n</Tip>\n\n## Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nCome up with some text you'd like to translate to another language. For T5, you need to prefix your input depending on the task you're working on. For translation from English to French, you should prefix your input as shown below:\n\n```py\n>>> text = \"translate English to French: Legumes share resources with nitrogen-fixing bacteria.\"\n```\n\nThe simplest way to try out your finetuned model for inference is to use it in a [`pipeline`]. Instantiate a `pipeline` for translation with your model, and pass your text to it:\n\n```py\n>>> from transformers import pipeline\n\n# Change `xx` to the language of the input and `yy` to the language of the desired output.\n# Examples: \"en\" for English, \"fr\" for French, \"de\" for German, \"es\" for Spanish, \"zh\" for Chinese, etc; translation_en_to_fr translates English to French\n# You can view all the lists of languages here - https://huggingface.co/languages\n>>> translator = pipeline(\"translation_xx_to_yy\", model=\"my_awesome_opus_books_model\")\n>>> translator(text)\n[{'translation_text': 'Legumes partagent des ressources avec des bact\u00e9ries azotantes.'}]\n```\n\nYou can also manually replicate the results of the `pipeline` if you'd like:\n\n<frameworkcontent>\n<pt>\nTokenize the text and return the `input_ids` as PyTorch tensors:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"my_awesome_opus_books_model\")\n>>> inputs = tokenizer(text, return_tensors=\"pt\").input_ids\n```\n\nUse the [`~generation.GenerationMixin.generate`] method to create the translation. For more details about the different text generation strategies and parameters for controlling generation, check out the [Text Generation](../main_classes/text_generation) API.\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(\"my_awesome_opus_books_model\")\n>>> outputs = model.generate(inputs, max_new_tokens=40, do_sample=True, top_k=30, top_p=0.95)\n```\n\nDecode the generated token ids back into text:\n\n```py\n>>> tokenizer.decode(outputs[0], skip_special_tokens=True)\n'Les lign\u00e9es partagent des ressources avec des bact\u00e9ries enfixant l'azote.'"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 1d9ec70a-1482-4e6a-8faf-a32ef5becb27
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/translation", "title": "Translation", "tokens": 3616, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"1d9ec70a-1482-4e6a-8faf-a32ef5becb27\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"97624bdeedd431cf36331c5b0795d99b91d7cbc9fc9b9edbeaf34a68929f4d30\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"592c4bee-2a58-44da-858d-f94ca974c123\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/translation\", \"title\": \"Translation\", \"tokens\": 3616, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"ed3f6dfed18ed5808baf7cfc727eaa3de998b9e8e800d8dc1e9c8b69b88e7819\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11956, \"end_char_idx\": 14466, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "ref_doc_id": "2ead920d-ae8e-4c3f-8d0c-cb0977ef2faf", "chroma:document": "# Examples: \"en\" for English, \"fr\" for French, \"de\" for German, \"es\" for Spanish, \"zh\" for Chinese, etc; translation_en_to_fr translates English to French\n# You can view all the lists of languages here - https://huggingface.co/languages\n>>> translator = pipeline(\"translation_xx_to_yy\", model=\"my_awesome_opus_books_model\")\n>>> translator(text)\n[{'translation_text': 'Legumes partagent des ressources avec des bact\u00e9ries azotantes.'}]\n```\n\nYou can also manually replicate the results of the `pipeline` if you'd like:\n\n<frameworkcontent>\n<pt>\nTokenize the text and return the `input_ids` as PyTorch tensors:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"my_awesome_opus_books_model\")\n>>> inputs = tokenizer(text, return_tensors=\"pt\").input_ids\n```\n\nUse the [`~generation.GenerationMixin.generate`] method to create the translation. For more details about the different text generation strategies and parameters for controlling generation, check out the [Text Generation](../main_classes/text_generation) API.\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(\"my_awesome_opus_books_model\")\n>>> outputs = model.generate(inputs, max_new_tokens=40, do_sample=True, top_k=30, top_p=0.95)\n```\n\nDecode the generated token ids back into text:\n\n```py\n>>> tokenizer.decode(outputs[0], skip_special_tokens=True)\n'Les lign\u00e9es partagent des ressources avec des bact\u00e9ries enfixant l'azote.'\n```\n</pt>\n<tf>\nTokenize the text and return the `input_ids` as TensorFlow tensors:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"my_awesome_opus_books_model\")\n>>> inputs = tokenizer(text, return_tensors=\"tf\").input_ids\n```\n\nUse the [`~transformers.generation_tf_utils.TFGenerationMixin.generate`] method to create the translation. For more details about the different text generation strategies and parameters for controlling generation, check out the [Text Generation](../main_classes/text_generation) API.\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(\"my_awesome_opus_books_model\")\n>>> outputs = model.generate(inputs, max_new_tokens=40, do_sample=True, top_k=30, top_p=0.95)\n```\n\nDecode the generated token ids back into text:\n\n```py\n>>> tokenizer.decode(outputs[0], skip_special_tokens=True)\n'Les lugumes partagent les ressources avec des bact\u00e9ries fixatrices d'azote.'\n```\n</tf>\n</frameworkcontent>"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: ea84be60-abe0-4444-a941-70de7ce64fc5
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"ea84be60-abe0-4444-a941-70de7ce64fc5\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c5d9289b-c427-43ae-85b0-e2477c8c0e0f\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3311d53bd580533cd05c4af34f34169d9ecb33adad1a073090d19bfefc810391\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3090, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "# Document Question Answering\n\n[[open-in-colab]]\n\nDocument Question Answering, also referred to as Document Visual Question Answering, is a task that involves providing\nanswers to questions posed about document images. The input to models supporting this task is typically a combination of an image and\na question, and the output is an answer expressed in natural language. These models utilize multiple modalities, including\ntext, the positions of words (bounding boxes), and the image itself.\n\nThis guide illustrates how to:\n\n- Fine-tune [LayoutLMv2](../model_doc/layoutlmv2) on the [DocVQA dataset](https://huggingface.co/datasets/nielsr/docvqa_1200_examples_donut).\n- Use your fine-tuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/image-to-text)\n\n</Tip>\n\nLayoutLMv2 solves the document question-answering task by adding a question-answering head on top of the final hidden\nstates of the tokens, to predict the positions of the start and end tokens of the\nanswer. In other words, the problem is treated as extractive question answering: given the context, extract which piece\nof information answers the question. The context comes from the output of an OCR engine, here it is Google's Tesseract.\n\nBefore you begin, make sure you have all the necessary libraries installed. LayoutLMv2 depends on detectron2, torchvision and tesseract.\n\n```bash\npip install -q transformers datasets\n```\n\n```bash\npip install 'git+https://github.com/facebookresearch/detectron2.git'\npip install torchvision\n```\n\n```bash\nsudo apt install tesseract-ocr\npip install -q pytesseract\n```\n\nOnce you have installed all of the dependencies, restart your runtime.\n\nWe encourage you to share your model with the community. Log in to your Hugging Face account to upload it to the \ud83e\udd17 Hub.\nWhen prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\nLet's define some global variables.\n\n```py\n>>> model_checkpoint = \"microsoft/layoutlmv2-base-uncased\"\n>>> batch_size = 4\n```\n\n## Load the data\n\nIn this guide we use a small sample of preprocessed DocVQA that you can find on \ud83e\udd17 Hub. If you'd like to use the full\nDocVQA dataset, you can register and download it on [DocVQA homepage](https://rrc.cvc.uab.es/?ch=17). If you do so, to\nproceed with this guide check out [how to load files into a \ud83e\udd17 dataset](https://huggingface.co/docs/datasets/loading#local-and-remote-files).\n\n```py\n>>> from datasets import load_dataset\n\n>>> dataset = load_dataset(\"nielsr/docvqa_1200_examples\")\n>>> dataset\nDatasetDict({\n    train: Dataset({\n        features: ['id', 'image', 'query', 'answers', 'words', 'bounding_boxes', 'answer'],\n        num_rows: 1000\n    })\n    test: Dataset({\n        features: ['id', 'image', 'query', 'answers', 'words', 'bounding_boxes', 'answer'],\n        num_rows: 200\n    })\n})\n```\n\nAs you can see, the dataset is split into train and test sets already. Take a look at a random example to familiarize\nyourself with the features."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c5d9289b-c427-43ae-85b0-e2477c8c0e0f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"c5d9289b-c427-43ae-85b0-e2477c8c0e0f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"ea84be60-abe0-4444-a941-70de7ce64fc5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"cac7c435be5c5d274480dd3315ba006b56fcba1820decbf902461fa6bd5b4c3f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c8da2e0e-2d7b-413b-9179-b87a79325a71\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"73b6abd76c0efa6563d6d8be28363ac08bfc38810478c4a81fcaa1c24f7e7f8b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1758, \"end_char_idx\": 4821, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "We encourage you to share your model with the community. Log in to your Hugging Face account to upload it to the \ud83e\udd17 Hub.\nWhen prompted, enter your token to log in:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\nLet's define some global variables.\n\n```py\n>>> model_checkpoint = \"microsoft/layoutlmv2-base-uncased\"\n>>> batch_size = 4\n```\n\n## Load the data\n\nIn this guide we use a small sample of preprocessed DocVQA that you can find on \ud83e\udd17 Hub. If you'd like to use the full\nDocVQA dataset, you can register and download it on [DocVQA homepage](https://rrc.cvc.uab.es/?ch=17). If you do so, to\nproceed with this guide check out [how to load files into a \ud83e\udd17 dataset](https://huggingface.co/docs/datasets/loading#local-and-remote-files).\n\n```py\n>>> from datasets import load_dataset\n\n>>> dataset = load_dataset(\"nielsr/docvqa_1200_examples\")\n>>> dataset\nDatasetDict({\n    train: Dataset({\n        features: ['id', 'image', 'query', 'answers', 'words', 'bounding_boxes', 'answer'],\n        num_rows: 1000\n    })\n    test: Dataset({\n        features: ['id', 'image', 'query', 'answers', 'words', 'bounding_boxes', 'answer'],\n        num_rows: 200\n    })\n})\n```\n\nAs you can see, the dataset is split into train and test sets already. Take a look at a random example to familiarize\nyourself with the features.\n\n```py\n>>> dataset[\"train\"].features\n```\n\nHere's what the individual fields represent:\n* `id`: the example's id\n* `image`: a PIL.Image.Image object containing the document image\n* `query`: the question string - natural language asked question, in several languages\n* `answers`: a list of correct answers provided by human annotators\n* `words` and `bounding_boxes`: the results of OCR, which we will not use here\n* `answer`: an answer matched by a different model which we will not use here\n\nLet's leave only English questions, and drop the `answer` feature which appears to contain predictions by another model.\nWe'll also take the first of the answers from the set provided by the annotators. Alternatively, you can randomly sample it.\n\n```py\n>>> updated_dataset = dataset.map(lambda example: {\"question\": example[\"query\"][\"en\"]}, remove_columns=[\"query\"])\n>>> updated_dataset = updated_dataset.map(\n...     lambda example: {\"answer\": example[\"answers\"][0]}, remove_columns=[\"answer\", \"answers\"]\n... )\n```\n\nNote that the LayoutLMv2 checkpoint that we use in this guide has been trained with `max_position_embeddings = 512` (you can\nfind this information in the [checkpoint's `config.json` file](https://huggingface.co/microsoft/layoutlmv2-base-uncased/blob/main/config.json#L18)).\nWe can truncate the examples but to avoid the situation where the answer might be at the end of a large document and end up truncated,\nhere we'll remove the few examples where the embedding is likely to end up longer than 512.\nIf most of the documents in your dataset are long, you can implement a sliding window strategy - check out [this notebook](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb) for details."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c8da2e0e-2d7b-413b-9179-b87a79325a71
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"c8da2e0e-2d7b-413b-9179-b87a79325a71\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c5d9289b-c427-43ae-85b0-e2477c8c0e0f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"cad004b9bfb0723f48bb1329c11e86bf0c9e4704ffa958fe04db04f63b4d531c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c8b2409a-b103-4826-a6f0-de3f49499487\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"7c636516b9b7234d61be84065d312d4873b3300bbe82fb0cebb494b32fdf33e4\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3092, \"end_char_idx\": 6396, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "```py\n>>> dataset[\"train\"].features\n```\n\nHere's what the individual fields represent:\n* `id`: the example's id\n* `image`: a PIL.Image.Image object containing the document image\n* `query`: the question string - natural language asked question, in several languages\n* `answers`: a list of correct answers provided by human annotators\n* `words` and `bounding_boxes`: the results of OCR, which we will not use here\n* `answer`: an answer matched by a different model which we will not use here\n\nLet's leave only English questions, and drop the `answer` feature which appears to contain predictions by another model.\nWe'll also take the first of the answers from the set provided by the annotators. Alternatively, you can randomly sample it.\n\n```py\n>>> updated_dataset = dataset.map(lambda example: {\"question\": example[\"query\"][\"en\"]}, remove_columns=[\"query\"])\n>>> updated_dataset = updated_dataset.map(\n...     lambda example: {\"answer\": example[\"answers\"][0]}, remove_columns=[\"answer\", \"answers\"]\n... )\n```\n\nNote that the LayoutLMv2 checkpoint that we use in this guide has been trained with `max_position_embeddings = 512` (you can\nfind this information in the [checkpoint's `config.json` file](https://huggingface.co/microsoft/layoutlmv2-base-uncased/blob/main/config.json#L18)).\nWe can truncate the examples but to avoid the situation where the answer might be at the end of a large document and end up truncated,\nhere we'll remove the few examples where the embedding is likely to end up longer than 512.\nIf most of the documents in your dataset are long, you can implement a sliding window strategy - check out [this notebook](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb) for details.\n\n```py\n>>> updated_dataset = updated_dataset.filter(lambda x: len(x[\"words\"]) + len(x[\"question\"].split()) < 512)\n```\n\nAt this point let's also remove the OCR features from this dataset. These are a result of OCR for fine-tuning a different\nmodel. They would still require some processing if we wanted to use them, as they do not match the input requirements\nof the model we use in this guide. Instead, we can use the [`LayoutLMv2Processor`] on the original data for both OCR and\ntokenization. This way we'll get the inputs that match model's expected input. If you want to process images manually,\ncheck out the [`LayoutLMv2` model documentation](../model_doc/layoutlmv2) to learn what input format the model expects.\n\n```py\n>>> updated_dataset = updated_dataset.remove_columns(\"words\")\n>>> updated_dataset = updated_dataset.remove_columns(\"bounding_boxes\")\n```\n\nFinally, the data exploration won't be complete if we don't peek at an image example.\n\n```py\n>>> updated_dataset[\"train\"][11][\"image\"]\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/docvqa_example.jpg\" alt=\"DocVQA Image Example\"/>\n </div>\n\n## Preprocess the data\n\nThe Document Question Answering task is a multimodal task, and you need to make sure that the inputs from each modality\nare preprocessed according to the model's expectations. Let's start by loading the [`LayoutLMv2Processor`], which internally combines an image processor that can handle image data and a tokenizer that can encode text data."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c8b2409a-b103-4826-a6f0-de3f49499487
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"c8b2409a-b103-4826-a6f0-de3f49499487\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c8da2e0e-2d7b-413b-9179-b87a79325a71\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3c5d624efd9c575748938da1e63a1b0a2432cb041797aa8f38eca2aa701d245c\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"e96a3171-2597-46a2-ab7f-e584c2e39869\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"ef191df0b3d976823f38e66de4ec3b0ed801134b0ab52fe498f73f629556a331\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 4823, \"end_char_idx\": 8148, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "```py\n>>> updated_dataset = updated_dataset.filter(lambda x: len(x[\"words\"]) + len(x[\"question\"].split()) < 512)\n```\n\nAt this point let's also remove the OCR features from this dataset. These are a result of OCR for fine-tuning a different\nmodel. They would still require some processing if we wanted to use them, as they do not match the input requirements\nof the model we use in this guide. Instead, we can use the [`LayoutLMv2Processor`] on the original data for both OCR and\ntokenization. This way we'll get the inputs that match model's expected input. If you want to process images manually,\ncheck out the [`LayoutLMv2` model documentation](../model_doc/layoutlmv2) to learn what input format the model expects.\n\n```py\n>>> updated_dataset = updated_dataset.remove_columns(\"words\")\n>>> updated_dataset = updated_dataset.remove_columns(\"bounding_boxes\")\n```\n\nFinally, the data exploration won't be complete if we don't peek at an image example.\n\n```py\n>>> updated_dataset[\"train\"][11][\"image\"]\n```\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/docvqa_example.jpg\" alt=\"DocVQA Image Example\"/>\n </div>\n\n## Preprocess the data\n\nThe Document Question Answering task is a multimodal task, and you need to make sure that the inputs from each modality\nare preprocessed according to the model's expectations. Let's start by loading the [`LayoutLMv2Processor`], which internally combines an image processor that can handle image data and a tokenizer that can encode text data.\n\n```py\n>>> from transformers import AutoProcessor\n\n>>> processor = AutoProcessor.from_pretrained(model_checkpoint)\n```\n\n### Preprocessing document images\n\nFirst, let's prepare the document images for the model with the help of the `image_processor` from the processor.\nBy default, image processor resizes the images to 224x224, makes sure they have the correct order of color channels,\napplies OCR with tesseract to get words and normalized bounding boxes. In this tutorial, all of these defaults are exactly what we need.\nWrite a function that applies the default image processing to a batch of images and returns the results of OCR.\n\n```py\n>>> image_processor = processor.image_processor\n\n\n>>> def get_ocr_words_and_boxes(examples):\n...     images = [image.convert(\"RGB\") for image in examples[\"image\"]]\n...     encoded_inputs = image_processor(images)\n\n...     examples[\"image\"] = encoded_inputs.pixel_values\n...     examples[\"words\"] = encoded_inputs.words\n...     examples[\"boxes\"] = encoded_inputs.boxes\n\n...     return examples\n```\n\nTo apply this preprocessing to the entire dataset in a fast way, use [`~datasets.Dataset.map`].\n\n```py\n>>> dataset_with_ocr = updated_dataset.map(get_ocr_words_and_boxes, batched=True, batch_size=2)\n```\n\n### Preprocessing text data\n\nOnce we have applied OCR to the images, we need to encode the text part of the dataset to prepare it for the model.\nThis involves converting the words and boxes that we got in the previous step to token-level `input_ids`, `attention_mask`,\n`token_type_ids` and `bbox`. For preprocessing text, we'll need the `tokenizer` from the processor.\n\n```py\n>>> tokenizer = processor.tokenizer\n```\n\nOn top of the preprocessing mentioned above, we also need to add the labels for the model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: e96a3171-2597-46a2-ab7f-e584c2e39869
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"e96a3171-2597-46a2-ab7f-e584c2e39869\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c8b2409a-b103-4826-a6f0-de3f49499487\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c2f66e10e73ac773e3c037d0c7640a6d28664a81bef4873ce48f3efcd3c2312e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"16f1f39d-c952-4dc7-ab5d-38f75a7a8a80\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"213a74c67dd3d49267d3a4713aed4d3b93947a49030f0b9a211a41b332a28ddc\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6398, \"end_char_idx\": 9193, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "```py\n>>> from transformers import AutoProcessor\n\n>>> processor = AutoProcessor.from_pretrained(model_checkpoint)\n```\n\n### Preprocessing document images\n\nFirst, let's prepare the document images for the model with the help of the `image_processor` from the processor.\nBy default, image processor resizes the images to 224x224, makes sure they have the correct order of color channels,\napplies OCR with tesseract to get words and normalized bounding boxes. In this tutorial, all of these defaults are exactly what we need.\nWrite a function that applies the default image processing to a batch of images and returns the results of OCR.\n\n```py\n>>> image_processor = processor.image_processor\n\n\n>>> def get_ocr_words_and_boxes(examples):\n...     images = [image.convert(\"RGB\") for image in examples[\"image\"]]\n...     encoded_inputs = image_processor(images)\n\n...     examples[\"image\"] = encoded_inputs.pixel_values\n...     examples[\"words\"] = encoded_inputs.words\n...     examples[\"boxes\"] = encoded_inputs.boxes\n\n...     return examples\n```\n\nTo apply this preprocessing to the entire dataset in a fast way, use [`~datasets.Dataset.map`].\n\n```py\n>>> dataset_with_ocr = updated_dataset.map(get_ocr_words_and_boxes, batched=True, batch_size=2)\n```\n\n### Preprocessing text data\n\nOnce we have applied OCR to the images, we need to encode the text part of the dataset to prepare it for the model.\nThis involves converting the words and boxes that we got in the previous step to token-level `input_ids`, `attention_mask`,\n`token_type_ids` and `bbox`. For preprocessing text, we'll need the `tokenizer` from the processor.\n\n```py\n>>> tokenizer = processor.tokenizer\n```\n\nOn top of the preprocessing mentioned above, we also need to add the labels for the model. For `xxxForQuestionAnswering` models\nin \ud83e\udd17 Transformers, the labels consist of the `start_positions` and `end_positions`, indicating which token is at the\nstart and which token is at the end of the answer.\n\nLet's start with that. Define a helper function that can find a sublist (the answer split into words) in a larger list (the words list).\n\nThis function will take two lists as input, `words_list` and `answer_list`. It will then iterate over the `words_list` and check\nif the current word in the `words_list` (words_list[i]) is equal to the first word of answer_list (answer_list[0]) and if\nthe sublist of `words_list` starting from the current word and of the same length as `answer_list` is equal `to answer_list`.\nIf this condition is true, it means that a match has been found, and the function will record the match, its starting index (idx),\nand its ending index (idx + len(answer_list) - 1). If more than one match was found, the function will return only the first one.\nIf no match is found, the function returns (`None`, 0, and 0)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 16f1f39d-c952-4dc7-ab5d-38f75a7a8a80
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"16f1f39d-c952-4dc7-ab5d-38f75a7a8a80\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"e96a3171-2597-46a2-ab7f-e584c2e39869\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"506d558068d4251ed82cbd3df45135a6aa984d9f1c95aad9a79277de941a3f1a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"922ed5a7-7b11-4547-bbbe-b88ed5ba47be\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1be996007826f65ac69e35b683e3a4fd5423b560e465f82e8437dd414e3bf2e8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7534, \"end_char_idx\": 10500, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "```py\n>>> dataset_with_ocr = updated_dataset.map(get_ocr_words_and_boxes, batched=True, batch_size=2)\n```\n\n### Preprocessing text data\n\nOnce we have applied OCR to the images, we need to encode the text part of the dataset to prepare it for the model.\nThis involves converting the words and boxes that we got in the previous step to token-level `input_ids`, `attention_mask`,\n`token_type_ids` and `bbox`. For preprocessing text, we'll need the `tokenizer` from the processor.\n\n```py\n>>> tokenizer = processor.tokenizer\n```\n\nOn top of the preprocessing mentioned above, we also need to add the labels for the model. For `xxxForQuestionAnswering` models\nin \ud83e\udd17 Transformers, the labels consist of the `start_positions` and `end_positions`, indicating which token is at the\nstart and which token is at the end of the answer.\n\nLet's start with that. Define a helper function that can find a sublist (the answer split into words) in a larger list (the words list).\n\nThis function will take two lists as input, `words_list` and `answer_list`. It will then iterate over the `words_list` and check\nif the current word in the `words_list` (words_list[i]) is equal to the first word of answer_list (answer_list[0]) and if\nthe sublist of `words_list` starting from the current word and of the same length as `answer_list` is equal `to answer_list`.\nIf this condition is true, it means that a match has been found, and the function will record the match, its starting index (idx),\nand its ending index (idx + len(answer_list) - 1). If more than one match was found, the function will return only the first one.\nIf no match is found, the function returns (`None`, 0, and 0).\n\n```py\n>>> def subfinder(words_list, answer_list):\n...     matches = []\n...     start_indices = []\n...     end_indices = []\n...     for idx, i in enumerate(range(len(words_list))):\n...         if words_list[i] == answer_list[0] and words_list[i : i + len(answer_list)] == answer_list:\n...             matches.append(answer_list)\n...             start_indices.append(idx)\n...             end_indices.append(idx + len(answer_list) - 1)\n...     if matches:\n...         return matches[0], start_indices[0], end_indices[0]\n...     else:\n...         return None, 0, 0\n```\n\nTo illustrate how this function finds the position of the answer, let's use it on an example:\n\n```py\n>>> example = dataset_with_ocr[\"train\"][1]\n>>> words = [word.lower() for word in example[\"words\"]]\n>>> match, word_idx_start, word_idx_end = subfinder(words, example[\"answer\"].lower().split())\n>>> print(\"Question: \", example[\"question\"])\n>>> print(\"Words:\", words)\n>>> print(\"Answer: \", example[\"answer\"])\n>>> print(\"start_index\", word_idx_start)\n>>> print(\"end_index\", word_idx_end)\nQuestion:  Who is in  cc in this letter?\nWords: ['wie', 'baw', 'brown', '&', 'williamson', 'tobacco', 'corporation', 'research', '&', 'development', 'internal', 'correspondence', 'to:', 'r.', 'h.', 'honeycutt', 'ce:', 't.f.', 'riehl', 'from:', '.', 'c.j."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 922ed5a7-7b11-4547-bbbe-b88ed5ba47be
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"922ed5a7-7b11-4547-bbbe-b88ed5ba47be\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"16f1f39d-c952-4dc7-ab5d-38f75a7a8a80\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"e72099d6650ec9fc97dad590d0f3864bca50700bec6225d43fe51b164161ec2a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"27c51fcd-d620-42e8-be93-05611cf3f876\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"ea9c2ad6fe327da3d8086b56381da8f049b2d2abf9ddd19de0e2b8824ebccb45\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9052, \"end_char_idx\": 11606, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "If more than one match was found, the function will return only the first one.\nIf no match is found, the function returns (`None`, 0, and 0).\n\n```py\n>>> def subfinder(words_list, answer_list):\n...     matches = []\n...     start_indices = []\n...     end_indices = []\n...     for idx, i in enumerate(range(len(words_list))):\n...         if words_list[i] == answer_list[0] and words_list[i : i + len(answer_list)] == answer_list:\n...             matches.append(answer_list)\n...             start_indices.append(idx)\n...             end_indices.append(idx + len(answer_list) - 1)\n...     if matches:\n...         return matches[0], start_indices[0], end_indices[0]\n...     else:\n...         return None, 0, 0\n```\n\nTo illustrate how this function finds the position of the answer, let's use it on an example:\n\n```py\n>>> example = dataset_with_ocr[\"train\"][1]\n>>> words = [word.lower() for word in example[\"words\"]]\n>>> match, word_idx_start, word_idx_end = subfinder(words, example[\"answer\"].lower().split())\n>>> print(\"Question: \", example[\"question\"])\n>>> print(\"Words:\", words)\n>>> print(\"Answer: \", example[\"answer\"])\n>>> print(\"start_index\", word_idx_start)\n>>> print(\"end_index\", word_idx_end)\nQuestion:  Who is in  cc in this letter?\nWords: ['wie', 'baw', 'brown', '&', 'williamson', 'tobacco', 'corporation', 'research', '&', 'development', 'internal', 'correspondence', 'to:', 'r.', 'h.', 'honeycutt', 'ce:', 't.f.', 'riehl', 'from:', '.', 'c.j.', 'cook', 'date:', 'may', '8,', '1995', 'subject:', 'review', 'of', 'existing', 'brainstorming', 'ideas/483', 'the', 'major', 'function', 'of', 'the', 'product', 'innovation', 'graup', 'is', 'to', 'develop', 'marketable', 'nove!', 'products', 'that', 'would', 'be', 'profitable', 'to', 'manufacture', 'and', 'sell.', 'novel', 'is', 'defined', 'as:', 'of', 'a', 'new', 'kind,', 'or', 'different', 'from', 'anything', 'seen', 'or', 'known', 'before.', 'innovation', 'is', 'defined', 'as:', 'something', 'new', 'or', 'different', 'introduced;', 'act', 'of', 'innovating;', 'introduction', 'of', 'new', 'things', 'or', 'methods.', 'the', 'products', 'may', 'incorporate', 'the', 'latest', 'technologies,', 'materials', 'and', 'know-how', 'available', 'to', 'give', 'then', 'a', 'unique', 'taste', 'or', 'look.', 'the', 'first', 'task', 'of', 'the', 'product', 'innovation', 'group', 'was', 'to', 'assemble,', 'review', 'and', 'categorize', 'a', 'list', 'of', 'existing', 'brainstorming', 'ideas.', 'ideas', 'were', 'grouped', 'into', 'two', 'major', 'categories', 'labeled', 'appearance', 'and', 'taste/aroma."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 27c51fcd-d620-42e8-be93-05611cf3f876
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"27c51fcd-d620-42e8-be93-05611cf3f876\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"922ed5a7-7b11-4547-bbbe-b88ed5ba47be\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"8ab87b63fed3d835f7970c7b3cb08a176b735062628e597684361dd1ee6c9585\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"b12f4735-ab4a-4bfe-a826-c185e49a8f1c\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6c23259061dd04b077ba3316885b24d1c5eb20b10475aadfe7b2a78f363e73f5\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 10492, \"end_char_idx\": 12569, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "', 'c.j.', 'cook', 'date:', 'may', '8,', '1995', 'subject:', 'review', 'of', 'existing', 'brainstorming', 'ideas/483', 'the', 'major', 'function', 'of', 'the', 'product', 'innovation', 'graup', 'is', 'to', 'develop', 'marketable', 'nove!', 'products', 'that', 'would', 'be', 'profitable', 'to', 'manufacture', 'and', 'sell.', 'novel', 'is', 'defined', 'as:', 'of', 'a', 'new', 'kind,', 'or', 'different', 'from', 'anything', 'seen', 'or', 'known', 'before.', 'innovation', 'is', 'defined', 'as:', 'something', 'new', 'or', 'different', 'introduced;', 'act', 'of', 'innovating;', 'introduction', 'of', 'new', 'things', 'or', 'methods.', 'the', 'products', 'may', 'incorporate', 'the', 'latest', 'technologies,', 'materials', 'and', 'know-how', 'available', 'to', 'give', 'then', 'a', 'unique', 'taste', 'or', 'look.', 'the', 'first', 'task', 'of', 'the', 'product', 'innovation', 'group', 'was', 'to', 'assemble,', 'review', 'and', 'categorize', 'a', 'list', 'of', 'existing', 'brainstorming', 'ideas.', 'ideas', 'were', 'grouped', 'into', 'two', 'major', 'categories', 'labeled', 'appearance', 'and', 'taste/aroma.', 'these', 'categories', 'are', 'used', 'for', 'novel', 'products', 'that', 'may', 'differ', 'from', 'a', 'visual', 'and/or', 'taste/aroma', 'point', 'of', 'view', 'compared', 'to', 'canventional', 'cigarettes.', 'other', 'categories', 'include', 'a', 'combination', 'of', 'the', 'above,', 'filters,', 'packaging', 'and', 'brand', 'extensions.', 'appearance', 'this', 'category', 'is', 'used', 'for', 'novel', 'cigarette', 'constructions', 'that', 'yield', 'visually', 'different', 'products', 'with', 'minimal', 'changes', 'in', 'smoke', 'chemistry', 'two', 'cigarettes', 'in', 'cne.', 'emulti-plug', 'te', 'build', 'yaur', 'awn', 'cigarette.', 'eswitchable', 'menthol', 'or', 'non', 'menthol', 'cigarette.', '*cigarettes', 'with', 'interspaced', 'perforations', 'to', 'enable', 'smoker', 'to', 'separate', 'unburned', 'section', 'for', 'future', 'smoking.', '\u00abshort', 'cigarette,', 'tobacco', 'section', '30', 'mm.', '\u00abextremely', 'fast', 'buming', 'cigarette."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: b12f4735-ab4a-4bfe-a826-c185e49a8f1c
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"b12f4735-ab4a-4bfe-a826-c185e49a8f1c\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"27c51fcd-d620-42e8-be93-05611cf3f876\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"74d3177ca4c57ef59428745a8e3b053126181647678349f42aa4daccd9ce7bb5\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"a71a6321-cc61-4629-81ac-46390ef95855\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"253e53b788e21308ff0d22b7e8776240d5b82a2929f38ffb5874994a70b8906d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11492, \"end_char_idx\": 13724, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "', 'ideas', 'were', 'grouped', 'into', 'two', 'major', 'categories', 'labeled', 'appearance', 'and', 'taste/aroma.', 'these', 'categories', 'are', 'used', 'for', 'novel', 'products', 'that', 'may', 'differ', 'from', 'a', 'visual', 'and/or', 'taste/aroma', 'point', 'of', 'view', 'compared', 'to', 'canventional', 'cigarettes.', 'other', 'categories', 'include', 'a', 'combination', 'of', 'the', 'above,', 'filters,', 'packaging', 'and', 'brand', 'extensions.', 'appearance', 'this', 'category', 'is', 'used', 'for', 'novel', 'cigarette', 'constructions', 'that', 'yield', 'visually', 'different', 'products', 'with', 'minimal', 'changes', 'in', 'smoke', 'chemistry', 'two', 'cigarettes', 'in', 'cne.', 'emulti-plug', 'te', 'build', 'yaur', 'awn', 'cigarette.', 'eswitchable', 'menthol', 'or', 'non', 'menthol', 'cigarette.', '*cigarettes', 'with', 'interspaced', 'perforations', 'to', 'enable', 'smoker', 'to', 'separate', 'unburned', 'section', 'for', 'future', 'smoking.', '\u00abshort', 'cigarette,', 'tobacco', 'section', '30', 'mm.', '\u00abextremely', 'fast', 'buming', 'cigarette.', '\u00abnovel', 'cigarette', 'constructions', 'that', 'permit', 'a', 'significant', 'reduction', 'iretobacco', 'weight', 'while', 'maintaining', 'smoking', 'mechanics', 'and', 'visual', 'characteristics.', 'higher', 'basis', 'weight', 'paper:', 'potential', 'reduction', 'in', 'tobacco', 'weight.', '\u00abmore', 'rigid', 'tobacco', 'column;', 'stiffing', 'agent', 'for', 'tobacco;', 'e.g.', 'starch', '*colored', 'tow', 'and', 'cigarette', 'papers;', 'seasonal', 'promotions,', 'e.g.', 'pastel', 'colored', 'cigarettes', 'for', 'easter', 'or', 'in', 'an', 'ebony', 'and', 'ivory', 'brand', 'containing', 'a', 'mixture', 'of', 'all', 'black', '(black', 'paper', 'and', 'tow)', 'and', 'ail', 'white', 'cigarettes.', '499150498']\nAnswer:  T.F. Riehl\nstart_index 17\nend_index 18\n```\n\nOnce examples are encoded, however, they will look like this:\n\n```py\n>>> encoding = tokenizer(example[\"question\"], example[\"words\"], example[\"boxes\"])\n>>> tokenizer.decode(encoding[\"input_ids\"])\n[CLS] who is in cc in this letter? [SEP] wie baw brown & williamson tobacco corporation research & development ...\n```\n\nWe'll need to find the position of the answer in the encoded input."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: a71a6321-cc61-4629-81ac-46390ef95855
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"a71a6321-cc61-4629-81ac-46390ef95855\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"b12f4735-ab4a-4bfe-a826-c185e49a8f1c\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"445b0700783de8e079d7d1f0f60e15ad6ec36eacaadb511009808ffd7d2f4720\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"792171e3-87ae-45fd-90ff-51dbf83a9118\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"54858536e85b1941144a1b1dd7af4ca00dafcc251b84a08e6b6410c9b52b4338\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "', '\u00abextremely', 'fast', 'buming', 'cigarette.', '\u00abnovel', 'cigarette', 'constructions', 'that', 'permit', 'a', 'significant', 'reduction', 'iretobacco', 'weight', 'while', 'maintaining', 'smoking', 'mechanics', 'and', 'visual', 'characteristics.', 'higher', 'basis', 'weight', 'paper:', 'potential', 'reduction', 'in', 'tobacco', 'weight.', '\u00abmore', 'rigid', 'tobacco', 'column;', 'stiffing', 'agent', 'for', 'tobacco;', 'e.g.', 'starch', '*colored', 'tow', 'and', 'cigarette', 'papers;', 'seasonal', 'promotions,', 'e.g.', 'pastel', 'colored', 'cigarettes', 'for', 'easter', 'or', 'in', 'an', 'ebony', 'and', 'ivory', 'brand', 'containing', 'a', 'mixture', 'of', 'all', 'black', '(black', 'paper', 'and', 'tow)', 'and', 'ail', 'white', 'cigarettes.', '499150498']\nAnswer:  T.F. Riehl\nstart_index 17\nend_index 18\n```\n\nOnce examples are encoded, however, they will look like this:\n\n```py\n>>> encoding = tokenizer(example[\"question\"], example[\"words\"], example[\"boxes\"])\n>>> tokenizer.decode(encoding[\"input_ids\"])\n[CLS] who is in cc in this letter? [SEP] wie baw brown & williamson tobacco corporation research & development ...\n```\n\nWe'll need to find the position of the answer in the encoded input.\n* `token_type_ids` tells us which tokens are part of the question, and which ones are part of the document's words.\n* `tokenizer.cls_token_id` will help find the special token at the beginning of the input.\n* `word_ids` will help match the answer found in the original `words` to the same answer in the full encoded input and determine\nthe start/end position of the answer in the encoded input.\n\nWith that in mind, let's create a function to encode a batch of examples in the dataset:\n\n```py\n>>> def encode_dataset(examples, max_length=512):\n.     questions = examples[\"question\"]\n.     words = examples[\"words\"]\n.     boxes = examples[\"boxes\"]\n.     answers = examples[\"answer\"]\n\n.     # encode the batch of examples and initialize the start_positions and end_positions\n.     encoding = tokenizer(questions, words, boxes, max_length=max_length, padding=\"max_length\", truncation=True)\n.     start_positions = []\n.     end_positions = []\n\n.     # loop through the examples in the batch\n.     for i in range(len(questions)):\n.         cls_index = encoding[\"input_ids\"][i].index(tokenizer.cls_token_id)\n\n.         # find the position of the answer in example's words\n.         words_example = [word.lower() for word in words[i]]\n.         answer = answers[i]\n.         match, word_idx_start, word_idx_end = subfinder(words_example, answer.lower().split())\n\n.         if match:\n.             # if match is found, use `token_type_ids` to find where words start in the encoding\n.             token_type_ids = encoding[\"token_type_ids\"][i]\n.             token_start_index = 0\n.             while token_type_ids[token_start_index] != 1:\n.                 token_start_index += 1\n\n."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 792171e3-87ae-45fd-90ff-51dbf83a9118
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"792171e3-87ae-45fd-90ff-51dbf83a9118\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"a71a6321-cc61-4629-81ac-46390ef95855\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"49245f42e3ccf604f1e0b3d67d84b3079aed4545152c6946f0708f4ccd7d3a9a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4d87a9a6-5055-45a5-b7cb-b6801882f0b5\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c927672daa6187c26f9de6e8cbcd69667d0de0f3c3bdca2b4b5c19d3cfa5e200\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "* `token_type_ids` tells us which tokens are part of the question, and which ones are part of the document's words.\n* `tokenizer.cls_token_id` will help find the special token at the beginning of the input.\n* `word_ids` will help match the answer found in the original `words` to the same answer in the full encoded input and determine\nthe start/end position of the answer in the encoded input.\n\nWith that in mind, let's create a function to encode a batch of examples in the dataset:\n\n```py\n>>> def encode_dataset(examples, max_length=512):\n.     questions = examples[\"question\"]\n.     words = examples[\"words\"]\n.     boxes = examples[\"boxes\"]\n.     answers = examples[\"answer\"]\n\n.     # encode the batch of examples and initialize the start_positions and end_positions\n.     encoding = tokenizer(questions, words, boxes, max_length=max_length, padding=\"max_length\", truncation=True)\n.     start_positions = []\n.     end_positions = []\n\n.     # loop through the examples in the batch\n.     for i in range(len(questions)):\n.         cls_index = encoding[\"input_ids\"][i].index(tokenizer.cls_token_id)\n\n.         # find the position of the answer in example's words\n.         words_example = [word.lower() for word in words[i]]\n.         answer = answers[i]\n.         match, word_idx_start, word_idx_end = subfinder(words_example, answer.lower().split())\n\n.         if match:\n.             # if match is found, use `token_type_ids` to find where words start in the encoding\n.             token_type_ids = encoding[\"token_type_ids\"][i]\n.             token_start_index = 0\n.             while token_type_ids[token_start_index] != 1:\n.                 token_start_index += 1\n\n.             token_end_index = len(encoding[\"input_ids\"][i]) - 1\n.             while token_type_ids[token_end_index] != 1:\n.                 token_end_index -= 1\n\n.             word_ids = encoding.word_ids(i)[token_start_index : token_end_index + 1]\n.             start_position = cls_index\n.             end_position = cls_index\n\n.             # loop over word_ids and increase `token_start_index` until it matches the answer position in words\n.             # once it matches, save the `token_start_index` as the `start_position` of the answer in the encoding\n.             for id in word_ids:\n.                 if id == word_idx_start:\n.                     start_position = token_start_index\n.                 else:\n.                     token_start_index += 1\n\n.             # similarly loop over `word_ids` starting from the end to find the `end_position` of the answer\n.             for id in word_ids[::-1]:\n.                 if id == word_idx_end:\n.                     end_position = token_end_index\n.                 else:\n.                     token_end_index -= 1\n\n.             start_positions.append(start_position)\n.             end_positions.append(end_position)\n\n.         else:\n.             start_positions.append(cls_index)\n.             end_positions.append(cls_index)\n\n.     encoding[\"image\"] = examples[\"image\"]\n.     encoding[\"start_positions\"] = start_positions\n.     encoding[\"end_positions\"] = end_positions\n\n.     return encoding\n```\n\nNow that we have this preprocessing function, we can encode the entire dataset:\n\n```py\n>>> encoded_train_dataset = dataset_with_ocr[\"train\"].map(\n.     encode_dataset, batched=True, batch_size=2, remove_columns=dataset_with_ocr[\"train\"].column_names\n. )\n>>> encoded_test_dataset = dataset_with_ocr[\"test\"].map(\n.     encode_dataset,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4d87a9a6-5055-45a5-b7cb-b6801882f0b5
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4d87a9a6-5055-45a5-b7cb-b6801882f0b5\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"792171e3-87ae-45fd-90ff-51dbf83a9118\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"40ff8c00e45ca710b6202f1d4e8dd710a181554caeeafaaf720d67c14534d2ad\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"310bc914-ce2d-4a9e-81a0-82d781790bd2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"5aaf0084b882778a32ed68bbac54f34aee5df2b88aa330d7e5bfd78ae2a700e8\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "token_end_index = len(encoding[\"input_ids\"][i]) - 1\n.             while token_type_ids[token_end_index] != 1:\n.                 token_end_index -= 1\n\n.             word_ids = encoding.word_ids(i)[token_start_index : token_end_index + 1]\n.             start_position = cls_index\n.             end_position = cls_index\n\n.             # loop over word_ids and increase `token_start_index` until it matches the answer position in words\n.             # once it matches, save the `token_start_index` as the `start_position` of the answer in the encoding\n.             for id in word_ids:\n.                 if id == word_idx_start:\n.                     start_position = token_start_index\n.                 else:\n.                     token_start_index += 1\n\n.             # similarly loop over `word_ids` starting from the end to find the `end_position` of the answer\n.             for id in word_ids[::-1]:\n.                 if id == word_idx_end:\n.                     end_position = token_end_index\n.                 else:\n.                     token_end_index -= 1\n\n.             start_positions.append(start_position)\n.             end_positions.append(end_position)\n\n.         else:\n.             start_positions.append(cls_index)\n.             end_positions.append(cls_index)\n\n.     encoding[\"image\"] = examples[\"image\"]\n.     encoding[\"start_positions\"] = start_positions\n.     encoding[\"end_positions\"] = end_positions\n\n.     return encoding\n```\n\nNow that we have this preprocessing function, we can encode the entire dataset:\n\n```py\n>>> encoded_train_dataset = dataset_with_ocr[\"train\"].map(\n.     encode_dataset, batched=True, batch_size=2, remove_columns=dataset_with_ocr[\"train\"].column_names\n. )\n>>> encoded_test_dataset = dataset_with_ocr[\"test\"].map(\n.     encode_dataset, batched=True, batch_size=2, remove_columns=dataset_with_ocr[\"test\"].column_names\n. )\n```\n\nLet's check what the features of the encoded dataset look like:\n\n```py\n>>> encoded_train_dataset.features\n{'image': Sequence(feature=Sequence(feature=Sequence(feature=Value(dtype='uint8', id=None), length=-1, id=None), length=-1, id=None), length=-1, id=None),\n 'input_ids': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None),\n 'token_type_ids': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),\n 'attention_mask': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),\n 'bbox': Sequence(feature=Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), length=-1, id=None),\n 'start_positions': Value(dtype='int64', id=None),\n 'end_positions': Value(dtype='int64', id=None)}\n```\n\n## Evaluation\n\nEvaluation for document question answering requires a significant amount of postprocessing. To avoid taking up too much\nof your time, this guide skips the evaluation step. The [`Trainer`] still calculates the evaluation loss during training so\nyou're not completely in the dark about your model's performance. Extractive question answering is typically evaluated using F1/exact match.\nIf you'd like to implement it yourself, check out the [Question Answering chapter](https://huggingface.co/course/chapter7/7?fw=pt#postprocessing)\nof the Hugging Face course for inspiration.\n\n## Train\n\nCongratulations! You've successfully navigated the toughest part of this guide and now you are ready to train your own model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 310bc914-ce2d-4a9e-81a0-82d781790bd2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"310bc914-ce2d-4a9e-81a0-82d781790bd2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4d87a9a6-5055-45a5-b7cb-b6801882f0b5\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"22ba5c29400ec391ce9a4faeae571c4030cd38b756a78b1ff71de192c1313c52\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"2a96325b-ec4e-41f9-a8f3-dc9aac74f75b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"2e1b30a09304fb5cc22c2f9e4cdf2223944f6f2965c3355c478a475343182708\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": null, \"end_char_idx\": null, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "column_names\n. )\n>>> encoded_test_dataset = dataset_with_ocr[\"test\"].map(\n.     encode_dataset, batched=True, batch_size=2, remove_columns=dataset_with_ocr[\"test\"].column_names\n. )\n```\n\nLet's check what the features of the encoded dataset look like:\n\n```py\n>>> encoded_train_dataset.features\n{'image': Sequence(feature=Sequence(feature=Sequence(feature=Value(dtype='uint8', id=None), length=-1, id=None), length=-1, id=None), length=-1, id=None),\n 'input_ids': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None),\n 'token_type_ids': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),\n 'attention_mask': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),\n 'bbox': Sequence(feature=Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), length=-1, id=None),\n 'start_positions': Value(dtype='int64', id=None),\n 'end_positions': Value(dtype='int64', id=None)}\n```\n\n## Evaluation\n\nEvaluation for document question answering requires a significant amount of postprocessing. To avoid taking up too much\nof your time, this guide skips the evaluation step. The [`Trainer`] still calculates the evaluation loss during training so\nyou're not completely in the dark about your model's performance. Extractive question answering is typically evaluated using F1/exact match.\nIf you'd like to implement it yourself, check out the [Question Answering chapter](https://huggingface.co/course/chapter7/7?fw=pt#postprocessing)\nof the Hugging Face course for inspiration.\n\n## Train\n\nCongratulations! You've successfully navigated the toughest part of this guide and now you are ready to train your own model.\nTraining involves the following steps:\n* Load the model with [`AutoModelForDocumentQuestionAnswering`] using the same checkpoint as in the preprocessing.\n* Define your training hyperparameters in [`TrainingArguments`].\n* Define a function to batch examples together, here the [`DefaultDataCollator`] will do just fine\n* Pass the training arguments to [`Trainer`] along with the model, dataset, and data collator.\n* Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> from transformers import AutoModelForDocumentQuestionAnswering\n\n>>> model = AutoModelForDocumentQuestionAnswering.from_pretrained(model_checkpoint)\n```\n\nIn the [`TrainingArguments`] use `output_dir` to specify where to save your model, and configure hyperparameters as you see fit.\nIf you wish to share your model with the community, set `push_to_hub` to `True` (you must be signed in to Hugging Face to upload your model).\nIn this case the `output_dir` will also be the name of the repo where your model checkpoint will be pushed.\n\n```py\n>>> from transformers import TrainingArguments\n\n>>> # REPLACE THIS WITH YOUR REPO ID\n>>> repo_id = \"MariaK/layoutlmv2-base-uncased_finetuned_docvqa\"\n\n>>> training_args = TrainingArguments(\n...     output_dir=repo_id,\n...     per_device_train_batch_size=4,\n...     num_train_epochs=20,\n...     save_steps=200,\n...     logging_steps=50,\n...     eval_strategy=\"steps\",\n...     learning_rate=5e-5,\n...     save_total_limit=2,\n...     remove_unused_columns=False,\n...     push_to_hub=True,\n... )\n```\n\nDefine a simple data collator to batch examples together."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 2a96325b-ec4e-41f9-a8f3-dc9aac74f75b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"2a96325b-ec4e-41f9-a8f3-dc9aac74f75b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"310bc914-ce2d-4a9e-81a0-82d781790bd2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"55cab5a5f987f1deb3fc3c95567cce47e47407959d4e64d1915b7f478810e242\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"44833d75-21df-4409-a7ff-cf54d202bd71\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"22529ad00d60bfd6d24635cffbca33838f8fe8329dcc8e26d21e8de59a4d3278\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18719, \"end_char_idx\": 21662, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "## Train\n\nCongratulations! You've successfully navigated the toughest part of this guide and now you are ready to train your own model.\nTraining involves the following steps:\n* Load the model with [`AutoModelForDocumentQuestionAnswering`] using the same checkpoint as in the preprocessing.\n* Define your training hyperparameters in [`TrainingArguments`].\n* Define a function to batch examples together, here the [`DefaultDataCollator`] will do just fine\n* Pass the training arguments to [`Trainer`] along with the model, dataset, and data collator.\n* Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> from transformers import AutoModelForDocumentQuestionAnswering\n\n>>> model = AutoModelForDocumentQuestionAnswering.from_pretrained(model_checkpoint)\n```\n\nIn the [`TrainingArguments`] use `output_dir` to specify where to save your model, and configure hyperparameters as you see fit.\nIf you wish to share your model with the community, set `push_to_hub` to `True` (you must be signed in to Hugging Face to upload your model).\nIn this case the `output_dir` will also be the name of the repo where your model checkpoint will be pushed.\n\n```py\n>>> from transformers import TrainingArguments\n\n>>> # REPLACE THIS WITH YOUR REPO ID\n>>> repo_id = \"MariaK/layoutlmv2-base-uncased_finetuned_docvqa\"\n\n>>> training_args = TrainingArguments(\n...     output_dir=repo_id,\n...     per_device_train_batch_size=4,\n...     num_train_epochs=20,\n...     save_steps=200,\n...     logging_steps=50,\n...     eval_strategy=\"steps\",\n...     learning_rate=5e-5,\n...     save_total_limit=2,\n...     remove_unused_columns=False,\n...     push_to_hub=True,\n... )\n```\n\nDefine a simple data collator to batch examples together.\n\n```py\n>>> from transformers import DefaultDataCollator\n\n>>> data_collator = DefaultDataCollator()\n```\n\nFinally, bring everything together, and call [`~Trainer.train`]:\n\n```py\n>>> from transformers import Trainer\n\n>>> trainer = Trainer(\n...     model=model,\n...     args=training_args,\n...     data_collator=data_collator,\n...     train_dataset=encoded_train_dataset,\n...     eval_dataset=encoded_test_dataset,\n...     tokenizer=processor,\n... )\n\n>>> trainer.train()\n```\n\nTo add the final model to \ud83e\udd17 Hub, create a model card and call `push_to_hub`:\n\n```py\n>>> trainer.create_model_card()\n>>> trainer.push_to_hub()\n```\n\n## Inference\n\nNow that you have finetuned a LayoutLMv2 model, and uploaded it to the \ud83e\udd17 Hub, you can use it for inference. The simplest\nway to try out your finetuned model for inference is to use it in a [`Pipeline`].\n\nLet's take an example:\n```py\n>>> example = dataset[\"test\"][2]\n>>> question = example[\"query\"][\"en\"]\n>>> image = example[\"image\"]\n>>> print(question)\n>>> print(example[\"answers\"])\n'Who is \u2018presiding\u2019 TRRF GENERAL SESSION (PART 1)?'\n['TRRF Vice President', 'lee a. waller']\n```\n\nNext, instantiate a pipeline for\ndocument question answering with your model, and pass the image + question combination to it."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 44833d75-21df-4409-a7ff-cf54d202bd71
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/document_question_answering", "title": "Document Question Answering", "tokens": 5943, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"44833d75-21df-4409-a7ff-cf54d202bd71\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"7ca9a770-dab0-4685-8db8-c0f2f122c489\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"bd3d869f99ff537c3e08ebaf3d945e0a111a2f510fea93f42e25a291804ea986\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"2a96325b-ec4e-41f9-a8f3-dc9aac74f75b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/document_question_answering\", \"title\": \"Document Question Answering\", \"tokens\": 5943, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"ed4be3b966dfacd1bb06d0e0d2fdaa77b8b0225c42645422d1029ff3f0c37bea\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 20423, \"end_char_idx\": 23379, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "ref_doc_id": "7ca9a770-dab0-4685-8db8-c0f2f122c489", "chroma:document": "```py\n>>> from transformers import DefaultDataCollator\n\n>>> data_collator = DefaultDataCollator()\n```\n\nFinally, bring everything together, and call [`~Trainer.train`]:\n\n```py\n>>> from transformers import Trainer\n\n>>> trainer = Trainer(\n...     model=model,\n...     args=training_args,\n...     data_collator=data_collator,\n...     train_dataset=encoded_train_dataset,\n...     eval_dataset=encoded_test_dataset,\n...     tokenizer=processor,\n... )\n\n>>> trainer.train()\n```\n\nTo add the final model to \ud83e\udd17 Hub, create a model card and call `push_to_hub`:\n\n```py\n>>> trainer.create_model_card()\n>>> trainer.push_to_hub()\n```\n\n## Inference\n\nNow that you have finetuned a LayoutLMv2 model, and uploaded it to the \ud83e\udd17 Hub, you can use it for inference. The simplest\nway to try out your finetuned model for inference is to use it in a [`Pipeline`].\n\nLet's take an example:\n```py\n>>> example = dataset[\"test\"][2]\n>>> question = example[\"query\"][\"en\"]\n>>> image = example[\"image\"]\n>>> print(question)\n>>> print(example[\"answers\"])\n'Who is \u2018presiding\u2019 TRRF GENERAL SESSION (PART 1)?'\n['TRRF Vice President', 'lee a. waller']\n```\n\nNext, instantiate a pipeline for\ndocument question answering with your model, and pass the image + question combination to it.\n\n```py\n>>> from transformers import pipeline\n\n>>> qa_pipeline = pipeline(\"document-question-answering\", model=\"MariaK/layoutlmv2-base-uncased_finetuned_docvqa\")\n>>> qa_pipeline(image, question)\n[{'score': 0.9949808120727539,\n  'answer': 'Lee A. Waller',\n  'start': 55,\n  'end': 57}]\n```\n\nYou can also manually replicate the results of the pipeline if you'd like:\n1. Take an image and a question, prepare them for the model using the processor from your model.\n2. Forward the result or preprocessing through the model.\n3. The model returns `start_logits` and `end_logits`, which indicate which token is at the start of the answer and\nwhich token is at the end of the answer. Both have shape (batch_size, sequence_length).\n4. Take an argmax on the last dimension of both the `start_logits` and `end_logits` to get the predicted `start_idx` and `end_idx`.\n5. Decode the answer with the tokenizer.\n\n```py\n>>> import torch\n>>> from transformers import AutoProcessor\n>>> from transformers import AutoModelForDocumentQuestionAnswering\n\n>>> processor = AutoProcessor.from_pretrained(\"MariaK/layoutlmv2-base-uncased_finetuned_docvqa\")\n>>> model = AutoModelForDocumentQuestionAnswering.from_pretrained(\"MariaK/layoutlmv2-base-uncased_finetuned_docvqa\")\n\n>>> with torch.no_grad():\n...     encoding = processor(image.convert(\"RGB\"), question, return_tensors=\"pt\")\n...     outputs = model(**encoding)\n...     start_logits = outputs.start_logits\n...     end_logits = outputs.end_logits\n...     predicted_start_idx = start_logits.argmax(-1).item()\n...     predicted_end_idx = end_logits.argmax(-1).item()\n\n>>> processor.tokenizer.decode(encoding.input_ids.squeeze()[predicted_start_idx : predicted_end_idx + 1])\n'lee a. waller'\n```"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 9e2b8155-7697-43e9-898e-5c921e08fdbd
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"9e2b8155-7697-43e9-898e-5c921e08fdbd\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"aaac8269-9142-412a-a3c5-5f71bbb50262\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"23212da2355ce05c4a828ec946ef7f3ba3f0a16cbfa6a29ddbe3703169d54c2b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3066, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "# Image tasks with IDEFICS\n\n[[open-in-colab]]\n\nWhile individual tasks can be tackled by fine-tuning specialized models, an alternative approach \nthat has recently emerged and gained popularity is to use large models for a diverse set of tasks without fine-tuning. \nFor instance, large language models can handle such NLP tasks as summarization, translation, classification, and more. \nThis approach is no longer limited to a single modality, such as text, and in this guide, we will illustrate how you can \nsolve image-text tasks with a large multimodal model called IDEFICS. \n\n[IDEFICS](../model_doc/idefics) is an open-access vision and language model based on [Flamingo](https://huggingface.co/papers/2204.14198), \na state-of-the-art visual language model initially developed by DeepMind. The model accepts arbitrary sequences of image \nand text inputs and generates coherent text as output. It can answer questions about images, describe visual content, \ncreate stories grounded in multiple images, and so on. IDEFICS comes in two variants - [80 billion parameters](https://huggingface.co/HuggingFaceM4/idefics-80b) \nand [9 billion parameters](https://huggingface.co/HuggingFaceM4/idefics-9b), both of which are available on the \ud83e\udd17 Hub. For each variant, you can also find fine-tuned instructed \nversions of the model adapted for conversational use cases.\n\nThis model is exceptionally versatile and can be used for a wide range of image and multimodal tasks. However, \nbeing a large model means it requires significant computational resources and infrastructure. It is up to you to decide whether \nthis approach suits your use case better than fine-tuning specialized models for each individual task. \n\nIn this guide, you'll learn how to: \n- [Load IDEFICS](#loading-the-model) and [load the quantized version of the model](#quantized-model)\n- Use IDEFICS for: \n  - [Image captioning](#image-captioning)\n  - [Prompted image captioning](#prompted-image-captioning)\n  - [Few-shot prompting](#few-shot-prompting)\n  - [Visual question answering](#visual-question-answering)\n  - [Image classification](#image-classification)\n  - [Image-guided text generation](#image-guided-text-generation)\n- [Run inference in batch mode](#running-inference-in-batch-mode)\n- [Run IDEFICS instruct for conversational use](#idefics-instruct-for-conversational-use)\n\nBefore you begin, make sure you have all the necessary libraries installed. \n\n```bash\npip install -q bitsandbytes sentencepiece accelerate transformers\n```\n\n<Tip>\nTo run the following examples with a non-quantized version of the model checkpoint you will need at least 20GB of GPU memory.\n</Tip>\n\n## Loading the model\n\nLet's start by loading the model's 9 billion parameters checkpoint: \n\n```py\n>>> checkpoint = \"HuggingFaceM4/idefics-9b\"\n```\n\nJust like for other Transformers models, you need to load a processor and the model itself from the checkpoint. \nThe IDEFICS processor wraps a [`LlamaTokenizer`] and IDEFICS image processor into a single processor to take care of \npreparing text and image inputs for the model."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: aaac8269-9142-412a-a3c5-5f71bbb50262
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"aaac8269-9142-412a-a3c5-5f71bbb50262\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"9e2b8155-7697-43e9-898e-5c921e08fdbd\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"f8860b38887097e25f54e5b9ee05863af376245773043139a77bf4d4176bb075\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"8ee9ee0a-4dda-4115-8e3e-ebfc20fdecab\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"0e3a1787316ae602c29724caaa88031d9c8349d04c11dd73e2d51ae20a66c287\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1462, \"end_char_idx\": 4670, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "However, \nbeing a large model means it requires significant computational resources and infrastructure. It is up to you to decide whether \nthis approach suits your use case better than fine-tuning specialized models for each individual task. \n\nIn this guide, you'll learn how to: \n- [Load IDEFICS](#loading-the-model) and [load the quantized version of the model](#quantized-model)\n- Use IDEFICS for: \n  - [Image captioning](#image-captioning)\n  - [Prompted image captioning](#prompted-image-captioning)\n  - [Few-shot prompting](#few-shot-prompting)\n  - [Visual question answering](#visual-question-answering)\n  - [Image classification](#image-classification)\n  - [Image-guided text generation](#image-guided-text-generation)\n- [Run inference in batch mode](#running-inference-in-batch-mode)\n- [Run IDEFICS instruct for conversational use](#idefics-instruct-for-conversational-use)\n\nBefore you begin, make sure you have all the necessary libraries installed. \n\n```bash\npip install -q bitsandbytes sentencepiece accelerate transformers\n```\n\n<Tip>\nTo run the following examples with a non-quantized version of the model checkpoint you will need at least 20GB of GPU memory.\n</Tip>\n\n## Loading the model\n\nLet's start by loading the model's 9 billion parameters checkpoint: \n\n```py\n>>> checkpoint = \"HuggingFaceM4/idefics-9b\"\n```\n\nJust like for other Transformers models, you need to load a processor and the model itself from the checkpoint. \nThe IDEFICS processor wraps a [`LlamaTokenizer`] and IDEFICS image processor into a single processor to take care of \npreparing text and image inputs for the model.\n\n```py\n>>> import torch\n\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor\n\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> model = IdeficsForVisionText2Text.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, device_map=\"auto\")\n```\n\nSetting `device_map` to `\"auto\"` will automatically determine how to load and store the model weights in the most optimized \nmanner given existing devices.\n\n### Quantized model\n\nIf high-memory GPU availability is an issue, you can load the quantized version of the model. To load the model and the \nprocessor in 4bit precision, pass a `BitsAndBytesConfig` to the `from_pretrained` method and the model will be compressed \non the fly while loading.\n\n```py\n>>> import torch\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor, BitsAndBytesConfig\n\n>>> quantization_config = BitsAndBytesConfig(\n...     load_in_4bit=True,\n...     bnb_4bit_compute_dtype=torch.float16,\n... )\n\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> model = IdeficsForVisionText2Text.from_pretrained(\n...     checkpoint,\n...     quantization_config=quantization_config,\n...     device_map=\"auto\"\n... )\n```\n\nNow that you have the model loaded in one of the suggested ways, let's move on to exploring tasks that you can use IDEFICS for.\n\n## Image captioning\nImage captioning is the task of predicting a caption for a given image. A common application is to aid visually impaired \npeople navigate through different situations, for instance, explore image content online. \n\nTo illustrate the task, get an image to be captioned, e.g."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 8ee9ee0a-4dda-4115-8e3e-ebfc20fdecab
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"8ee9ee0a-4dda-4115-8e3e-ebfc20fdecab\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"aaac8269-9142-412a-a3c5-5f71bbb50262\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b8695312d2ea15464b66d0686cc53131a07384dc40bb8ed7ee2c65462f60e135\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"88c44e75-908c-461c-bd48-8ea782d40eaf\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"6314744aa142dec6c5699d8efa4a2c87b252929610d77a0eb3e9fa5fd71c0a4b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3068, \"end_char_idx\": 5366, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "```py\n>>> import torch\n\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor\n\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> model = IdeficsForVisionText2Text.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, device_map=\"auto\")\n```\n\nSetting `device_map` to `\"auto\"` will automatically determine how to load and store the model weights in the most optimized \nmanner given existing devices.\n\n### Quantized model\n\nIf high-memory GPU availability is an issue, you can load the quantized version of the model. To load the model and the \nprocessor in 4bit precision, pass a `BitsAndBytesConfig` to the `from_pretrained` method and the model will be compressed \non the fly while loading.\n\n```py\n>>> import torch\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor, BitsAndBytesConfig\n\n>>> quantization_config = BitsAndBytesConfig(\n...     load_in_4bit=True,\n...     bnb_4bit_compute_dtype=torch.float16,\n... )\n\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> model = IdeficsForVisionText2Text.from_pretrained(\n...     checkpoint,\n...     quantization_config=quantization_config,\n...     device_map=\"auto\"\n... )\n```\n\nNow that you have the model loaded in one of the suggested ways, let's move on to exploring tasks that you can use IDEFICS for.\n\n## Image captioning\nImage captioning is the task of predicting a caption for a given image. A common application is to aid visually impaired \npeople navigate through different situations, for instance, explore image content online. \n\nTo illustrate the task, get an image to be captioned, e.g.:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-im-captioning.jpg\" alt=\"Image of a puppy in a flower bed\"/>\n</div>\n\nPhoto by [Hendo Wang](https://unsplash.com/@hendoo). \n\nIDEFICS accepts text and image prompts. However, to caption an image, you do not have to provide a text prompt to the \nmodel, only the preprocessed input image. Without a text prompt, the model will start generating text from the \nBOS (beginning-of-sequence) token thus creating a caption.\n\nAs image input to the model, you can use either an image object (`PIL.Image`) or a url from which the image can be retrieved."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 88c44e75-908c-461c-bd48-8ea782d40eaf
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"88c44e75-908c-461c-bd48-8ea782d40eaf\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8ee9ee0a-4dda-4115-8e3e-ebfc20fdecab\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"5b4f5570c180f7bf576e64b5461c69788eb727d15f0b2b85151b95c023ab0b7d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"88db9bf4-44a4-443b-8ff9-8301fc56f5f2\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"65ea3011d628dee36c517d306deb9deeaf604bef2ac68d605ca3af5917cb66a1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3789, \"end_char_idx\": 6581, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "```py\n>>> import torch\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor, BitsAndBytesConfig\n\n>>> quantization_config = BitsAndBytesConfig(\n...     load_in_4bit=True,\n...     bnb_4bit_compute_dtype=torch.float16,\n... )\n\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> model = IdeficsForVisionText2Text.from_pretrained(\n...     checkpoint,\n...     quantization_config=quantization_config,\n...     device_map=\"auto\"\n... )\n```\n\nNow that you have the model loaded in one of the suggested ways, let's move on to exploring tasks that you can use IDEFICS for.\n\n## Image captioning\nImage captioning is the task of predicting a caption for a given image. A common application is to aid visually impaired \npeople navigate through different situations, for instance, explore image content online. \n\nTo illustrate the task, get an image to be captioned, e.g.:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-im-captioning.jpg\" alt=\"Image of a puppy in a flower bed\"/>\n</div>\n\nPhoto by [Hendo Wang](https://unsplash.com/@hendoo). \n\nIDEFICS accepts text and image prompts. However, to caption an image, you do not have to provide a text prompt to the \nmodel, only the preprocessed input image. Without a text prompt, the model will start generating text from the \nBOS (beginning-of-sequence) token thus creating a caption.\n\nAs image input to the model, you can use either an image object (`PIL.Image`) or a url from which the image can be retrieved.\n\n```py\n>>> prompt = [\n...     \"https://images.unsplash.com/photo-1583160247711-2191776b4b91?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3542&q=80\",\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=10, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nA puppy in a flower bed\n```\n\n<Tip>\n\nIt is a good idea to include the `bad_words_ids` in the call to `generate` to avoid errors arising when increasing \nthe `max_new_tokens`: the model will want to generate a new `<image>` or `<fake_token_around_image>` token when there \nis no image being generated by the model.\nYou can set it on-the-fly as in this guide, or store in the `GenerationConfig` as described in the [Text generation strategies](../generation_strategies) guide.\n</Tip>\n\n## Prompted image captioning\n\nYou can extend image captioning by providing a text prompt, which the model will continue given the image."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 88db9bf4-44a4-443b-8ff9-8301fc56f5f2
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"88db9bf4-44a4-443b-8ff9-8301fc56f5f2\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"88c44e75-908c-461c-bd48-8ea782d40eaf\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"8b21268d119b02729689bc0fee05003f69fe679e255c0c1cac8f5ea07c3c3daa\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"dd0c6638-6c3e-49c5-9f36-ce73fa9df0d3\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"654548ee4c21d47b1c2af7c733295e93ef9e27c3a9323b1a09b738509de0df7a\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5111, \"end_char_idx\": 7991, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Without a text prompt, the model will start generating text from the \nBOS (beginning-of-sequence) token thus creating a caption.\n\nAs image input to the model, you can use either an image object (`PIL.Image`) or a url from which the image can be retrieved.\n\n```py\n>>> prompt = [\n...     \"https://images.unsplash.com/photo-1583160247711-2191776b4b91?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3542&q=80\",\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=10, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nA puppy in a flower bed\n```\n\n<Tip>\n\nIt is a good idea to include the `bad_words_ids` in the call to `generate` to avoid errors arising when increasing \nthe `max_new_tokens`: the model will want to generate a new `<image>` or `<fake_token_around_image>` token when there \nis no image being generated by the model.\nYou can set it on-the-fly as in this guide, or store in the `GenerationConfig` as described in the [Text generation strategies](../generation_strategies) guide.\n</Tip>\n\n## Prompted image captioning\n\nYou can extend image captioning by providing a text prompt, which the model will continue given the image. Let's take \nanother image to illustrate:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-prompted-im-captioning.jpg\" alt=\"Image of the Eiffel Tower at night\"/>\n</div>\n\nPhoto by [Denys Nevozhai](https://unsplash.com/@dnevozhai).\n   \nTextual and image prompts can be passed to the model's processor as a single list to create appropriate inputs.\n\n```py\n>>> prompt = [\n...     \"https://images.unsplash.com/photo-1543349689-9a4d426bee8e?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3501&q=80\",\n...     \"This is an image of \",\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=10, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nThis is an image of the Eiffel Tower in Paris, France.\n```\n\n## Few-shot prompting\n\nWhile IDEFICS demonstrates great zero-shot results, your task may require a certain format of the caption, or come with \nother restrictions or requirements that increase task's complexity. Few-shot prompting can be used to enable in-context learning."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: dd0c6638-6c3e-49c5-9f36-ce73fa9df0d3
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"dd0c6638-6c3e-49c5-9f36-ce73fa9df0d3\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"88db9bf4-44a4-443b-8ff9-8301fc56f5f2\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"fa539d43a77412145ceab596d8890de2036b120262a1fa2bb51ad0eacce7e8c2\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"68d39bae-1a7c-4cda-98d8-a39abd151386\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"da22b7ba743231e97503e1c29f8db4dea794aabfdcdcc7948f3d4cd4e7218d15\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 6582, \"end_char_idx\": 9041, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Let's take \nanother image to illustrate:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-prompted-im-captioning.jpg\" alt=\"Image of the Eiffel Tower at night\"/>\n</div>\n\nPhoto by [Denys Nevozhai](https://unsplash.com/@dnevozhai).\n   \nTextual and image prompts can be passed to the model's processor as a single list to create appropriate inputs.\n\n```py\n>>> prompt = [\n...     \"https://images.unsplash.com/photo-1543349689-9a4d426bee8e?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3501&q=80\",\n...     \"This is an image of \",\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=10, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nThis is an image of the Eiffel Tower in Paris, France.\n```\n\n## Few-shot prompting\n\nWhile IDEFICS demonstrates great zero-shot results, your task may require a certain format of the caption, or come with \nother restrictions or requirements that increase task's complexity. Few-shot prompting can be used to enable in-context learning.\nBy providing examples in the prompt, you can steer the model to generate results that mimic the format of given examples. \n\nLet's use the previous image of the Eiffel Tower as an example for the model and build a prompt that demonstrates to the model \nthat in addition to learning what the object in an image is, we would also like to get some interesting information about it. \nThen, let's see, if we can get the same response format for an image of the Statue of Liberty:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-few-shot.jpg\" alt=\"Image of the Statue of Liberty\"/>\n</div>\n\nPhoto by [Juan Mayobre](https://unsplash.com/@jmayobres).\n  \n```py\n>>> prompt = [\"User:\",\n...            \"https://images.unsplash.com/photo-1543349689-9a4d426bee8e?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3501&q=80\",\n...            \"Describe this image.\\nAssistant: An image of the Eiffel Tower at night."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 68d39bae-1a7c-4cda-98d8-a39abd151386
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"68d39bae-1a7c-4cda-98d8-a39abd151386\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"dd0c6638-6c3e-49c5-9f36-ce73fa9df0d3\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"44e1da43cb969f2d6fa55a275cae6f6acbbe691e613624877436973cc4b988f2\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"65b140f3-2c26-436b-9875-381f343bed2a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"8ab4a099d58b7dd269830a0ff351a502f2e7bd7ef074339997e740a7193538d3\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7713, \"end_char_idx\": 10618, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "```\n\n## Few-shot prompting\n\nWhile IDEFICS demonstrates great zero-shot results, your task may require a certain format of the caption, or come with \nother restrictions or requirements that increase task's complexity. Few-shot prompting can be used to enable in-context learning.\nBy providing examples in the prompt, you can steer the model to generate results that mimic the format of given examples. \n\nLet's use the previous image of the Eiffel Tower as an example for the model and build a prompt that demonstrates to the model \nthat in addition to learning what the object in an image is, we would also like to get some interesting information about it. \nThen, let's see, if we can get the same response format for an image of the Statue of Liberty:\n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-few-shot.jpg\" alt=\"Image of the Statue of Liberty\"/>\n</div>\n\nPhoto by [Juan Mayobre](https://unsplash.com/@jmayobres).\n  \n```py\n>>> prompt = [\"User:\",\n...            \"https://images.unsplash.com/photo-1543349689-9a4d426bee8e?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3501&q=80\",\n...            \"Describe this image.\\nAssistant: An image of the Eiffel Tower at night. Fun fact: the Eiffel Tower is the same height as an 81-storey building.\\n\",\n...            \"User:\",\n...            \"https://images.unsplash.com/photo-1524099163253-32b7f0256868?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3387&q=80\",\n...            \"Describe this image.\\nAssistant:\"\n...            ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=30, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nUser: Describe this image.\nAssistant: An image of the Eiffel Tower at night. Fun fact: the Eiffel Tower is the same height as an 81-storey building. \nUser: Describe this image.\nAssistant: An image of the Statue of Liberty. Fun fact: the Statue of Liberty is 151 feet tall.\n```\n\nNotice that just from a single example (i.e., 1-shot) the model has learned how to perform the task. For more complex tasks, \nfeel free to experiment with a larger number of examples (e.g., 3-shot, 5-shot, etc.).\n\n## Visual question answering\n\nVisual Question Answering (VQA) is the task of answering open-ended questions based on an image. Similar to image \ncaptioning it can be used in accessibility applications, but also in education (reasoning about visual materials), customer \nservice (questions about products based on images), and image retrieval."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 65b140f3-2c26-436b-9875-381f343bed2a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"65b140f3-2c26-436b-9875-381f343bed2a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"68d39bae-1a7c-4cda-98d8-a39abd151386\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"b5fdca2836360a87e76a373fd7328ee9774efd6b7cf9ef04a9ef3697c3ffcd3a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4dd64ddd-08e1-4324-9d0c-3bc2d13a27a4\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c8f66bfa86885f3915a179717b6d0338d4942e643b5475e337780cba910cd25b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 9811, \"end_char_idx\": 12703, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Assistant: An image of the Eiffel Tower at night. Fun fact: the Eiffel Tower is the same height as an 81-storey building. \nUser: Describe this image.\nAssistant: An image of the Statue of Liberty. Fun fact: the Statue of Liberty is 151 feet tall.\n```\n\nNotice that just from a single example (i.e., 1-shot) the model has learned how to perform the task. For more complex tasks, \nfeel free to experiment with a larger number of examples (e.g., 3-shot, 5-shot, etc.).\n\n## Visual question answering\n\nVisual Question Answering (VQA) is the task of answering open-ended questions based on an image. Similar to image \ncaptioning it can be used in accessibility applications, but also in education (reasoning about visual materials), customer \nservice (questions about products based on images), and image retrieval.\n\nLet's get a new image for this task: \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-vqa.jpg\" alt=\"Image of a couple having a picnic\"/>\n</div>\n\nPhoto by [Jarritos Mexican Soda](https://unsplash.com/@jarritos). \n\nYou can steer the model from image captioning to visual question answering by prompting it with appropriate instructions: \n\n```py\n>>> prompt = [\n...     \"Instruction: Provide an answer to the question. Use the image to answer.\\n\",\n...     \"https://images.unsplash.com/photo-1623944889288-cd147dbb517c?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3540&q=80\",\n...     \"Question: Where are these people and what's the weather like? Answer:\"\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=20, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nInstruction: Provide an answer to the question. Use the image to answer.\n Question: Where are these people and what's the weather like? Answer: They're in a park in New York City, and it's a beautiful day.\n```\n\n## Image classification\n\nIDEFICS is capable of classifying images into different categories without being explicitly trained on data containing \nlabeled examples from those specific categories. Given a list of categories and using its image and text understanding \ncapabilities, the model can infer which category the image likely belongs to. \n\nSay, we have this image of a vegetable stand: \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-classification.jpg\" alt=\"Image of a vegetable stand\"/>\n</div>\n\nPhoto by [Peter Wendt](https://unsplash.com/@peterwendt)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4dd64ddd-08e1-4324-9d0c-3bc2d13a27a4
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4dd64ddd-08e1-4324-9d0c-3bc2d13a27a4\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"65b140f3-2c26-436b-9875-381f343bed2a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"89c58d9b593d46fcebc83a01668db25b75b2a65689be2029916086dc24b2e06d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"c06a8637-01e6-41f2-9e2e-3cf4ee8a9065\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"20c5eabf06d276d9cda6a51d857c804a7c0d6e1057a6c11ff9896d4510ae0363\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 11420, \"end_char_idx\": 14556, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Answer:\"\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=20, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nInstruction: Provide an answer to the question. Use the image to answer.\n Question: Where are these people and what's the weather like? Answer: They're in a park in New York City, and it's a beautiful day.\n```\n\n## Image classification\n\nIDEFICS is capable of classifying images into different categories without being explicitly trained on data containing \nlabeled examples from those specific categories. Given a list of categories and using its image and text understanding \ncapabilities, the model can infer which category the image likely belongs to. \n\nSay, we have this image of a vegetable stand: \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-classification.jpg\" alt=\"Image of a vegetable stand\"/>\n</div>\n\nPhoto by [Peter Wendt](https://unsplash.com/@peterwendt).\n\nWe can instruct the model to classify the image into one of the categories that we have:\n\n```py\n>>> categories = ['animals','vegetables', 'city landscape', 'cars', 'office']\n>>> prompt = [f\"Instruction: Classify the following image into a single category from the following list: {categories}.\\n\",\n...     \"https://images.unsplash.com/photo-1471193945509-9ad0617afabf?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3540&q=80\",    \n...     \"Category: \"\n... ]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=6, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0])\nInstruction: Classify the following image into a single category from the following list: ['animals', 'vegetables', 'city landscape', 'cars', 'office'].\nCategory: Vegetables\n```  \n\nIn the example above we instruct the model to classify the image into a single category, however, you can also prompt the model to do rank classification.\n\n## Image-guided text generation\n\nFor more creative applications, you can use image-guided text generation to generate text based on an image. This can be \nuseful to create descriptions of products, ads, descriptions of a scene, etc. \n\nLet's prompt IDEFICS to write a story based on a simple image of a red door: \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-story-generation.jpg\" alt=\"Image of a red door with a pumpkin on the steps\"/>\n</div>\n\nPhoto by [Craig Tidball](https://unsplash.com/@devonshiremedia)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: c06a8637-01e6-41f2-9e2e-3cf4ee8a9065
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"c06a8637-01e6-41f2-9e2e-3cf4ee8a9065\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4dd64ddd-08e1-4324-9d0c-3bc2d13a27a4\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"d23047bac1019360c1aeec82f0d48ac8cd687c761c4198ce78551b8b84152fd8\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"42b674f4-1a85-433f-8eae-0c51895d69f8\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"76fcaeea0553411f5c081b1fc41752115607bd0eefda7001fc7ce7e42967a99c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13754, \"end_char_idx\": 16529, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Category: Vegetables\n```  \n\nIn the example above we instruct the model to classify the image into a single category, however, you can also prompt the model to do rank classification.\n\n## Image-guided text generation\n\nFor more creative applications, you can use image-guided text generation to generate text based on an image. This can be \nuseful to create descriptions of products, ads, descriptions of a scene, etc. \n\nLet's prompt IDEFICS to write a story based on a simple image of a red door: \n\n<div class=\"flex justify-center\">\n     <img src=\"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/idefics-story-generation.jpg\" alt=\"Image of a red door with a pumpkin on the steps\"/>\n</div>\n\nPhoto by [Craig Tidball](https://unsplash.com/@devonshiremedia).\n  \n```py\n>>> prompt = [\"Instruction: Use the image to write a story. \\n\",\n...     \"https://images.unsplash.com/photo-1517086822157-2b0358e7684a?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=2203&q=80\",\n...     \"Story: \\n\"]\n\n>>> inputs = processor(prompt, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, num_beams=2, max_new_tokens=200, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> print(generated_text[0]) \nInstruction: Use the image to write a story. \n Story: \nOnce upon a time, there was a little girl who lived in a house with a red door.  She loved her red door.  It was the prettiest door in the whole world.\n\nOne day, the little girl was playing in her yard when she noticed a man standing on her doorstep.  He was wearing a long black coat and a top hat.\n\nThe little girl ran inside and told her mother about the man.\n\nHer mother said, \u201cDon\u2019t worry, honey.  He\u2019s just a friendly ghost.\u201d\n\nThe little girl wasn\u2019t sure if she believed her mother, but she went outside anyway.\n\nWhen she got to the door, the man was gone.\n\nThe next day, the little girl was playing in her yard again when she noticed the man standing on her doorstep.\n\nHe was wearing a long black coat and a top hat.\n\nThe little girl ran\n```\n\nLooks like IDEFICS noticed the pumpkin on the doorstep and went with a spooky Halloween story about a ghost.\n\n<Tip>\n\nFor longer outputs like this, you will greatly benefit from tweaking the text generation strategy. This can help \nyou significantly improve the quality of the generated output. Check out [Text generation strategies](../generation_strategies) \nto learn more. \n</Tip>\n\n## Running inference in batch mode\n\nAll of the earlier sections illustrated IDEFICS for a single example."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 42b674f4-1a85-433f-8eae-0c51895d69f8
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"42b674f4-1a85-433f-8eae-0c51895d69f8\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"c06a8637-01e6-41f2-9e2e-3cf4ee8a9065\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"8ea45a9dd57cb1c0dc7350805d9fe4b41cf003e23f85455d49f30db6fc51861f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"38d43943-4841-4ff5-becc-5df2892b1926\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"3caf6a68132596513471071bf1e0996a13ea2fc60dab38e30eb2d8189643694c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 15282, \"end_char_idx\": 17893, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "Story: \nOnce upon a time, there was a little girl who lived in a house with a red door.  She loved her red door.  It was the prettiest door in the whole world.\n\nOne day, the little girl was playing in her yard when she noticed a man standing on her doorstep.  He was wearing a long black coat and a top hat.\n\nThe little girl ran inside and told her mother about the man.\n\nHer mother said, \u201cDon\u2019t worry, honey.  He\u2019s just a friendly ghost.\u201d\n\nThe little girl wasn\u2019t sure if she believed her mother, but she went outside anyway.\n\nWhen she got to the door, the man was gone.\n\nThe next day, the little girl was playing in her yard again when she noticed the man standing on her doorstep.\n\nHe was wearing a long black coat and a top hat.\n\nThe little girl ran\n```\n\nLooks like IDEFICS noticed the pumpkin on the doorstep and went with a spooky Halloween story about a ghost.\n\n<Tip>\n\nFor longer outputs like this, you will greatly benefit from tweaking the text generation strategy. This can help \nyou significantly improve the quality of the generated output. Check out [Text generation strategies](../generation_strategies) \nto learn more. \n</Tip>\n\n## Running inference in batch mode\n\nAll of the earlier sections illustrated IDEFICS for a single example. In a very similar fashion, you can run inference \nfor a batch of examples by passing a list of prompts:\n\n```py\n>>> prompts = [\n...     [   \"https://images.unsplash.com/photo-1543349689-9a4d426bee8e?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3501&q=80\",\n...         \"This is an image of \",\n...     ],\n...     [   \"https://images.unsplash.com/photo-1623944889288-cd147dbb517c?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3540&q=80\",\n...         \"This is an image of \",\n...     ],\n...     [   \"https://images.unsplash.com/photo-1471193945509-9ad0617afabf?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=3540&q=80\",\n...         \"This is an image of \",\n...     ],\n... ]\n\n>>> inputs = processor(prompts, return_tensors=\"pt\").to(\"cuda\")\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, max_new_tokens=10, bad_words_ids=bad_words_ids)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> for i,t in enumerate(generated_text):\n...     print(f\"{i}:\\n{t}\\n\") \n0:\nThis is an image of the Eiffel Tower in Paris, France.\n\n1:\nThis is an image of a couple on a picnic blanket."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 38d43943-4841-4ff5-becc-5df2892b1926
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/idefics", "title": "Image tasks with IDEFICS", "tokens": 5432, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"38d43943-4841-4ff5-becc-5df2892b1926\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"bd6082c1-b9d1-466a-8a4b-5d5d95044237\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"3e60ba6acb9b94b0caa16c4dcf8fd1e857d79b0cb4554071b24b5fbe5185a8c9\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"42b674f4-1a85-433f-8eae-0c51895d69f8\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/idefics\", \"title\": \"Image tasks with IDEFICS\", \"tokens\": 5432, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c67678b28c19b6750e75084ff1a0c524c9cc087c53a18025d1d4555a08ae8cd2\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 17841, \"end_char_idx\": 20184, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "ref_doc_id": "bd6082c1-b9d1-466a-8a4b-5d5d95044237", "chroma:document": "1:\nThis is an image of a couple on a picnic blanket.\n\n2:\nThis is an image of a vegetable stand.\n```\n\n## IDEFICS instruct for conversational use\n\nFor conversational use cases, you can find fine-tuned instructed versions of the model on the \ud83e\udd17 Hub: \n`HuggingFaceM4/idefics-80b-instruct` and `HuggingFaceM4/idefics-9b-instruct`.\n\nThese checkpoints are the result of fine-tuning the respective base models on a mixture of supervised and instruction \nfine-tuning datasets, which boosts the downstream performance while making the models more usable in conversational settings.\n\nThe use and prompting for the conversational use is very similar to using the base models: \n\n```py\n>>> import torch\n>>> from transformers import IdeficsForVisionText2Text, AutoProcessor\n\n>>> device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n>>> checkpoint = \"HuggingFaceM4/idefics-9b-instruct\"\n>>> model = IdeficsForVisionText2Text.from_pretrained(checkpoint, torch_dtype=torch.bfloat16).to(device)\n>>> processor = AutoProcessor.from_pretrained(checkpoint)\n\n>>> prompts = [\n...     [\n...         \"User: What is in this image?\",\n...         \"https://upload.wikimedia.org/wikipedia/commons/8/86/Id%C3%A9fix.JPG\",\n...         \"<end_of_utterance>\",\n\n...         \"\\nAssistant: This picture depicts Idefix, the dog of Obelix in Asterix and Obelix. Idefix is running on the ground.<end_of_utterance>\",\n\n...         \"\\nUser:\",\n...         \"https://static.wikia.nocookie.net/asterix/images/2/25/R22b.gif/revision/latest?cb=20110815073052\",\n...         \"And who is that?<end_of_utterance>\",\n\n...         \"\\nAssistant:\",\n...     ],\n... ]\n\n>>> # --batched mode\n>>> inputs = processor(prompts, add_end_of_utterance_token=False, return_tensors=\"pt\").to(device)\n>>> # --single sample mode\n>>> # inputs = processor(prompts[0], return_tensors=\"pt\").to(device)\n\n>>> # Generation args\n>>> exit_condition = processor.tokenizer(\"<end_of_utterance>\", add_special_tokens=False).input_ids\n>>> bad_words_ids = processor.tokenizer([\"<image>\", \"<fake_token_around_image>\"], add_special_tokens=False).input_ids\n\n>>> generated_ids = model.generate(**inputs, eos_token_id=exit_condition, bad_words_ids=bad_words_ids, max_length=100)\n>>> generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)\n>>> for i, t in enumerate(generated_text):\n...     print(f\"{i}:\\n{t}\\n\")\n```"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 7a692bc0-87fe-45d9-a603-910fdff0c79f
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"7a692bc0-87fe-45d9-a603-910fdff0c79f\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"7a5fcec4-6019-48d4-b5e5-ae98b28e970a\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d69b9fbd8f133968faf60fad49c0cf076cb76dd8d11684fbd15c54d237a1639c\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 0, \"end_char_idx\": 3520, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "# Summarization\n\n[[open-in-colab]]\n\n<Youtube id=\"yHnr5Dk2zCI\"/>\n\nSummarization creates a shorter version of a document or an article that captures all the important information. Along with translation, it is another example of a task that can be formulated as a sequence-to-sequence task. Summarization can be:\n\n- Extractive: extract the most relevant information from a document.\n- Abstractive: generate new text that captures the most relevant information.\n\nThis guide will show you how to:\n\n1. Finetune [T5](https://huggingface.co/google-t5/t5-small) on the California state bill subset of the [BillSum](https://huggingface.co/datasets/billsum) dataset for abstractive summarization.\n2. Use your finetuned model for inference.\n\n<Tip>\n\nTo see all architectures and checkpoints compatible with this task, we recommend checking the [task-page](https://huggingface.co/tasks/summarization)\n\n</Tip>\n\nBefore you begin, make sure you have all the necessary libraries installed:\n\n```bash\npip install transformers datasets evaluate rouge_score\n```\n\nWe encourage you to login to your Hugging Face account so you can upload and share your model with the community. When prompted, enter your token to login:\n\n```py\n>>> from huggingface_hub import notebook_login\n\n>>> notebook_login()\n```\n\n## Load BillSum dataset\n\nStart by loading the smaller California state bill subset of the BillSum dataset from the \ud83e\udd17 Datasets library:\n\n```py\n>>> from datasets import load_dataset\n\n>>> billsum = load_dataset(\"billsum\", split=\"ca_test\")\n```\n\nSplit the dataset into a train and test set with the [`~datasets.Dataset.train_test_split`] method:\n\n```py\n>>> billsum = billsum.train_test_split(test_size=0.2)\n```\n\nThen take a look at an example:\n\n```py\n>>> billsum[\"train\"][0]\n{'summary': 'Existing law authorizes state agencies to enter into contracts for the acquisition of goods or services upon approval by the Department of General Services. Existing law sets forth various requirements and prohibitions for those contracts, including, but not limited to, a prohibition on entering into contracts for the acquisition of goods or services of $100,000 or more with a contractor that discriminates between spouses and domestic partners or same-sex and different-sex couples in the provision of benefits. Existing law provides that a contract entered into in violation of those requirements and prohibitions is void and authorizes the state or any person acting on behalf of the state to bring a civil action seeking a determination that a contract is in violation and therefore void. Under existing law, a willful violation of those requirements and prohibitions is a misdemeanor.\\nThis bill would also prohibit a state agency from entering into contracts for the acquisition of goods or services of $100,000 or more with a contractor that discriminates between employees on the basis of gender identity in the provision of benefits, as specified. By expanding the scope of a crime, this bill would impose a state-mandated local program.\\nThe California Constitution requires the state to reimburse local agencies and school districts for certain costs mandated by the state. Statutory provisions establish procedures for making that reimbursement.\\nThis bill would provide that no reimbursement is required by this act for a specified reason.',\n 'text': 'The people of the State of California do enact as follows:\\n\\n\\nSECTION 1.\\nSection 10295.35 is added to the Public Contract Code, to read:\\n10295.35.\\n(a) (1) Notwithstanding any other law,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 7a5fcec4-6019-48d4-b5e5-ae98b28e970a
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"7a5fcec4-6019-48d4-b5e5-ae98b28e970a\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"7a692bc0-87fe-45d9-a603-910fdff0c79f\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c7158908fd4c06f7316c01779fa4d818caec8dbdd842f445f658e9a49991f34e\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"6283eded-36d7-4abb-8041-61762349422b\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c9ac8e7815f0574e35d09a6dd26f2828a1c4909300a12220195a21dade7bd7da\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 1919, \"end_char_idx\": 5868, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "Existing law sets forth various requirements and prohibitions for those contracts, including, but not limited to, a prohibition on entering into contracts for the acquisition of goods or services of $100,000 or more with a contractor that discriminates between spouses and domestic partners or same-sex and different-sex couples in the provision of benefits. Existing law provides that a contract entered into in violation of those requirements and prohibitions is void and authorizes the state or any person acting on behalf of the state to bring a civil action seeking a determination that a contract is in violation and therefore void. Under existing law, a willful violation of those requirements and prohibitions is a misdemeanor.\\nThis bill would also prohibit a state agency from entering into contracts for the acquisition of goods or services of $100,000 or more with a contractor that discriminates between employees on the basis of gender identity in the provision of benefits, as specified. By expanding the scope of a crime, this bill would impose a state-mandated local program.\\nThe California Constitution requires the state to reimburse local agencies and school districts for certain costs mandated by the state. Statutory provisions establish procedures for making that reimbursement.\\nThis bill would provide that no reimbursement is required by this act for a specified reason.',\n 'text': 'The people of the State of California do enact as follows:\\n\\n\\nSECTION 1.\\nSection 10295.35 is added to the Public Contract Code, to read:\\n10295.35.\\n(a) (1) Notwithstanding any other law, a state agency shall not enter into any contract for the acquisition of goods or services in the amount of one hundred thousand dollars ($100,000) or more with a contractor that, in the provision of benefits, discriminates between employees on the basis of an employee\u2019s or dependent\u2019s actual or perceived gender identity, including, but not limited to, the employee\u2019s or dependent\u2019s identification as transgender.\\n(2) For purposes of this section, \u201ccontract\u201d includes contracts with a cumulative amount of one hundred thousand dollars ($100,000) or more per contractor in each fiscal year.\\n(3) For purposes of this section, an employee health plan is discriminatory if the plan is not consistent with Section 1365.5 of the Health and Safety Code and Section 10140 of the Insurance Code.\\n(4) The requirements of this section shall apply only to those portions of a contractor\u2019s operations that occur under any of the following conditions:\\n(A) Within the state.\\n(B) On real property outside the state if the property is owned by the state or if the state has a right to occupy the property, and if the contractor\u2019s presence at that location is connected to a contract with the state.\\n(C) Elsewhere in the United States where work related to a state contract is being performed.\\n(b) Contractors shall treat as confidential, to the maximum extent allowed by law or by the requirement of the contractor\u2019s insurance provider, any request by an employee or applicant for employment benefits or any documentation of eligibility for benefits submitted by an employee or applicant for employment.\\n(c) After taking all reasonable measures to find a contractor that complies with this section, as determined by the state agency, the requirements of this section may be waived under any of the following circumstances:\\n(1) There is only one prospective contractor willing to enter into a specific contract with the state agency.\\n(2) The contract is necessary to respond to an emergency, as determined by the state agency, that endangers the public health, welfare, or safety, or the contract is necessary for the provision of essential services, and no entity that complies with the requirements of this section capable of responding to the emergency is immediately available.\\n(3) The requirements of this section violate, or are inconsistent with,"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 6283eded-36d7-4abb-8041-61762349422b
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"6283eded-36d7-4abb-8041-61762349422b\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"7a5fcec4-6019-48d4-b5e5-ae98b28e970a\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"92b25a146fdf7a84d567208b5a108d27dc64c3515bc360a477842cc7b18abf2f\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"d1d20e9e-02ca-4664-8c2e-b2d54022898e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"d06e8648aeacf19c052162a6d4ea3dd77baecd5b96e99a2427822bf7e0b9c80b\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 3855, \"end_char_idx\": 7575, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "but not limited to, the employee\u2019s or dependent\u2019s identification as transgender.\\n(2) For purposes of this section, \u201ccontract\u201d includes contracts with a cumulative amount of one hundred thousand dollars ($100,000) or more per contractor in each fiscal year.\\n(3) For purposes of this section, an employee health plan is discriminatory if the plan is not consistent with Section 1365.5 of the Health and Safety Code and Section 10140 of the Insurance Code.\\n(4) The requirements of this section shall apply only to those portions of a contractor\u2019s operations that occur under any of the following conditions:\\n(A) Within the state.\\n(B) On real property outside the state if the property is owned by the state or if the state has a right to occupy the property, and if the contractor\u2019s presence at that location is connected to a contract with the state.\\n(C) Elsewhere in the United States where work related to a state contract is being performed.\\n(b) Contractors shall treat as confidential, to the maximum extent allowed by law or by the requirement of the contractor\u2019s insurance provider, any request by an employee or applicant for employment benefits or any documentation of eligibility for benefits submitted by an employee or applicant for employment.\\n(c) After taking all reasonable measures to find a contractor that complies with this section, as determined by the state agency, the requirements of this section may be waived under any of the following circumstances:\\n(1) There is only one prospective contractor willing to enter into a specific contract with the state agency.\\n(2) The contract is necessary to respond to an emergency, as determined by the state agency, that endangers the public health, welfare, or safety, or the contract is necessary for the provision of essential services, and no entity that complies with the requirements of this section capable of responding to the emergency is immediately available.\\n(3) The requirements of this section violate, or are inconsistent with, the terms or conditions of a grant, subvention, or agreement, if the agency has made a good faith attempt to change the terms or conditions of any grant, subvention, or agreement to authorize application of this section.\\n(4) The contractor is providing wholesale or bulk water, power, or natural gas, the conveyance or transmission of the same, or ancillary services, as required for ensuring reliable services in accordance with good utility practice, if the purchase of the same cannot practically be accomplished through the standard competitive bidding procedures and the contractor is not providing direct retail services to end users.\\n(d) (1) A contractor shall not be deemed to discriminate in the provision of benefits if the contractor, in providing the benefits, pays the actual costs incurred in obtaining the benefit.\\n(2) If a contractor is unable to provide a certain benefit, despite taking reasonable measures to do so, the contractor shall not be deemed to discriminate in the provision of benefits.\\n(e) (1) Every contract subject to this chapter shall contain a statement by which the contractor certifies that the contractor is in compliance with this section.\\n(2) The department or other contracting agency shall enforce this section pursuant to its existing enforcement powers.\\n(3) (A) If a contractor falsely certifies that it is in compliance with this section, the contract with that contractor shall be subject to Article 9 (commencing with Section 10420), unless, within a time period specified by the department or other contracting agency, the contractor provides to the department or agency proof that it has complied, or is in the process of complying, with this section."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: d1d20e9e-02ca-4664-8c2e-b2d54022898e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"d1d20e9e-02ca-4664-8c2e-b2d54022898e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"6283eded-36d7-4abb-8041-61762349422b\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"915f288c389674783d90f69bae2a86f2b199aadf4cb9a2869cf4103229d01776\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"8b2d638c-bdcf-4091-9dc2-5fca4978ca4e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"e184f27f91ed33251037adbf26dbdde4ba8d1122c0f70d760ea4c9dc986bdfb1\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 5541, \"end_char_idx\": 9360, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "that endangers the public health, welfare, or safety, or the contract is necessary for the provision of essential services, and no entity that complies with the requirements of this section capable of responding to the emergency is immediately available.\\n(3) The requirements of this section violate, or are inconsistent with, the terms or conditions of a grant, subvention, or agreement, if the agency has made a good faith attempt to change the terms or conditions of any grant, subvention, or agreement to authorize application of this section.\\n(4) The contractor is providing wholesale or bulk water, power, or natural gas, the conveyance or transmission of the same, or ancillary services, as required for ensuring reliable services in accordance with good utility practice, if the purchase of the same cannot practically be accomplished through the standard competitive bidding procedures and the contractor is not providing direct retail services to end users.\\n(d) (1) A contractor shall not be deemed to discriminate in the provision of benefits if the contractor, in providing the benefits, pays the actual costs incurred in obtaining the benefit.\\n(2) If a contractor is unable to provide a certain benefit, despite taking reasonable measures to do so, the contractor shall not be deemed to discriminate in the provision of benefits.\\n(e) (1) Every contract subject to this chapter shall contain a statement by which the contractor certifies that the contractor is in compliance with this section.\\n(2) The department or other contracting agency shall enforce this section pursuant to its existing enforcement powers.\\n(3) (A) If a contractor falsely certifies that it is in compliance with this section, the contract with that contractor shall be subject to Article 9 (commencing with Section 10420), unless, within a time period specified by the department or other contracting agency, the contractor provides to the department or agency proof that it has complied, or is in the process of complying, with this section.\\n(B) The application of the remedies or penalties contained in Article 9 (commencing with Section 10420) to a contract subject to this chapter shall not preclude the application of any existing remedies otherwise available to the department or other contracting agency under its existing enforcement powers.\\n(f) Nothing in this section is intended to regulate the contracting practices of any local jurisdiction.\\n(g) This section shall be construed so as not to conflict with applicable federal laws, rules, or regulations. In the event that a court or agency of competent jurisdiction holds that federal law, rule, or regulation invalidates any clause, sentence, paragraph, or section of this code or the application thereof to any person or circumstances, it is the intent of the state that the court or agency sever that clause, sentence, paragraph, or section so that the remainder of this section shall remain in effect.\\nSEC. 2.\\nSection 10295.35 of the Public Contract Code shall not be construed to create any new enforcement authority or responsibility in the Department of General Services or any other contracting agency.\\nSEC. 3.\\nNo reimbursement is required by this act pursuant to Section 6 of Article XIII\\u2009B of the California Constitution because the only costs that may be incurred by a local agency or school district will be incurred because this act creates a new crime or infraction, eliminates a crime or infraction, or changes the penalty for a crime or infraction, within the meaning of Section 17556 of the Government Code, or changes the definition of a crime within the meaning of Section 6 of Article XIII\\u2009B of the California Constitution.',\n 'title': 'An act to add Section 10295.35 to the Public Contract Code, relating to public contracts.'}"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 8b2d638c-bdcf-4091-9dc2-5fca4978ca4e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"8b2d638c-bdcf-4091-9dc2-5fca4978ca4e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"d1d20e9e-02ca-4664-8c2e-b2d54022898e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"cc5df8dfdb3f11e8041b33b70a4d4429c81cf1fc345eea286c0d315a6c41b24b\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"f6e6a177-2312-4799-b4e0-986bc8e29e76\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"bdd516d048cb095cdf53b9aa952df7a6cbe211995fb48bd543965af445200623\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 7442, \"end_char_idx\": 10203, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "the contractor provides to the department or agency proof that it has complied, or is in the process of complying, with this section.\\n(B) The application of the remedies or penalties contained in Article 9 (commencing with Section 10420) to a contract subject to this chapter shall not preclude the application of any existing remedies otherwise available to the department or other contracting agency under its existing enforcement powers.\\n(f) Nothing in this section is intended to regulate the contracting practices of any local jurisdiction.\\n(g) This section shall be construed so as not to conflict with applicable federal laws, rules, or regulations. In the event that a court or agency of competent jurisdiction holds that federal law, rule, or regulation invalidates any clause, sentence, paragraph, or section of this code or the application thereof to any person or circumstances, it is the intent of the state that the court or agency sever that clause, sentence, paragraph, or section so that the remainder of this section shall remain in effect.\\nSEC. 2.\\nSection 10295.35 of the Public Contract Code shall not be construed to create any new enforcement authority or responsibility in the Department of General Services or any other contracting agency.\\nSEC. 3.\\nNo reimbursement is required by this act pursuant to Section 6 of Article XIII\\u2009B of the California Constitution because the only costs that may be incurred by a local agency or school district will be incurred because this act creates a new crime or infraction, eliminates a crime or infraction, or changes the penalty for a crime or infraction, within the meaning of Section 17556 of the Government Code, or changes the definition of a crime within the meaning of Section 6 of Article XIII\\u2009B of the California Constitution.',\n 'title': 'An act to add Section 10295.35 to the Public Contract Code, relating to public contracts.'}\n```\n\nThere are two fields that you'll want to use:\n\n- `text`: the text of the bill which'll be the input to the model.\n- `summary`: a condensed version of `text` which'll be the model target.\n\n## Preprocess\n\nThe next step is to load a T5 tokenizer to process `text` and `summary`:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> checkpoint = \"google-t5/t5-small\"\n>>> tokenizer = AutoTokenizer.from_pretrained(checkpoint)\n```\n\nThe preprocessing function you want to create needs to:\n\n1. Prefix the input with a prompt so T5 knows this is a summarization task. Some models capable of multiple NLP tasks require prompting for specific tasks.\n2. Use the keyword `text_target` argument when tokenizing labels.\n3. Truncate sequences to be no longer than the maximum length set by the `max_length` parameter.\n\n```py\n>>> prefix = \"summarize: \""}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: f6e6a177-2312-4799-b4e0-986bc8e29e76
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"f6e6a177-2312-4799-b4e0-986bc8e29e76\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"8b2d638c-bdcf-4091-9dc2-5fca4978ca4e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"dd0dda660f36abd526a9a23cfe6282da7d88765cd50f136883e97c27ed26c46d\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"6fbc4c1e-cd80-4de6-9ed3-5aa131eedfbb\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"c67ea74fee14e2d3caffe00d7a6ea3311b1f5f8c114cbef7d0fef2f8de4a0fe6\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 8510, \"end_char_idx\": 12162, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "2.\\nSection 10295.35 of the Public Contract Code shall not be construed to create any new enforcement authority or responsibility in the Department of General Services or any other contracting agency.\\nSEC. 3.\\nNo reimbursement is required by this act pursuant to Section 6 of Article XIII\\u2009B of the California Constitution because the only costs that may be incurred by a local agency or school district will be incurred because this act creates a new crime or infraction, eliminates a crime or infraction, or changes the penalty for a crime or infraction, within the meaning of Section 17556 of the Government Code, or changes the definition of a crime within the meaning of Section 6 of Article XIII\\u2009B of the California Constitution.',\n 'title': 'An act to add Section 10295.35 to the Public Contract Code, relating to public contracts.'}\n```\n\nThere are two fields that you'll want to use:\n\n- `text`: the text of the bill which'll be the input to the model.\n- `summary`: a condensed version of `text` which'll be the model target.\n\n## Preprocess\n\nThe next step is to load a T5 tokenizer to process `text` and `summary`:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> checkpoint = \"google-t5/t5-small\"\n>>> tokenizer = AutoTokenizer.from_pretrained(checkpoint)\n```\n\nThe preprocessing function you want to create needs to:\n\n1. Prefix the input with a prompt so T5 knows this is a summarization task. Some models capable of multiple NLP tasks require prompting for specific tasks.\n2. Use the keyword `text_target` argument when tokenizing labels.\n3. Truncate sequences to be no longer than the maximum length set by the `max_length` parameter.\n\n```py\n>>> prefix = \"summarize: \"\n\n\n>>> def preprocess_function(examples):\n...     inputs = [prefix + doc for doc in examples[\"text\"]]\n...     model_inputs = tokenizer(inputs, max_length=1024, truncation=True)\n\n...     labels = tokenizer(text_target=examples[\"summary\"], max_length=128, truncation=True)\n\n...     model_inputs[\"labels\"] = labels[\"input_ids\"]\n...     return model_inputs\n```\n\nTo apply the preprocessing function over the entire dataset, use \ud83e\udd17 Datasets [`~datasets.Dataset.map`] method. You can speed up the `map` function by setting `batched=True` to process multiple elements of the dataset at once:\n\n```py\n>>> tokenized_billsum = billsum.map(preprocess_function, batched=True)\n```\n\nNow create a batch of examples using [`DataCollatorForSeq2Seq`]. It's more efficient to *dynamically pad* the sentences to the longest length in a batch during collation, instead of padding the whole dataset to the maximum length.\n\n<frameworkcontent>\n<pt>\n\n```py\n>>> from transformers import DataCollatorForSeq2Seq\n\n>>> data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint)\n```\n</pt>\n<tf>\n\n```py\n>>> from transformers import DataCollatorForSeq2Seq\n\n>>> data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint, return_tensors=\"tf\")\n```\n</tf>\n</frameworkcontent>\n\n## Evaluate\n\nIncluding a metric during training is often helpful for evaluating your model's performance. You can quickly load a evaluation method with the \ud83e\udd17 [Evaluate](https://huggingface.co/docs/evaluate/index) library. For this task, load the [ROUGE](https://huggingface.co/spaces/evaluate-metric/rouge) metric (see the \ud83e\udd17 Evaluate [quick tour](https://huggingface.co/docs/evaluate/a_quick_tour) to learn more about how to load and compute a metric):\n\n```py\n>>> import evaluate\n\n>>> rouge = evaluate.load(\"rouge\")\n```\n\nThen create a function that passes your predictions and labels to [`~evaluate.EvaluationModule.compute`] to calculate the ROUGE metric:\n\n```py\n>>> import numpy as np"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 6fbc4c1e-cd80-4de6-9ed3-5aa131eedfbb
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"6fbc4c1e-cd80-4de6-9ed3-5aa131eedfbb\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"f6e6a177-2312-4799-b4e0-986bc8e29e76\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"a9bcc77ad663fe4b59591c2ec2f105bfa391d78dd501d87a6fc882956688feae\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"4aa83cb7-5d2d-46f0-967b-af2308d482bb\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"1fc095916afa675f55c71c898912f5e8a12a05820a51c9f84f0ce208ff8be328\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 12165, \"end_char_idx\": 15070, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": ">>> def compute_metrics(eval_pred):\n...     predictions, labels = eval_pred\n...     decoded_preds = tokenizer.batch_decode(predictions, skip_special_tokens=True)\n...     labels = np.where(labels != -100, labels, tokenizer.pad_token_id)\n...     decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n\n...     result = rouge.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)\n\n...     prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in predictions]\n...     result[\"gen_len\"] = np.mean(prediction_lens)\n\n...     return {k: round(v, 4) for k, v in result.items()}\n```\n\nYour `compute_metrics` function is ready to go now, and you'll return to it when you setup your training.\n\n## Train\n\n<frameworkcontent>\n<pt>\n<Tip>\n\nIf you aren't familiar with finetuning a model with the [`Trainer`], take a look at the basic tutorial [here](../training#train-with-pytorch-trainer)!\n\n</Tip>\n\nYou're ready to start training your model now! Load T5 with [`AutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nAt this point, only three steps remain:\n\n1. Define your training hyperparameters in [`Seq2SeqTrainingArguments`]. The only required parameter is `output_dir` which specifies where to save your model. You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the ROUGE metric and save the training checkpoint.\n2. Pass the training arguments to [`Seq2SeqTrainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = Seq2SeqTrainingArguments(\n...     output_dir=\"my_awesome_billsum_model\",\n...     eval_strategy=\"epoch\",\n...     learning_rate=2e-5,\n...     per_device_train_batch_size=16,\n...     per_device_eval_batch_size=16,\n...     weight_decay=0.01,\n...     save_total_limit=3,\n...     num_train_epochs=4,\n...     predict_with_generate=True,\n...     fp16=True,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Seq2SeqTrainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=tokenized_billsum[\"train\"],\n...     eval_dataset=tokenized_billsum[\"test\"],\n...     tokenizer=tokenizer,\n...     data_collator=data_collator,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n<tf>\n<Tip>\n\nIf you aren't familiar with finetuning a model with Keras, take a look at the basic tutorial [here](../training#train-a-tensorflow-model-with-keras)!"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 4aa83cb7-5d2d-46f0-967b-af2308d482bb
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"4aa83cb7-5d2d-46f0-967b-af2308d482bb\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"6fbc4c1e-cd80-4de6-9ed3-5aa131eedfbb\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"7cfbbe55fa36f9f4290779c9e7e000f9cd289982a015cbcd362139e0c9df294a\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"817b8032-ab83-47c6-8470-afd8b69a4d84\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"76f7488be7188048a3312e3b809ec8bb3c8e382229592e80113100cfe3b302de\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 13570, \"end_char_idx\": 16590, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "You'll push this model to the Hub by setting `push_to_hub=True` (you need to be signed in to Hugging Face to upload your model). At the end of each epoch, the [`Trainer`] will evaluate the ROUGE metric and save the training checkpoint.\n2. Pass the training arguments to [`Seq2SeqTrainer`] along with the model, dataset, tokenizer, data collator, and `compute_metrics` function.\n3. Call [`~Trainer.train`] to finetune your model.\n\n```py\n>>> training_args = Seq2SeqTrainingArguments(\n...     output_dir=\"my_awesome_billsum_model\",\n...     eval_strategy=\"epoch\",\n...     learning_rate=2e-5,\n...     per_device_train_batch_size=16,\n...     per_device_eval_batch_size=16,\n...     weight_decay=0.01,\n...     save_total_limit=3,\n...     num_train_epochs=4,\n...     predict_with_generate=True,\n...     fp16=True,\n...     push_to_hub=True,\n... )\n\n>>> trainer = Seq2SeqTrainer(\n...     model=model,\n...     args=training_args,\n...     train_dataset=tokenized_billsum[\"train\"],\n...     eval_dataset=tokenized_billsum[\"test\"],\n...     tokenizer=tokenizer,\n...     data_collator=data_collator,\n...     compute_metrics=compute_metrics,\n... )\n\n>>> trainer.train()\n```\n\nOnce training is completed, share your model to the Hub with the [`~transformers.Trainer.push_to_hub`] method so everyone can use your model:\n\n```py\n>>> trainer.push_to_hub()\n```\n</pt>\n<tf>\n<Tip>\n\nIf you aren't familiar with finetuning a model with Keras, take a look at the basic tutorial [here](../training#train-a-tensorflow-model-with-keras)!\n\n</Tip>\nTo finetune a model in TensorFlow, start by setting up an optimizer function, learning rate schedule, and some training hyperparameters:\n\n```py\n>>> from transformers import create_optimizer, AdamWeightDecay\n\n>>> optimizer = AdamWeightDecay(learning_rate=2e-5, weight_decay_rate=0.01)\n```\n\nThen you can load T5 with [`TFAutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nConvert your datasets to the `tf.data.Dataset` format with [`~transformers.TFPreTrainedModel.prepare_tf_dataset`]:\n\n```py\n>>> tf_train_set = model.prepare_tf_dataset(\n...     tokenized_billsum[\"train\"],\n...     shuffle=True,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n\n>>> tf_test_set = model.prepare_tf_dataset(\n...     tokenized_billsum[\"test\"],\n...     shuffle=False,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n```\n\nConfigure the model for training with [`compile`](https://keras.io/api/models/model_training_apis/#compile-method). Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> import tensorflow as tf\n\n>>> model.compile(optimizer=optimizer)  # No loss argument!\n```\n\nThe last two things to setup before you start training is to compute the ROUGE score from the predictions, and provide a way to push your model to the Hub. Both are done by using [Keras callbacks](../main_classes/keras_callbacks)."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 817b8032-ab83-47c6-8470-afd8b69a4d84
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"817b8032-ab83-47c6-8470-afd8b69a4d84\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"4aa83cb7-5d2d-46f0-967b-af2308d482bb\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"9e018253aeb8aa9f16bce63257b76d4b5819840f2864aa3e4a37b1a7ca8198e3\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"956c6798-b689-44e0-9d99-d143e973b22e\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"063f3f076384dff4a8b28e20d5c36e2d8758bfd57f5946327c7d3786d17d860d\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 15072, \"end_char_idx\": 18303, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "</Tip>\nTo finetune a model in TensorFlow, start by setting up an optimizer function, learning rate schedule, and some training hyperparameters:\n\n```py\n>>> from transformers import create_optimizer, AdamWeightDecay\n\n>>> optimizer = AdamWeightDecay(learning_rate=2e-5, weight_decay_rate=0.01)\n```\n\nThen you can load T5 with [`TFAutoModelForSeq2SeqLM`]:\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(checkpoint)\n```\n\nConvert your datasets to the `tf.data.Dataset` format with [`~transformers.TFPreTrainedModel.prepare_tf_dataset`]:\n\n```py\n>>> tf_train_set = model.prepare_tf_dataset(\n...     tokenized_billsum[\"train\"],\n...     shuffle=True,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n\n>>> tf_test_set = model.prepare_tf_dataset(\n...     tokenized_billsum[\"test\"],\n...     shuffle=False,\n...     batch_size=16,\n...     collate_fn=data_collator,\n... )\n```\n\nConfigure the model for training with [`compile`](https://keras.io/api/models/model_training_apis/#compile-method). Note that Transformers models all have a default task-relevant loss function, so you don't need to specify one unless you want to:\n\n```py\n>>> import tensorflow as tf\n\n>>> model.compile(optimizer=optimizer)  # No loss argument!\n```\n\nThe last two things to setup before you start training is to compute the ROUGE score from the predictions, and provide a way to push your model to the Hub. Both are done by using [Keras callbacks](../main_classes/keras_callbacks).\n\nPass your `compute_metrics` function to [`~transformers.KerasMetricCallback`]:\n\n```py\n>>> from transformers.keras_callbacks import KerasMetricCallback\n\n>>> metric_callback = KerasMetricCallback(metric_fn=compute_metrics, eval_dataset=tf_validation_set)\n```\n\nSpecify where to push your model and tokenizer in the [`~transformers.PushToHubCallback`]:\n\n```py\n>>> from transformers.keras_callbacks import PushToHubCallback\n\n>>> push_to_hub_callback = PushToHubCallback(\n...     output_dir=\"my_awesome_billsum_model\",\n...     tokenizer=tokenizer,\n... )\n```\n\nThen bundle your callbacks together:\n\n```py\n>>> callbacks = [metric_callback, push_to_hub_callback]\n```\n\nFinally, you're ready to start training your model! Call [`fit`](https://keras.io/api/models/model_training_apis/#fit-method) with your training and validation datasets, the number of epochs, and your callbacks to finetune the model:\n\n```py\n>>> model.fit(x=tf_train_set, validation_data=tf_test_set, epochs=3, callbacks=callbacks)\n```\n\nOnce training is completed, your model is automatically uploaded to the Hub so everyone can use it!\n</tf>\n</frameworkcontent>\n\n<Tip>\n\nFor a more in-depth example of how to finetune a model for summarization, take a look at the corresponding\n[PyTorch notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization.ipynb)\nor [TensorFlow notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization-tf.ipynb).\n\n</Tip>\n\n## Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nCome up with some text you'd like to summarize. For T5, you need to prefix your input depending on the task you're working on."}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 956c6798-b689-44e0-9d99-d143e973b22e
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"956c6798-b689-44e0-9d99-d143e973b22e\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"817b8032-ab83-47c6-8470-afd8b69a4d84\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"74a71bb1dedafba0f2f7cc50a5826c97194ab0fac0e06be36528742aaf47aa71\", \"class_name\": \"RelatedNodeInfo\"}, \"3\": {\"node_id\": \"7a6ca9a6-12b1-4aff-b7e8-4ae8a9117df4\", \"node_type\": \"1\", \"metadata\": {}, \"hash\": \"16dcb01e3c780f50e688b261f9a493f76dbd2d759ed0de3a5afbbca6c2c676fe\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 17302, \"end_char_idx\": 19550, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "Call [`fit`](https://keras.io/api/models/model_training_apis/#fit-method) with your training and validation datasets, the number of epochs, and your callbacks to finetune the model:\n\n```py\n>>> model.fit(x=tf_train_set, validation_data=tf_test_set, epochs=3, callbacks=callbacks)\n```\n\nOnce training is completed, your model is automatically uploaded to the Hub so everyone can use it!\n</tf>\n</frameworkcontent>\n\n<Tip>\n\nFor a more in-depth example of how to finetune a model for summarization, take a look at the corresponding\n[PyTorch notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization.ipynb)\nor [TensorFlow notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/main/examples/summarization-tf.ipynb).\n\n</Tip>\n\n## Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nCome up with some text you'd like to summarize. For T5, you need to prefix your input depending on the task you're working on. For summarization you should prefix your input as shown below:\n\n```py\n>>> text = \"summarize: The Inflation Reduction Act lowers prescription drug costs, health care costs, and energy costs. It's the most aggressive action on tackling the climate crisis in American history, which will lift up American workers and create good-paying, union jobs across the country. It'll lower the deficit and ask the ultra-wealthy and corporations to pay their fair share. And no one making under $400,000 per year will pay a penny more in taxes.\"\n```\n\nThe simplest way to try out your finetuned model for inference is to use it in a [`pipeline`]. Instantiate a `pipeline` for summarization with your model, and pass your text to it:\n\n```py\n>>> from transformers import pipeline\n\n>>> summarizer = pipeline(\"summarization\", model=\"stevhliu/my_awesome_billsum_model\")\n>>> summarizer(text)\n[{\"summary_text\": \"The Inflation Reduction Act lowers prescription drug costs, health care costs, and energy costs. It's the most aggressive action on tackling the climate crisis in American history, which will lift up American workers and create good-paying, union jobs across the country.\"}]\n```\n\nYou can also manually replicate the results of the `pipeline` if you'd like:"}


created_at: 2024-07-28 18:57:50
topic: persistent://default/default/5523a7e1-1ad4-449b-b059-70d441f3d0d1
id: 7a6ca9a6-12b1-4aff-b7e8-4ae8a9117df4
encoding: FLOAT32
metadata:
{"url": "https://huggingface.co/docs/transformers/tasks/summarization", "title": "Summarization", "tokens": 5008, "retrieve_doc": true, "source": "transformers", "_node_content": "{\"id_\": \"7a6ca9a6-12b1-4aff-b7e8-4ae8a9117df4\", \"embedding\": null, \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"excluded_embed_metadata_keys\": [\"url\", \"tokens\", \"retrieve_doc\", \"source\"], \"excluded_llm_metadata_keys\": [\"title\", \"tokens\", \"retrieve_doc\", \"source\"], \"relationships\": {\"1\": {\"node_id\": \"cdcb25e3-2871-4e5b-83cf-cfd84aefcd97\", \"node_type\": \"4\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"c3d648d8a928b2b8b0f3f62054d010ad49f16e95df5d6868f3d8266a0d71666e\", \"class_name\": \"RelatedNodeInfo\"}, \"2\": {\"node_id\": \"956c6798-b689-44e0-9d99-d143e973b22e\", \"node_type\": \"1\", \"metadata\": {\"url\": \"https://huggingface.co/docs/transformers/tasks/summarization\", \"title\": \"Summarization\", \"tokens\": 5008, \"retrieve_doc\": true, \"source\": \"transformers\"}, \"hash\": \"914e98619a7325698aadb8469f32e55408e0f3f687eb31c221fde0bca0549fad\", \"class_name\": \"RelatedNodeInfo\"}}, \"text\": \"\", \"mimetype\": \"text/plain\", \"start_char_idx\": 18082, \"end_char_idx\": 21890, \"text_template\": \"{metadata_str}\\n\\n{content}\", \"metadata_template\": \"{key}: {value}\", \"metadata_seperator\": \"\\n\", \"class_name\": \"TextNode\"}", "_node_type": "TextNode", "document_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "ref_doc_id": "cdcb25e3-2871-4e5b-83cf-cfd84aefcd97", "chroma:document": "</Tip>\n\n## Inference\n\nGreat, now that you've finetuned a model, you can use it for inference!\n\nCome up with some text you'd like to summarize. For T5, you need to prefix your input depending on the task you're working on. For summarization you should prefix your input as shown below:\n\n```py\n>>> text = \"summarize: The Inflation Reduction Act lowers prescription drug costs, health care costs, and energy costs. It's the most aggressive action on tackling the climate crisis in American history, which will lift up American workers and create good-paying, union jobs across the country. It'll lower the deficit and ask the ultra-wealthy and corporations to pay their fair share. And no one making under $400,000 per year will pay a penny more in taxes.\"\n```\n\nThe simplest way to try out your finetuned model for inference is to use it in a [`pipeline`]. Instantiate a `pipeline` for summarization with your model, and pass your text to it:\n\n```py\n>>> from transformers import pipeline\n\n>>> summarizer = pipeline(\"summarization\", model=\"stevhliu/my_awesome_billsum_model\")\n>>> summarizer(text)\n[{\"summary_text\": \"The Inflation Reduction Act lowers prescription drug costs, health care costs, and energy costs. It's the most aggressive action on tackling the climate crisis in American history, which will lift up American workers and create good-paying, union jobs across the country.\"}]\n```\n\nYou can also manually replicate the results of the `pipeline` if you'd like:\n\n\n<frameworkcontent>\n<pt>\nTokenize the text and return the `input_ids` as PyTorch tensors:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"stevhliu/my_awesome_billsum_model\")\n>>> inputs = tokenizer(text, return_tensors=\"pt\").input_ids\n```\n\nUse the [`~generation.GenerationMixin.generate`] method to create the summarization. For more details about the different text generation strategies and parameters for controlling generation, check out the [Text Generation](../main_classes/text_generation) API.\n\n```py\n>>> from transformers import AutoModelForSeq2SeqLM\n\n>>> model = AutoModelForSeq2SeqLM.from_pretrained(\"stevhliu/my_awesome_billsum_model\")\n>>> outputs = model.generate(inputs, max_new_tokens=100, do_sample=False)\n```\n\nDecode the generated token ids back into text:\n\n```py\n>>> tokenizer.decode(outputs[0], skip_special_tokens=True)\n'the inflation reduction act lowers prescription drug costs, health care costs, and energy costs. it's the most aggressive action on tackling the climate crisis in american history. it will ask the ultra-wealthy and corporations to pay their fair share.'\n```\n</pt>\n<tf>\nTokenize the text and return the `input_ids` as TensorFlow tensors:\n\n```py\n>>> from transformers import AutoTokenizer\n\n>>> tokenizer = AutoTokenizer.from_pretrained(\"stevhliu/my_awesome_billsum_model\")\n>>> inputs = tokenizer(text, return_tensors=\"tf\").input_ids\n```\n\nUse the [`~transformers.generation_tf_utils.TFGenerationMixin.generate`] method to create the summarization. For more details about the different text generation strategies and parameters for controlling generation, check out the [Text Generation](../main_classes/text_generation) API.\n\n```py\n>>> from transformers import TFAutoModelForSeq2SeqLM\n\n>>> model = TFAutoModelForSeq2SeqLM.from_pretrained(\"stevhliu/my_awesome_billsum_model\")\n>>> outputs = model.generate(inputs, max_new_tokens=100, do_sample=False)\n```\n\nDecode the generated token ids back into text:\n\n```py\n>>> tokenizer.decode(outputs[0], skip_special_tokens=True)\n'the inflation reduction act lowers prescription drug costs, health care costs, and energy costs. it's the most aggressive action on tackling the climate crisis in american history. it will ask the ultra-wealthy and corporations to pay their fair share.'\n```\n</tf>\n</frameworkcontent>"}


