notebooks/cross-lingual-books-alignment/cross-lingual-books-alignment.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7ae282b9-2041-4594-b10a-845f9a4f0382",
   "metadata": {},
   "source": [
    "# Cross-lingual Books Alignment with Transformers and OpenVINO™\n",
    "\n",
    "Cross-lingual text alignment is the task of matching sentences in a pair of texts that are translations of each other. In this notebook, you'll learn how to use a deep learning model to create a parallel book in English and German\n",
    "\n",
    "This method helps you learn languages but also provides parallel texts that can be used to train machine translation models. This is particularly useful if one of the languages is low-resource or you don't have enough data to train a full-fledged translation model.\n",
    "\n",
    "The notebook shows how to accelerate the most computationally expensive part of the pipeline - getting vectors from sentences - using the OpenVINO™ framework.\n",
    "\n",
    "\n",
    "## Pipeline\n",
    "\n",
    "The notebook guides you through the entire process of creating a parallel book: from obtaining raw texts to building a visualization of aligned sentences. Here is the pipeline diagram:\n",
    "\n",
    "![](https://user-images.githubusercontent.com/51917466/254582697-18f3ab38-e264-4b2c-a088-8e54b855c1b2.png)\n",
    "\n",
    "Visualizing the result allows you to identify areas for improvement in the pipeline steps, as indicated in the diagram.\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "- `requests` - for getting books\n",
    "- `pysbd` - for splitting sentences\n",
    "- `transformers[torch]` and `openvino_dev` - for getting sentence embeddings\n",
    "- `seaborn` - for alignment matrix visualization\n",
    "- `ipywidgets` - for displaying HTML and JS output in the notebook \n",
    "\n",
    "#### Table of contents:\n",
    "\n",
    "- [Get Books](#Get-Books)\n",
    "- [Clean Text](#Clean-Text)\n",
    "- [Split Text](#Split-Text)\n",
    "- [Get Sentence Embeddings](#Get-Sentence-Embeddings)\n",
    "    - [Optimize the Model with OpenVINO](#Optimize-the-Model-with-OpenVINO)\n",
    "- [Calculate Sentence Alignment](#Calculate-Sentence-Alignment)\n",
    "- [Postprocess Sentence Alignment](#Postprocess-Sentence-Alignment)\n",
    "- [Visualize Sentence Alignment](#Visualize-Sentence-Alignment)\n",
    "- [Speed up Embeddings Computation](#Speed-up-Embeddings-Computation)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "c45a2e19-7983-4daa-805c-e632a9d672fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "import platform\n",
    "\n",
    "if platform.system() != \"Windows\":\n",
    "    %pip install -q \"matplotlib>=3.4\"\n",
    "else:\n",
    "    %pip install -q \"matplotlib>=3.4,<3.7\"\n",
    "\n",
    "%pip install -q --extra-index-url https://download.pytorch.org/whl/cpu requests pysbd transformers \"torch>=2.1\" \"openvino>=2023.1.0\" seaborn ipywidgets"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "34ddf5a3-f647-4229-8e45-ac988ee094ae",
   "metadata": {},
   "source": [
    "## Get Books\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "The first step is to get the books that we will be working with. For this notebook, we will use English and German versions of Anna Karenina by Leo Tolstoy. The texts can be obtained from the [Project Gutenberg site](https://www.gutenberg.org/). Since copyright laws are complex and differ from country to country, check the book's legal availability in your country. Refer to the Project Gutenberg Permissions, Licensing and other Common Requests [page](https://www.gutenberg.org/policy/permission.html) for more information.\n",
    "\n",
    "Find the books on Project Gutenberg [search page](https://www.gutenberg.org/ebooks/) and get the ID of each book. To get the texts, we will pass the IDs to the [Gutendex](http://gutendex.com/) API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8e4d3551-7879-42ab-ae16-707f059f8c7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "\n",
    "def get_book_by_id(book_id: int, gutendex_url: str = \"https://gutendex.com/\") -> str:\n",
    "    book_metadata_url = gutendex_url + \"/books/\" + str(book_id)\n",
    "    request = requests.get(book_metadata_url, timeout=30)\n",
    "    request.raise_for_status()\n",
    "\n",
    "    book_metadata = request.json()\n",
    "    text_format_key = \"text/plain\"\n",
    "    text_plain = [k for k in book_metadata[\"formats\"] if k.startswith(text_format_key)]\n",
    "    book_url = book_metadata[\"formats\"][text_plain[0]]\n",
    "    return requests.get(book_url).text\n",
    "\n",
    "\n",
    "en_book_id = 1399\n",
    "de_book_id = 44956\n",
    "\n",
    "anna_karenina_en = get_book_by_id(en_book_id)\n",
    "anna_karenina_de = get_book_by_id(de_book_id)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "46316fc8-8190-4274-9cfc-1883f06bc6d8",
   "metadata": {},
   "source": [
    "Let's check that we got the right books by showing a part of the texts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1fef94e7-9499-40cf-838a-62b32dbfef56",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Project Gutenberg eBook of Anna Karenina\n",
      "    \n",
      "This ebook is for the use of anyone anywhere in the United States and\n",
      "most other parts of the world at no cost and with almost no restrictions\n",
      "whatsoever. You may copy it, give it away or re-use it under the terms\n",
      "of the Project Gutenberg License included with this ebook or online\n",
      "at www.gutenberg.org. If you are not located in the United States,\n",
      "you will have to check the laws of the country where you are located\n",
      "before using this eBook.\n",
      "\n",
      "Title: Anna Karenina\n",
      "\n",
      "\n",
      "Author: graf Leo Tolstoy\n",
      "\n",
      "Translator: Constance Garnett\n",
      "\n",
      "Release date: July 1, 1998 [eBook #1399]\n",
      "                Most recently updated: April 9, 2023\n",
      "\n",
      "Language: English\n",
      "\n",
      "\n",
      "\n",
      "*** START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA ***\n",
      "[Illustration]\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " ANNA KARENINA \n",
      "\n",
      " by Leo Tolstoy \n",
      "\n",
      " Translated by Constance Garnett \n",
      "\n",
      "Contents\n",
      "\n",
      "\n",
      " PART ONE\n",
      " PART TWO\n",
      " PART THREE\n",
      " PART FOUR\n",
      " PART FIVE\n",
      " PART SIX\n",
      " PART SEVEN\n",
      " PART EIGHT\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "PART ONE\n",
      "\n",
      "Chapter 1\n",
      "\n",
      "\n",
      "Happy families are all alike; every unhappy family is unhappy in its\n",
      "own way.\n",
      "\n",
      "Everything was in confusion in the Oblonskys’ house. The wife had\n",
      "discovered that the husband was carrying on an intrigue with a French\n",
      "girl, who had been a governess in their family, and she had announced\n",
      "to her husband that she could not go on living in the same house with\n",
      "him. This position of affairs had now lasted three days, and not only\n",
      "the husband and wife themselves, but all the me\n"
     ]
    }
   ],
   "source": [
    "print(anna_karenina_en[:1500])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "96e5475c-1db9-4f1b-a894-35927afa50e8",
   "metadata": {},
   "source": [
    "which in a raw format looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d1419b6e-2c73-4627-a713-ffdce236b70f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\ufeffThe Project Gutenberg eBook of Anna Karenina\\r\\n    \\r\\nThis ebook is for the use of anyone anywhere in the United States and\\r\\nmost other parts of the world at no cost and with almost no restrictions\\r\\nwhatsoever. You may copy it, give it away or re-use it under the terms\\r\\nof the Project Gutenberg License included with this ebook or online\\r\\nat www.gutenberg.org. If you are not located in the United States,\\r\\nyou will have to check the laws of the country where you are located\\r\\nbefore using this eBook.\\r\\n\\r\\nTitle: Anna Karenina\\r\\n\\r\\n\\r\\nAuthor: graf Leo Tolstoy\\r\\n\\r\\nTranslator: Constance Garnett\\r\\n\\r\\nRelease date: July 1, 1998 [eBook #1399]\\r\\n                Most recently updated: April 9, 2023\\r\\n\\r\\nLanguage: English\\r\\n\\r\\n\\r\\n\\r\\n*** START OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA ***\\r\\n[Illustration]\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n ANNA KARENINA \\r\\n\\r\\n by Leo Tolstoy \\r\\n\\r\\n Translated by Constance Garnett \\r\\n\\r\\nContents\\r\\n\\r\\n\\r\\n PART ONE\\r\\n PART TWO\\r\\n PART THREE\\r\\n PART FOUR\\r\\n PART FIVE\\r\\n PART SIX\\r\\n PART SEVEN\\r\\n PART EIGHT\\r\\n\\r\\n\\r\\n\\r\\n\\r\\nPART ONE\\r\\n\\r\\nChapter 1\\r\\n\\r\\n\\r\\nHappy families are all alike; every unhappy family is unhappy in its\\r\\nown way.\\r\\n\\r\\nEverything was in confusion in the Oblonskys’ house. The wife had\\r\\ndiscovered that the husband was carrying on an intrigue with a French\\r\\ngirl, who had been a governess in their family, and she had announced\\r\\nto her husband that she could not go on living in the same house with\\r\\nhim. This position of affairs had now lasted three days, and not only\\r\\nthe husband and wife themselves, but all the me'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anna_karenina_en[:1500]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2727e361-e578-4ade-b81f-c33cfa593899",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The Project Gutenberg EBook of Anna Karenina, 1. Band, by Leo N. Tolstoi\\r\\n\\r\\nThis eBook is for the use of anyone anywhere at no cost and with\\r\\nalmost no restrictions whatsoever.  You may copy it, give it away or\\r\\nre-use it under the terms of the Project Gutenberg License included\\r\\nwith this eBook or online at www.gutenberg.org\\r\\n\\r\\n\\r\\nTitle: Anna Karenina, 1. Band\\r\\n\\r\\nAuthor: Leo N. Tolstoi\\r\\n\\r\\nRelease Date: February 18, 2014 [EBook #44956]\\r\\n\\r\\nLanguage: German\\r\\n\\r\\nCharacter set encoding: ISO-8859-1\\r\\n\\r\\n*** START OF THIS PROJECT GUTENBERG EBOOK ANNA KARENINA, 1. BAND ***\\r\\n\\r\\n\\r\\n\\r\\n\\r\\nProduced by Norbert H. Langkau, Jens Nordmann and the\\r\\nOnline Distributed Proofreading Team at http://www.pgdp.net\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n                             Anna Karenina.\\r\\n\\r\\n\\r\\n                        Roman aus dem Russischen\\r\\n\\r\\n                                  des\\r\\n\\r\\n                         Grafen Leo N. Tolstoi.\\r\\n\\r\\n\\r\\n\\r\\n                  Nach der siebenten Auflage übersetzt\\r\\n\\r\\n                                  von\\r\\n\\r\\n                              Hans Moser.\\r\\n\\r\\n\\r\\n                              Erster Band.\\r\\n\\r\\n\\r\\n\\r\\n                                Leipzig\\r\\n\\r\\n                Druck und Verlag von Philipp Reclam jun.\\r\\n\\r\\n                   *       *       *       *       *\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n                              Erster Teil.\\r\\n\\r\\n                               »Die Rache ist mein, ich will vergelten.«\\r\\n\\r\\n                                   1.\\r\\n\\r\\n\\r\\nAlle glücklichen Familien sind einander ähnlich; jede unglücklich'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "anna_karenina_de[:1500]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "aae4b990-19cf-4146-ab78-dde8877daa59",
   "metadata": {},
   "source": [
    "## Clean Text\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "The downloaded books may contain service information before and after the main text. The text might have different formatting styles and markup, for example, phrases from a different language enclosed in underscores for potential emphasis or italicization:\n",
    "\n",
    "\n",
    "> Yes, <spell>Alabin</spell> was giving a dinner on glass tables, and the\n",
    "tables sang, <spell>_Il mio tesoro_—not _Il mio tesoro_</spell> though, but something\n",
    "better, and there were some sort of little decanters on the table, and\n",
    "they were women, too,” he remembered.\n",
    "\n",
    "\n",
    "The next stages of the pipeline will be difficult to complete without cleaning and normalizing the text. Since formatting may differ, manual work is required at this stage. For example, the main content in the German version is enclosed in `*       *       *       *       *`, so it is safe to remove everything before the first occurrence and after the last occurrence of these asterisks.\n",
    "\n",
    "> **Hint**: There are text-cleaning libraries that clean up common flaws. If the source of the text is known, you can look for a library designed for that source, for example [`gutenberg_cleaner`](https://github.com/kiasar/gutenberg_cleaner). These libraries can reduce manual work and even automate the process.process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "baa89c79-c7bd-438e-ba6d-d76cd2ae5c3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "from contextlib import contextmanager\n",
    "from tqdm.auto import tqdm\n",
    "\n",
    "\n",
    "start_pattern_en = r\"\\nPART ONE\"\n",
    "anna_karenina_en = re.split(start_pattern_en, anna_karenina_en)[1].strip()\n",
    "\n",
    "end_pattern_en = \"*** END OF THE PROJECT GUTENBERG EBOOK ANNA KARENINA ***\"\n",
    "anna_karenina_en = anna_karenina_en.split(end_pattern_en)[0].strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1e7c82b6-e99d-4a60-a71f-694bdb9b7cb8",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_pattern_de = \"*       *       *       *       *\"\n",
    "anna_karenina_de = anna_karenina_de.split(start_pattern_de, maxsplit=1)[1].strip()\n",
    "anna_karenina_de = anna_karenina_de.rsplit(start_pattern_de, maxsplit=1)[0].strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "22b0bbd4-a903-4010-9c54-515b23f60cac",
   "metadata": {},
   "outputs": [],
   "source": [
    "anna_karenina_en = anna_karenina_en.replace(\"\\r\\n\", \"\\n\")\n",
    "anna_karenina_de = anna_karenina_de.replace(\"\\r\\n\", \"\\n\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e81f85cb-a790-4396-8f5b-a7ed47f91d73",
   "metadata": {},
   "source": [
    "For this notebook, we will work only with the first chapter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "71f0665f-7009-4da0-b97b-cb3f45893a44",
   "metadata": {},
   "outputs": [],
   "source": [
    "chapter_pattern_en = r\"Chapter \\d?\\d\"\n",
    "chapter_1_en = re.split(chapter_pattern_en, anna_karenina_en)[1].strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bb87955f-7c61-47db-a255-3005ed72f1bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "chapter_pattern_de = r\"\\d?\\d.\\n\\n\"\n",
    "chapter_1_de = re.split(chapter_pattern_de, anna_karenina_de)[1].strip()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a218f5e9-e5ef-4881-bb48-f3f73d2b6e49",
   "metadata": {},
   "source": [
    "Let's cut it out and define some cleaning functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "6ad2e38c-0050-4f9f-ad3d-f860ea636620",
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_single_newline(text: str) -> str:\n",
    "    return re.sub(r\"\\n(?!\\n)\", \" \", text)\n",
    "\n",
    "\n",
    "def unify_quotes(text: str) -> str:\n",
    "    return re.sub(r\"['\\\"»«“”]\", '\"', text)\n",
    "\n",
    "\n",
    "def remove_markup(text: str) -> str:\n",
    "    text = text.replace(\">=\", \"\").replace(\"=<\", \"\")\n",
    "    return re.sub(r\"_\\w|\\w_\", \"\", text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "95d6c3d8-c7b4-4f1b-ad70-2a81853cb0ae",
   "metadata": {},
   "source": [
    "Combine the cleaning functions into a single pipeline. The `tqdm` library is used to track the execution progress. Define a context manager to optionally disable the progress indicators if they are not needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2b534d3e-fb9d-4a38-99d1-225ab7e33612",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3a1b20e12bcf4d1c815cf1e6570f6aa1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bd3afb95a50d4b56b63bee24847102cf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "disable_tqdm = False\n",
    "\n",
    "\n",
    "@contextmanager\n",
    "def disable_tqdm_context():\n",
    "    global disable_tqdm\n",
    "    disable_tqdm = True\n",
    "    yield\n",
    "    disable_tqdm = False\n",
    "\n",
    "\n",
    "def clean_text(text: str) -> str:\n",
    "    text_cleaning_pipeline = [\n",
    "        remove_single_newline,\n",
    "        unify_quotes,\n",
    "        remove_markup,\n",
    "    ]\n",
    "    progress_bar = tqdm(text_cleaning_pipeline, disable=disable_tqdm)\n",
    "    for clean_func in progress_bar:\n",
    "        progress_bar.set_postfix_str(clean_func.__name__)\n",
    "        text = clean_func(text)\n",
    "    return text\n",
    "\n",
    "\n",
    "chapter_1_en = clean_text(chapter_1_en)\n",
    "chapter_1_de = clean_text(chapter_1_de)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6c02fd99-d3c3-4301-9f3f-5f3a9babac8a",
   "metadata": {},
   "source": [
    "## Split Text\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Dividing text into sentences is a challenging task in text processing. The problem is called [sentence boundary disambiguation](https://en.wikipedia.org/wiki/Sentence_boundary_disambiguation), which can be solved using heuristics or machine learning models. This notebook uses a `Segmenter` from the `pysbd` library, which is initialized with an [ISO language code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes), as the rules for splitting text into sentences may vary for different languages.\n",
    "\n",
    "> **Hint**: The `book_metadata` obtained from the Gutendex contains the language code as well, enabling automation of this part of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6e2bcac2-91f2-43d5-b690-7be4fb453578",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(32, 34)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pysbd\n",
    "\n",
    "\n",
    "splitter_en = pysbd.Segmenter(language=\"en\", clean=True)\n",
    "splitter_de = pysbd.Segmenter(language=\"de\", clean=True)\n",
    "\n",
    "\n",
    "sentences_en = splitter_en.segment(chapter_1_en)\n",
    "sentences_de = splitter_de.segment(chapter_1_de)\n",
    "\n",
    "len(sentences_en), len(sentences_de)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b4264e86-6524-4f95-bb5d-e0498b55dad0",
   "metadata": {},
   "source": [
    "## Get Sentence Embeddings\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "The next step is to transform sentences into vector representations. Transformer encoder models, like BERT, provide high-quality embeddings but can be slow. Additionally, the model should support both chosen languages. Training separate models for each language pair can be expensive, so there are many models pre-trained on multiple languages simultaneously, for example:\n",
    "\n",
    "- [multilingual-MiniLM](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2)\n",
    "- [distiluse-base-multilingual-cased](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased-v2)\n",
    "- [bert-base-multilingual-uncased](https://huggingface.co/bert-base-multilingual-uncased)\n",
    "- [LaBSE](https://huggingface.co/rasa/LaBSE)\n",
    "\n",
    "LaBSE stands for [Language-agnostic BERT Sentence Embedding](https://arxiv.org/pdf/2007.01852.pdf) and supports 109+ languages. It has the same architecture as the BERT model but has been trained on a different task: to produce identical embeddings for translation pairs.\n",
    "\n",
    "![](https://github.com/openvinotoolkit/openvino_notebooks/assets/29454499/627d3a39-7076-479f-a7b1-392f49a0b83e)\n",
    "\n",
    "This makes LaBSE a great choice for our task and it can be reused for different language pairs still producing good results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "999759e2-cddf-40a0-a3b5-e453882f7e77",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List, Union, Dict\n",
    "from transformers import AutoTokenizer, AutoModel, BertModel\n",
    "import numpy as np\n",
    "import torch\n",
    "from openvino.runtime import CompiledModel as OVModel\n",
    "import openvino as ov\n",
    "\n",
    "\n",
    "model_id = \"rasa/LaBSE\"\n",
    "pt_model = AutoModel.from_pretrained(model_id)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_id)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2947fe49-25d6-4e9e-9d09-4d26a2e2e473",
   "metadata": {},
   "source": [
    "The model has two outputs: `last_hidden_state` and `pooler_output`. For generating embeddings, you can use either the first vector from the `last_hidden_state`, which corresponds to the special `[CLS]` token, or the entire vector from the second input. Usually, the second option is used, but we will be using the first option as it also works well for our task. Fill free to experiment with different outputs to find the best fit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "83fa560e-9e62-46ea-b755-c2cff01e22b7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cf0e340fa3a84ce892532a1840813a3b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/32 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "06f5aff781ae4357919e4907a72133d7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/34 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def get_embeddings(\n",
    "    sentences: List[str],\n",
    "    embedding_model: Union[BertModel, OVModel],\n",
    ") -> np.ndarray:\n",
    "    if isinstance(embedding_model, OVModel):\n",
    "        embeddings = [embedding_model(tokenizer(sent, return_tensors=\"np\").data)[\"last_hidden_state\"][0][0] for sent in tqdm(sentences, disable=disable_tqdm)]\n",
    "        return np.vstack(embeddings)\n",
    "    else:\n",
    "        embeddings = [embedding_model(**tokenizer(sent, return_tensors=\"pt\"))[\"last_hidden_state\"][0][0] for sent in tqdm(sentences, disable=disable_tqdm)]\n",
    "        return torch.vstack(embeddings)\n",
    "\n",
    "\n",
    "embeddings_en_pt = get_embeddings(sentences_en, pt_model)\n",
    "embeddings_de_pt = get_embeddings(sentences_de, pt_model)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d659327a-b6e2-48d4-a011-02433a90d4c6",
   "metadata": {},
   "source": [
    "### Optimize the Model with OpenVINO\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "\n",
    "The LaBSE model is quite large and can be slow to infer on some hardware, so let's optimize it with OpenVINO. [Model Conversion API](https://docs.openvino.ai/2024/openvino-workflow/model-preparation/conversion-parameters.html) accepts the PyTorch/Transformers model object and additional information about model inputs. An `example_input` is needed to trace the model execution graph, as PyTorch constructs it dynamically during inference. The converted model must be compiled for the target device using the `Core` object before it can be used.\n",
    "\n",
    "For starting work, we should select device for inference first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "732e0ed3-0216-45be-8580-09b830daf9da",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ipywidgets as widgets\n",
    "\n",
    "core = ov.Core()\n",
    "device = widgets.Dropdown(\n",
    "    options=core.available_devices + [\"AUTO\"],\n",
    "    value=\"AUTO\",\n",
    "    description=\"Device:\",\n",
    "    disabled=False,\n",
    ")\n",
    "\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "fc01c535-46ef-4cd2-9130-4dda7c023e64",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3bf1ed94aaf3439f9eae332586dd2d6d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/32 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1f033aa7971b4880b29f57b8d51f6e0e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/34 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 3 inputs with dynamic axis [batch_size, sequence_length] and type int64\n",
    "inputs_info = [([-1, -1], ov.Type.i64)] * 3\n",
    "ov_model = ov.convert_model(\n",
    "    pt_model,\n",
    "    example_input=tokenizer(\"test\", return_tensors=\"pt\").data,\n",
    "    input=inputs_info,\n",
    ")\n",
    "\n",
    "core = ov.Core()\n",
    "compiled_model = core.compile_model(ov_model, device.value)\n",
    "\n",
    "embeddings_en = get_embeddings(sentences_en, compiled_model)\n",
    "embeddings_de = get_embeddings(sentences_de, compiled_model)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9dcba47f-a932-44d9-88c3-824a32a51d35",
   "metadata": {},
   "source": [
    "On an Intel Core i9-10980XE CPU, the PyTorch model processed 40-43 sentences per second. After optimization with OpenVINO, the processing speed increased to 56-60 sentences per second. This is about 40% performance boost with just a few lines of code. Let's check if the model predictions remain within an acceptable tolerance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "fc2db8bc-24fe-485c-9ba5-78dca160bc09",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.all(np.isclose(embeddings_en, embeddings_en_pt.detach().numpy(), atol=1e-3))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "462735e3-ec90-4746-a0e5-9242ac0b6f90",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Calculate Sentence Alignment\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "With the embedding matrices from the previous step, we can calculate the alignment:\n",
    "1. Calculate sentence similarity between each pair of sentences.\n",
    "1. Transform the values in the similarity matrix rows and columns to a specified range, for example `[-1, 1]`.\n",
    "1. Compare the values with a threshold to get boolean matrices with 0 and 1.\n",
    "1. Sentence pairs that have 1 in both matrices should be aligned according to the model.\n",
    "\n",
    "We visualize the resulting matrix and also make sure that the result of the converted model is the same as the original one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "3c90120e-2ea4-40ee-ae6a-02dfca19bfaf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA90AAAICCAYAAADS0a7IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/OQEPoAAAACXBIWXMAAA9hAAAPYQGoP6dpAABe/UlEQVR4nO3deVhV5fr/8c9WRFHMETVwKDDAVBRFzQE7Tg2ipZaailphpqlZahLl0TRPmkODQ2paTpVDppwcyyn15FCWJSlW5gzmmAOKjOv3Rz/2VwIENnuxEN+v6+q62mu6771dcHPvtZ5n2QzDMAQAAAAAAJyuiNUJAAAAAABQWNF0AwAAAABgEppuAAAAAABMQtMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0A/mkdevWevXVV+2v9+zZIz8/P+3Zs8fCrJAXfn5+mj59utVpAABygDpc+FCHby3tHN+wYYPVqdzxaLrvQL/++qtefPFFtWrVSnXr1lVwcLCeeeYZLV682NS4Z86c0fTp0xUdHW1qHCts27ZNfn5+atGihVJTU61Op8BbvXq1FixYkOPtW7duLT8/Pz399NOZrl++fLn8/Pzk5+enqKioXOfz448/avr06bpy5Uqu93W22NhYjR49Wq1bt1adOnXUtGlTvfDCC/rhhx+sTs3u66+/lp+fnz7//PMst/n222/l5+enRYsWSZJeffVVBQYGptumd+/e8vPz04ABAzLsf+rUKfn5+emjjz7KsO52+IyAW6EOOx91OHeowxmlNahp/9WuXVtt2rTRyJEjdfLkyVwdK+3zyu6/lStXmvRuUNC4WJ0A8tePP/6oPn36yNPTU127dpWHh4dOnz6tn3/+WYsWLVLv3r1Ni3327FnNmDFDXl5eqlWrlmlxrPDll1/Ky8tLMTEx2r17t5o1a5btPo0aNdL+/ftVrFixfMiwYFmzZo1+//33LIt3ZooXL649e/bo3Llz8vDwSLdu9erVKl68uBISEhzKZ9++fZoxY4Y6d+6su+66K8f77d+/X0WLFnUoZmZ++OEH9e/fX5LUtWtX+fj46Pz581q1apV69eql119/3dSf0Zz617/+pdKlS2v16tXq2rVrptusWbNGRYsWVUhISLbH27p1q3755RfVqVMn221vl88IyAp12BzU4dyhDmetd+/eqlu3rpKTk3Xw4EEtW7ZM27Zt05dffqnKlSvn6Bivvfaarl27Zn+9fft2rVmzRhERESpXrpx9eYMGDZyaOwoumu47zOzZs1W6dGmtWLEiwy+1CxcuWJTV7e369evasmWLhg0bppUrV2r16tU5KvZFihRR8eLF8yHDwqFBgwaKiorSunXr1LdvX/vyP//8U3v37lW7du301VdfmZ5HamqqkpKSVLx4caf++12+fFlDhw5ViRIltGTJElWvXt2+7plnnlFYWJjeeust1a5d2/Ii7erqqocfflgrV67UmTNnMvwRkpCQoI0bN6pZs2aqUKHCLY/l6empa9euacaMGZo9e/Ytt72dPiMgK9Rh56MO54/CXofTBAUF6ZFHHpEkPfHEE7rnnns0fvx4RUZG6vnnn8/RMdq2bZvu9fnz57VmzRq1bdtWVatWzVN+hmEoISFBJUqUyNNxkL+4vfwOc+LECdWsWTPTbxEz++P4v//9r7p06aKAgAA1btxYL7/8sk6fPp1um969e6tDhw46fPiwevfurXr16ik4OFhz5861b7Nnzx49+eSTkqSIiIhMb6v5+eefFRYWpoYNG6pevXoKDQ3NcLvo9OnT5efnp+PHj+vVV19VUFCQGjZsqIiICMXHx2ea/5NPPql69eqpUaNG6tWrl/73v/+l22bbtm3q2bOn6tevr8DAQPXv31+///57Dj7Nv23cuFE3btzQI488ovbt2+vrr7/O0Te9WY0l+/TTT9WmTRsFBAToySef1N69e9W7d+90Vz/S9l23bp1mzZqlli1bqm7duurbt6+OHz+e7nhp/z6HDh1SaGio6tWrp3bt2tnH93z33Xfq2rWrAgIC9PDDD2vnzp0Zcj1z5owiIiLUrFkz1alTRyEhIVqxYkWm7ye7nHr37q1vvvlGMTEx9vOgdevW2X5exYsX10MPPaQ1a9akW75mzRrdddddatGiRYZ9Dh06pFdffVVt2rRR3bp11bx5c0VEROivv/6ybzN9+nRNmjRJktSmTRt7TqdOnZL093ixcePG6csvv1RISIjq1q2rHTt22NeljSVLOwceeeQR3bhxw378S5cuqUWLFnrqqaeUkpKS5ftbtmyZzp07p1deeSVdMylJJUqU0MSJE2Wz2TRz5kz78pUrV8rPz0/ff/+9Ro8erSZNmqhBgwYaOXKkLl++nCFGTs71tNvAz5w5oxdeeEGBgYF64IEH9Pbbb6fL/7HHHlNqaqrWrVuXIc4333yjq1evqmPHjlm+3zSlSpVS3759tXXrVh04cOCW2zryGQEFDXWYOkwdLph1OCsPPPCApL+HPV27dk3169fX+PHjM2z3559/qlatWpozZ06OjpucnKyZM2eqbdu2qlOnjlq3bq133nlHiYmJ6bZr3bq1nn/+ee3YscP+u2Dp0qWSpCtXruitt96yD7dq2bKlRo4cqYsXL6Y7RmpqarbnKcxF032H8fLy0oEDB/Tbb79lu+2sWbMUHh6uGjVq6NVXX1WfPn20a9cu9erVK8OYm8uXL6tfv37y9/dXeHi4vL29NWXKFG3btk2S5OPjoxdffFGS1L17d02aNEmTJk1So0aNJMl+3GvXrmnw4MF6+eWXdeXKFfXt21f79+/PkNtLL72ka9euadiwYXr00Ue1cuVKzZgxI902M2bM0MiRI+Xi4qIXX3xRQ4YMUZUqVbR79277NmnfWpYsWVIjRozQCy+8oMOHD6tnz572X/bZWb16tZo0aSIPDw+FhITo2rVr2rJlS472/afPPvtM48aNU5UqVfTKK68oKChIgwYN0p9//pnp9nPnztXGjRv17LPP6vnnn9fPP/+sESNGZNju8uXLGjBggAICAvTKK6/I1dVVw4YN07p16zRs2DA9+OCDGj58uOLj4/Xiiy8qLi7Ovu/58+fVrVs3+7/R66+/rurVq+v111/PdDxYdjkNGDBAtWrVUrly5eznwWuvvZajz6dDhw7av3+/Tpw4YV+2Zs0aPfzww3JxyXjjzs6dO3Xy5El16dJF//73v9W+fXutW7dO/fv3l2EYkqR27dqpQ4cOkv7+QzQtp/Lly9uPs3v3bk2YMEGPPvqoXnvtNXl5eWWIVaJECb399ts6ceKE3n33XfvycePG6erVq5owYcItb4HbsmWLihcvrvbt22e6vlq1amrYsKH27NmT7o+JtBh//PGHBg8erE6dOmn16tUaNGiQ/T1KuTvXU1JSFBYWprJly2rkyJFq3LixPv74Yy1btsy+TaNGjVSlShWtXr06Q65r1qyRm5tbhm/6s9K3b1+VKVMm28lw8vIZAQUFdZg6TB0umHU4K2nvtWzZsipVqpTatm2r9evXZ2jg16xZI8MwcvSFsySNGjVK06ZN0/3336+IiAg1atRIc+bM0csvv5xh26NHj2r48OFq3ry5Xn/9ddWqVUvXrl1Tr1699Mknn9iXP/XUUzpy5IjOnDmTbv+cnqcwkYE7yv/+9z+jVq1aRq1atYzu3bsbkyZNMnbs2GEkJiam2+7UqVNGrVq1jFmzZqVb/uuvvxr3339/uuWhoaGGr6+vsWrVKvuyhIQEo3nz5saQIUPsy/bv32/4+voaX3zxRbpjpqamGg899JDx7LPPGqmpqfbl8fHxRuvWrY1nnnnGvmzatGmGr6+vERERke4YgwYNMho3bmx/fezYMcPf398YNGiQkZKSkiGeYRhGXFycERQUZIwaNSrd+nPnzhkNGzbMsDwz58+fN+6//35j+fLl9mXdu3c3Bg4cmGHbVq1aGeHh4fbXu3fvNnx9fY3du3cbhvH3Z9a4cWPjiSeeMJKSkuzbrVy50vD19TVCQ0Mz7Pvoo48aCQkJ9uULFy40fH19jV9//dW+LO3fZ/Xq1fZlf/zxh+Hr62v4+/sbP/30k335jh07Mvwbvfbaa0bz5s2Nixcvpns/L7/8stGwYUMjPj4+1zn179/faNWqVaafaWZatWpl9O/f30hOTjaaN29uzJw50zAMwzh8+LDh6+trfPfdd8YXX3xh+Pr6Gvv377fvl5bbzdasWWP4+voa33//vX3ZvHnzDF9fX+PkyZMZtk/7nH7//fdM102bNi3dsqlTpxr+/v7G999/b6xfv97w9fU1FixYkO17DAoKMh577LFbbvPmm28avr6+xqFDhwzDMOzvuXPnzul+hufOnWv4+voamzZtMgwjd+d6eHi44evra8yYMSPdtp06dTI6d+6cbtnbb79t+Pr6GkeOHLEvu3r1qlG3bl1j2LBh6bYNDw836tevn25ZaGioERISYhiGYUyfPt3w9fU1fvnlF8MwDOPkyZOGr6+vMW/evDx9RkBBQx2mDhsGdbgg1uG0z2/FihXGhQsXjDNnzhjffPON0apVK8PPz8/+vtL+jbZt25Zu/44dO6Y7R272z/cXHR1t+Pr6Gq+//nq67SZOnGj4+voau3btsi9r1aqV4evra2zfvj3dtu+//77h6+trfP311xnipf2M5eacgLm40n2Had68uZYuXarWrVvr0KFDmjdvnsLCwtSyZUtt3rzZvt3GjRuVmpqqRx99VBcvXrT/V7FiRdWoUSPDrVglS5bU448/bn/t6uqqunXr5mi2x+joaB07dkwdO3bUX3/9ZY91/fp1NW3aVN9//32GmUifeuqpdK+DgoJ06dIl+zfDmzZtUmpqqgYNGqQiRdKf5jabTdLf375euXJFISEh6d5jkSJFVK9evRw9QmTt2rWy2Wx66KGH7Ms6dOig7du3Z3p776388ssvunTpkrp165bu2+KOHTuqTJkyme7TpUsXubq62l8HBQVJUobPvWTJkukmtPL29tZdd90lHx8f1atXz7487f/T9jcMQ19//bVat24twzDSfU4tWrTQ1atXM9wSnNOcHFG0aFE98sgjWrt2raS/J865++677TH+6ebxTgkJCbp48aL9PWZ3K/PNGjVqpJo1a+Zo28GDB6tmzZoKDw/X2LFj1bhxY/Xp0yfb/a5du6ZSpUrdcpu09TdfAZH+vmp180RAPXr0kIuLi/0KlyPneo8ePdK9btiwYYarTo899pgkpbvV8KuvvlJCQkKOv+lPk3a1+59Xym6Wl88IKCiow9RhiTosFbw6nOa1115T06ZNFRwcrP79+ys+Pl4TJ05U3bp1JUnNmjVTpUqV0t3p9dtvv+nXX3+118XspNXnZ555Jt3yZ599Nt36NFWrVlVwcHC6ZV9//bX8/f3Vrl27DMdP+xlLY+Y5gZxhIrU7UEBAgGbMmKHExEQdOnRImzZt0oIFCzR06FBFRkaqZs2aOnbsmAzDSFfEbvbPW4iqVKmS4Qe8TJky+vXXX7PN59ixY5Kk8PDwLLe5evVquoLn6emZbn3a2LjLly/L3d1dJ06cUJEiReTj45Nt3JsnA7mZu7t7trl/+eWXCggI0KVLl3Tp0iVJUq1atZSUlKQNGzaoe/fu2R4jTWxsrCRlGKvq4uKS6W1UUtafwz9vO8zs36d06dKqUqVKhmU373/x4kVduXJFy5YtS3dr8c3+OW4opzk5qmPHjlq8eLEOHTqkNWvWqH379hneW5pLly5pxowZWrduXYYJiq5evZrjmLmZ9MTV1VVvvfWWnnzySRUvXlxvvfVWlvndrFSpUulmOs1M2vp/Np41atTIcCwPDw/FxMRIyv25Xrx48XS39Ul//zz/8w9Yf39/+fr6as2aNRoyZIikvxvwcuXKZTq271ZKly6tPn36aPr06Tp48GCm413z8hkBBQl1OH1c6nD6ZTfvTx3+W37U4TSDBg1SUFCQihQponLlysnHxyfdz1uRIkXUsWNHLVmyRPHx8XJzc7PP3p42AVt2YmJiVKRIkQznmoeHh+666y57/U6T2fs/ceJElr8f/snscwLZo+m+g7m6uiogIEABAQG65557FBERoQ0bNmjw4MFKTU2VzWbT3LlzMx3/UrJkyXSv8/K4BuP/j+kZOXJklo8w+We8f35r/s9j5SbupEmTMjz6Qsr+PR07dsz+LMrMfumtXr06V8XeETn9HLJ6L1ktT9s/7crGY489ps6dO2e6rZ+fn0M5OapevXqqXr26/vOf/+jUqVO3vKL60ksvad++fQoLC1OtWrVUsmRJpaamql+/frnKJ7czhKZNEpSQkKDjx4+rWrVq2e7j4+OjgwcPKjExMd230Tf79ddfVaxYMd1zzz25yie353pufp47duyoqVOnKioqSlWqVNGePXvUvXv3TMf2Zadv375auHChZsyYken4QjM/I8AK1GHqMHU4Z/KjDqfx9fXNdvb7Tp066aOPPtKmTZvUoUMHrVmzxv44zdzI6ZcBeZ2p3OxzAtmj6YYk2Z+Pe/bsWUl/f8trGIaqVq2qe++91ykxsvrFkvaL0N3dPUeP+MiJ6tWrKzU1VX/88UeWf0Ckxa1QoYJDcVevXq1ixYpp0qRJGX6Z/fDDD1q8eLFiY2MzfLuYlbTtTpw4YZ8pU/p7dsu0GUbzW/ny5VWqVCmlpqY67d9GynmRyUpISIhmzZolHx+fLP99L1++rF27dmnIkCEaPHiwfXnalRVn5nOzQ4cOaebMmerSpYsOHTqkUaNGafXq1dkW4n/961/at2+f1q9fn+4W0TSnTp3SDz/8oKZNm2YovsePH093zly7dk3nzp1Ty5YtJeX9XL+VDh066J133tGaNWvk6emplJSUXN9anqZ06dLq27evpk+fnukfl3n5jICCjjpMHc4MdTj3HK3DueHr66v7779fq1evVpUqVRQbG6tRo0bleH8vLy+lpqbq+PHj6e4GOX/+vK5cuZLlnRU3q169eq5m+Ye1GNN9h9m9e3em32qljR3x9vaW9Pc3xkWLFtWMGTMybG8YRrpHPeSUm5ubpIy3stSpU0fVq1fXxx9/nOmto/+8bSon2rZtqyJFimjmzJkZxqGlvZ/g4GC5u7trzpw5SkpKynXc1atXq2HDhmrfvr39ERVp//Xr10+SMjxW41bq1KmjsmXLavny5UpOTk4XJ7fj0pylaNGievjhh/XVV19lOtOuI/820t/nQm5uK/unrl27avDgwbe8FTKrqwcLFy7MNB8pd7e6ZSYpKUkRERGqVKmSXn/9dU2YMEHnz5/XW2+9le2+3bt3V4UKFTR58uQMY6wSEhIUEREhwzA0aNCgDPsuW7Ys3Tm8ZMkSJScn25vuvJ7rt+Lp6amgoCCtW7dOX375papWrZqnZ2T37dtXd911V6aP/crLZwQUFNRh6nBuUIdzJy91OLcef/xxffvtt1q4cKHKli1rr7k58eCDD0rK+FnMnz8/3fpbeeihh3To0CFt3LgxwzquYBc8XOm+w4wfP17x8fFq166dvL29lZSUpB9//FHr16+Xl5eXunTpIunvb89eeuklTZ06VTExMWrbtq1KlSqlU6dOadOmTerWrZvCwsJyFbt69eq66667tHTpUpUqVUolS5ZUQECAqlWrpvHjx+u5555Thw4d1KVLF1WuXFlnzpzRnj175O7urtmzZ+cqVo0aNTRgwAB98MEH6tmzpx566CG5uroqKipKlSpV0vDhw+Xu7q433nhDI0eOVJcuXdS+fXuVL19esbGx2rZtmxo0aKDRo0dnevyff/5Zx48fV69evTJdX7lyZfs3oP37989Rzq6urhoyZIjefPNN9e3bV48++qhiYmK0cuXKDGN+8tPw4cO1Z88edevWTV27dlXNmjV1+fJlHThwQLt27dJ3332X62PWrl1b69at04QJE1S3bl2VLFkyR88ITePl5WUfQ5wVd3d3NWrUSPPmzVNSUpIqV66sb7/9NtNH0NSuXVuS9O6776p9+/YqVqyYWrVqleF2yuzMmjVL0dHRWrBggdzd3eXv769Bgwbpvffe0yOPPHLLIlquXDlNmzZN/fv3V+fOndW1a1f5+Pjo/PnzWrVqlY4fP67XX38904Y2KSlJTz/9tB599FEdPXpUn332mRo2bKg2bdrYPwtHz/WceOyxx/Tvf/9bZ8+e1YABAxw+jvR/Y7szm1AtL58RUFBQh6nDuUUdzrm81OHc6tChgyZPnqyNGzeqR48e6SY0zY6/v786d+6sZcuW6cqVK2rUqJGioqK0atUqtW3bNt2dFlkJCwvTV199paFDh+qJJ55Q7dq1dfnyZW3ZskVjx46Vv79/Xt4enIym+w4zcuRIbdiwQdu2bbNfHfP09FTPnj01cODAdJMX9e/fX/fcc48WLFhgv+pUpUoVNW/ePFe/mNMUK1ZMEydO1DvvvKM33nhDycnJmjBhgqpVq6YmTZpo2bJl+uCDD/TJJ5/o+vXr8vDwUEBAgMPjsYYOHaqqVavqk08+0bvvvis3Nzf5+fmluy21Y8eOqlSpkj788EN99NFHSkxMVOXKlRUUFGT/wyczaTNW3upzaN26taZPn65Dhw7l+BdfaGioDMPQ/Pnz9fbbb8vf31+zZs3S+PHjVbx48Ry+c+eqWLGiPv/8c82cOVMbN27UkiVLVLZsWdWsWdPhZzz27NlT0dHRWrlypRYsWCAvLy+HzqnsTJ06VW+++aY+++wzGYah5s2ba+7cuRlmAA0ICNDQoUO1dOlS7dixQ6mpqdq8eXOuiv2BAwc0Z84chYaGpiuW/fv31+bNmzVq1CitXbs20wnC0gQFBenLL7/UnDlztGHDBp07d07u7u4KDAzUf/7znyxnhx09erRWr16tadOmKSkpSSEhIRo1alS62/UcPddz4uGHH9abb76pxMTEHM/ceitpY7szu+Lh6GcEFBTUYepwblGHc8YZdTg3KlasqObNm2vbtm2ZDnnKzvjx41W1alWtWrVKmzZtUsWKFfX888+nuxX/VkqVKqVPP/1U06dP18aNG7Vq1SpVqFBBTZs2VeXKlXOdD8xlM7j/ACjQUlNT1bRpU7Vr107jx4+3Oh0UICtXrlRERIRWrFhhf5QJAMC5qMPIyqBBg/Tbb79leos3cDPGdAMFSEJCQoZxOJGRkbp06ZIaN25sUVYAANwZqMPIqbNnzzp8lRt3Hm4vBwqQn376SRMmTNAjjzyismXL6uDBg1qxYoV8fX1z/OxHAADgGOowsnPy5En9+OOPWrFihVxcXEx/LB0KB5puoADx8vJSlSpVtHjxYl2+fFllypTR448/rhEjRmT5XGIAAOAc1GFk5/vvv1dERIQ8PT01ceLETJ8xD/wTY7oBAAAAADAJY7oBAAAAADAJTTcAAAAAACah6QYAAAAAwCSFciK1pPNHHNrPzTPYyZkAAFBwJCfGOLwvtRUAgIxyUlu50g0AAAAAgElougEAAAAAMAlNNwAAAAAAJrF0TPfFixf1xRdf6KefftL58+clSRUrVlRgYKC6dOmi8uXLW5keAAAAAAB5YtmV7v379+uRRx7R4sWLVbp0aQUFBSkoKEilS5fW4sWL9eijjyoqKsqq9AAAAAAAyDObYRiGFYG7desmf39/jR07VjabLd06wzA0ZswY/frrr1q2bFmuj80MqwAAZMTs5QAAOFeBnr380KFD6tu3b4aGW5JsNpv69u2r6OhoCzIDAAAAAMA5LGu6K1aseMvbx6OiolSxYsV8zAgAAAAAAOeybCK1sLAw/fvf/9Yvv/yipk2b2hvs8+fPa9euXfr88881cuRIq9IDAAAAACDPLGu6e/XqpXLlymnBggVasmSJUlJSJElFixZV7dq1NWHCBLVv396q9AAAAAAAyDPLJlK7WVJSkv766y9JUrly5VSsWLG8HY/JXgAAyICJ1AAAcK6c1FZLn9OdplixYqpUqZLVaQAAAAAA4FQF4kq3s7m4euVbrPjYHQ7txzf/AID8lpcr3QW9tlJXAQBWKNCPDAMAAAAAoLCj6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0AwAAAABgEppuAAAAAABMQtMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9gMwzCsTsLZXFy9rE4hW/GxO3K9j5tnsAmZAADuFMmJMQ7vW9BrqyN1VaK2AgDyJie1lSvdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgEksb7pv3LihvXv36vDhwxnWJSQkKDIyMv+TAgAAAADACSxtuo8ePar27dsrNDRUHTt2VGhoqM6ePWtff/XqVUVERFiYIQAAAAAAjrO06Z4yZYruu+8+7dy5Uxs2bFCpUqXUo0cPxcbGWpkWAAAAAABOYWnTvW/fPg0bNkzly5dXjRo1NHv2bLVo0UK9evXSyZMnrUwNAAAAAIA8s7TpvnHjhlxcXOyvbTabxo4dq1atWik0NFTHjh2zLjkAAAAAAPLIJftNzOPt7a2oqCj5+PikWz569GhJ0sCBA61ICwAAAAAAp7D0Sne7du20du3aTNeNHj1aISEhMgwjn7MCAAAAAMA5bEYh7GpdXL2sTiFb8bE7cr2Pm2ewCZkAAO4UyYkxDu9b0GurI3VVorYCAPImJ7WVpvs2QqMOAMiLwtx0O4raCgDIi5zUVktvLwcAAAAAoDCj6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0AwAAAABgEppuAAAAAABMQtMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9gMwzCsTsLZXFy9rE6hwIiP3eHQfm6ewU7OBABgteTEGIf3pbb+H2orACBNTmorV7oBAAAAADAJTTcAAAAAACah6QYAAAAAwCQFrukuhEPMAQAAAAB3qALXdNetW1d//PGH1WkAAAAAAJBnLlYFnjBhQqbLU1JS9OGHH6ps2bKSpIiIiHzMCgAAAAAA57Gs6V64cKH8/f1VunTpdMsNw9Aff/whNzc32Ww2i7IDAAAAACDvLGu6hw0bpmXLlik8PFxNmza1L69du7YmTpyomjVrWpUaAAAAAABOYVnT3b9/fz3wwAN65ZVX1Lp1aw0bNkzFihWzKh0AAAAAAJzO0onUAgICtHLlSl28eFFPPPGEfvvtN24pBwAAAAAUGpZd6U5TqlQpvf3221q7dq2eeeYZpaSkWJ0SAAAAAABOYXnTnSYkJEQNGzbUL7/8Ik9PT6vTAQAAAAAgzwpM0y1JVapUUZUqVaxOAwAAAAAAp7AZhmFYnYSzubh6WZ3CbS8+dkeu93HzDDYhEwCAsyQnxji8L7U176itAFD45KS2WjqRGgAAAAAAhRlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYxGYYhmF1Es7m4upldQp3pPjYHQ7t5+YZ7ORMAACZSU6McXhfaqs1HKmt1FUAyD85qa1c6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0AwAAAABgEppuAAAAAABMYmnTfeDAAZ08edL+OjIyUk899ZQefPBB9ejRQ2vXrrUwOwAAAAAA8sbSpjsiIsLedH/++ecaM2aM6tSpowEDBqhu3boaNWqUVqxYYWWKAAAAAAA4zMXK4MePH1eNGjUkSZ999plef/11devWzb6+bt26mj17tp588kmrUgQAAAAAwGGWXukuUaKE/vrrL0nSmTNnFBAQkG59vXr1dOrUKStSAwAAAAAgzyxtulu2bKklS5ZIkho1aqQNGzakW79+/XpVr17ditQAAAAAAMgzS28vHzFihHr06KHQ0FDVqVNH8+fP13fffScfHx8dPXpUP/30k2bOnGlligAAAAAAOMzSK92VK1dWZGSk6tevrx07dsgwDO3fv1/ffvutKleurCVLlujBBx+0MkUAAAAAABxmMwzDsDoJZ3Nx9bI6hTtSfOwOh/Zz8wx2ciYAgMwkJ8Y4vC+11RqO1FbqKgDkn5zUVppuWI4/KAAgf9B03xn4EhwA8k9Oaqult5cDAAAAAFCY0XQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElshmEYVifhbC6uXlanAJPFx+7It1hunsH5FgsAzJScGOPwvtTWwo/aCgC5l5PaypVuAAAAAABMQtMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCSWN92ffPKJRo4cqbVr10qSIiMj1b59ez3yyCN65513lJycbHGGAAAAAAA4xsXK4B988IHmzZunFi1aaMKECYqJidFHH32kp59+WkWKFNGCBQvk4uKiF1980co0AQAAAABwiKVN96pVqzRx4kQ99NBDOnTokLp06aKJEyfqsccekyR5e3tr8uTJNN0AAAAAgNuSpbeXnz17VnXq1JEk+fv7q0iRIqpVq5Z9/f3336+zZ89alR4AAAAAAHliadNdsWJFHT58WJJ07NgxpaSk2F9L0uHDh1W+fHmr0gMAAAAAIE8svb28Y8eOCg8PV5s2bbRr1y7169dPkyZN0qVLl2Sz2TR79mw9/PDDVqYIAAAAAIDDLG26X3zxRZUoUUI//fSTunXrpv79+8vf31+TJ09WfHy8WrduraFDh1qZIgAAAAAADrMZhmFYnYSzubh6WZ0CCpH42B253sfNM9iETAAgb5ITYxzel9oKZ6K2AigsclJbLX9ONwAAAAAAhRVNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT2AzDMKxOwtlcXL2sTgFwSHzsjnyJ4+YZnC9xABQsyYkxDu9LbcXtKL/qqkRtBe5UOamtXOkGAAAAAMAkLlYnkJiYqE2bNumnn37S+fPnJUkVK1ZUYGCg2rRpI1dXV4szBAAAAADAMZY23cePH1dYWJjOnj2revXqqUKFCpKk6OhoLV26VFWqVNHcuXNVo0YNK9MEAAAAAMAhljbdb7zxhnx9fRUZGSl3d/d06+Li4jRy5EiNGzdOH330kUUZAgAAAADgOEvHdP/444966aWXMjTckuTu7q6hQ4dq7969FmQGAAAAAEDeWdp0ly5dWjExWc/2FhMTo9KlS+djRgAAAAAAOI+lt5d37dpV4eHheuGFF/TAAw+oYsWKkqTz589r9+7dmjVrlkJDQ61MEQAAAAAAh1nadA8dOlRubm6aN2+eJk6cKJvNJkkyDEMVK1ZUv3799Nxzz1mZIgAAAAAADrMZhmFYnYQknTx5Mt0jw6pVq+bwsVxcvZyVFpCv4mN35EscN8/gfIkDoGBJTsx6SFd2qK24HeVXXZWorcCdKie11dIx3TerVq2aAgMDFRgYaG+4T58+rYiICIszAwAAAADAMQWm6c7M5cuXFRkZaXUaAAAAAAA4xNIx3Zs3b77l+pMnT+ZTJgAAAAAAOJ+lY7r9/f1ls9l0qxRsNpuio6NzdVzGnQG35ugYN8arAbc3xnQD5qG2AnemnNRWS690e3h4aMyYMWrbtm2m66Ojo9WlS5d8zgoAAAAAAOewdEx37dq1deDAgSzXZ3cVHAAAAACAgszSK939+vXT9evXs1xfvXp1LVq0KB8zAgAAAADAeSxtuoOCgm65vmTJkmrcuHE+ZQMAAAAAgHMV6EeGAQAAAABwO6PpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJrEZhmFYnYSzubh6WZ0CUCjFx+7I9T5unsEmZALAEcmJMQ7vS20FzEFtBW5vOamtXOkGAAAAAMAkBaLp/vPPP3Xt2rUMy5OSkvT9999bkBEAAAAAAHlnadN99uxZPfnkk2rVqpUaNWqkkSNHpmu+L1++rD59+liYIQAAAAAAjrO06Z46daqKFCmi5cuXa968eTp8+LD69Omjy5cv27cphEPOAQAAAAB3CEub7p07d2rUqFGqW7eumjVrpqVLl8rDw0N9+/bVpUuXJEk2m83KFAEAAAAAcJilTXdcXJzuuusu+2tXV1fNmDFDXl5e6tOnjy5cuGBhdgAAAAAA5I2lTXfVqlX166+/plvm4uKi999/X9WqVdOAAQMsygwAAAAAgLyztOlu2bKlli9fnmF5WuPt7+/PmG4AAAAAwG3LZljY1SYnJ+vGjRtyd3fPcv2ZM2fk5eWVq+O6uOZuewA5Ex+7I9f7uHkGm5AJAEckJ8Y4vC+1FTAHtRW4veWktlp6pdvFxSXLhluSzp07pxkzZuRjRgAAAAAAOI+lTXd2Ll++rMjISKvTAAAAAADAIS5WBt+8efMt1588eTKfMgEAAAAAwPksHdPt7+8vm812y8nSbDaboqOjc3Vcxp0BBYcjY9UkxqsBZmBMN1A4MA4cKDhyUlstvdLt4eGhMWPGqG3btpmuj46OVpcuXfI5KwAAAAAAnMPSMd21a9fWgQMHslyf3VVwAAAAAAAKMkuvdPfr10/Xr1/Pcn316tW1aNGifMwIAAAAAADnsbTpDgoKuuX6kiVLqnHjxvmUDQAAAAAAzlWgHxkGAAAAAMDtjKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDbDMAyrk3A2F1cvq1MAAKDASU6McXhfaisAABnlpLZypRsAAAAAAJPQdAMAAAAAYJIC2XS3adNGx44dszoNAAAAAADyxMXK4IsWLcp0+enTp7Vy5UpVrFhRktSnT5/8TAsAAAAAAKewdCI1f39/Va5cWUWLFk23PDY2VpUqVZKLi4tsNps2b96cq+My2QsAABkxkRoAAM6Vk9pq6ZXubt26af/+/Zo6dap8fHzsy2vXrq2PP/5YNWvWtDA7AAAAAADyxqGm+/r16/rwww+1e/duXbhwQampqenW5/TK9Lhx47Rx40aFhYWpX79+Cg0NdSQdAAAAAAAKJIea7lGjRum7777T448/Lg8PD9lsNocTaNeunQICAjRy5Eh98803mjBhgsPHAgAAAACgIHGo6d6+fbvmzJmjhg0bOiWJypUra8GCBfrwww/VuXNnWTjMHAAAAAAAp3Go6b7rrrtUtmxZpyZis9n0/PPPq3nz5vrhhx/k4eHh1OMDAAAAAJDfHHpO99ChQ/X+++8rPj7e2fmoTp066tu3r8qUKaPTp08rIiLC6TEAAAAAAMgPOX5kWKdOndKN3T5+/LgMw1DVqlXl4pL+gvmqVaucktyhQ4fUuXNnRUdH52o/HmsCAEBGPDIMAADncuojw9q2bZunZDKT3SznJ0+edHpMAAAAAADyS46vdJvB399fNpvtlhOn2Ww2rnQDd6D42B253sfNM9iETIDCgyvdwJ3LkboqUVuB7Dj1SvfNTp8+LZvNpipVqkiS9u/fr9WrV6tmzZrq3r17jo/j4eGhMWPGZHkVPTo6Wl26dHEkRQAAAAAALOfQRGrDhw/X7t27JUnnzp3T008/raioKL377ruaMWNGjo9Tu3ZtHThwIMv12V0FBwAAAACgIHOo6f79998VEBAgSVq/fr18fX21dOlSTZkyJVeTqPXr10+BgYFZrq9evboWLVrkSIoAAAAAAFjOodvLk5OT5erqKknauXOnWrduLUny9vbWuXPncnycoKCgW64vWbKkGjdu7EiKAAAAAABYzqEr3TVr1tTSpUu1d+9e7dy5Uy1btpQknT17VmXLlnVmfgAAAAAA3LYcarpHjBihZcuWqXfv3goJCZG/v78kacuWLfbbzgEAAAAAuNM5dHt5kyZNtHv3bsXFxalMmTL25d26dZObm5vTkgMAAAAA4HbmUNMtSUWLFk3XcEtS1apV85wQAAAAAACFRY6b7s6dO2vBggUqU6aMOnXqJJvNluW2uZnBHAAAAACAwirHTXebNm3sM5a3bdvWtIQAAAAAACgsbIZhGFYn4Wwurl5WpwAgj+Jjd+R6HzfPYBMyAQqP5MQYh/eltgK3N0fqqkRtBbKTk9rq0OzlAAAAAAAgezm+0t2oUaNbjuO+2XfffZenpPKKb+OBOxNXx4Fb40o3gNyitgK3lpPamuMx3a+99lqekgEAAAAA4E6Tq9nLne3PP/+Uq6urypcvL0nau3evlixZotOnT8vT01O9evVSYGCg0+MCAAAAAJAfHBrTHRcXl+V/iYmJOT7OkCFD9PPPP0uSNm3apN69e+v69etq0KCBbty4od69e2vr1q2OpAgAAAAAgOVyfKX7ZkFBQbcc312lShV17txZgwcPVpEiWff1hw8fVs2aNSVJH374oV5++WX179/fvv6TTz7RtGnT1KpVK0fSBAAAAADAUg413RMnTtS7776rzp07KyAgQJK0f/9+RUZGauDAgbp48aI+/vhjubq6asCAAVkep2jRorp27Zok6dSpU2rZsmW69S1bttSUKVMcSREAAAAAAMs51HSvWrVK4eHhat++vX1Z69at5evrq2XLlmnhwoW6++67NXv27Fs23Y0aNdLatWvl7++vWrVq6bvvvpO/v799/Z49e1S5cmVHUgQAAAAAwHIONd379u3T2LFjMyy///779dNPP0mSGjZsqNOnT9/yOCNGjFDPnj119uxZNWzYUO+++66ioqLk4+Ojo0ePat26dZnGAQAAAADgduDQRGp33323VqxYkWH5ihUrVKVKFUnSpUuXdNddd93yOD4+Pvr888+VlJSkefPmKT4+XqtXr9aMGTN0/Phxvfvuu+rSpYsjKQIAAAAAYDmHrnSPHDlSQ4cO1fbt21W3bl1J0i+//KIjR45o2rRpkqSoqKh0t59npXr16nrnnXdkGIYuXLig1NRUlStXTsWKFXMkNQAAAAAACgybYRiGIzuePHlSy5Yt07FjxyRJ9957r7p3766qVas6LbnTp09r2rRpmjBhQq72c3H1cloOAG4f8bE7cr2Pm2ewCZkABVNyYozD+1JbgTsTtRW4tZzUVoeb7vxw6NAhde7cWdHR0bnajz8MgDsTfxgAt0bTDSC3qK3AreWktjp0e7kkXblyRfv379eFCxf0z769U6dOOTrG5s2bb7n+5MmTjqYHAAAAAIDlHGq6t2zZohEjRuj69etyd3eXzWazr7PZbDluugcNGiSbzZahab/ZzccGAAAAAOB24tDt5Q8//LBatmypYcOGyc3NzeHgwcHBGjNmjNq2bZvp+ujoaHXp0oXbywGYxpHb5iRuncPtidvLAeQHaivuJDmprQ49MuzMmTPq06dPnhpuSapdu7YOHDiQ5frsroIDAAAAAFCQOXR7eYsWLRQVFaVq1arlKXi/fv10/fr1LNdXr15dixYtylMMAAAAAACs4lDT/eCDD2ry5Mn6448/5OvrKxeX9Idp06ZNjo4TFBR0y/UlS5ZU48aNHUkRAAAAAADLOTSm29/fP+sD2my5HoPtbIw7A5BTjDvDnYQx3QDyA7UVdxLTHhl26NAhR3YDAAAAAOCOkquJ1J577jldvXrV/vrDDz/UlStX7K//+usvtW/f3nnZAQAAAABwG8tV0/2///1PiYmJ9tezZ8/W5cuX7a9TUlJ09OhR52UHAAAAAMBtLFdN9z+Hf/M4LwAAAAAAsubQc7oBAAAAAED2ctV022w22Ww2s3IBAAAAAKBQydXs5YZh6NVXX5Wrq6skKTExUW+88Ybc3NzsrwEAAAAAwN9y1XR37tw53evHHnsswzadOnXKU0IAAAAAABQWNqMQzobm4upldQoACrn42B253sfNM9iETICcS06McXhfaisAs1FbcTvKSW1lIjUAAAAAAEySq9vLzbB161bt379fLVq0UMOGDbVr1y59/PHHSk1N1UMPPaTu3btbnSIAAAAAAA6x9Er30qVLNXjwYG3btk39+/fXf//7Xw0aNEiVK1eWl5eX3nrrLS1cuNDKFAEAAAAAcJilV7oXL16sMWPGqFu3btq9e7f69++v8PBw9erVS5JUv359zZs3T3379rUyTQAAAAAAHGLple5Tp06pRYsWkqQHHnhAqampatSokX1948aNFRsba1V6AAAAAADkiaVNd9myZe1N9ZkzZ5ScnKzTp0/b18fGxqpMmTJWpQcAAAAAQJ5Yent5mzZt9Prrr6tTp07asmWLOnXqpIkTJ8pms8lms2nSpElq3ry5lSkCAAAAAOAwS5vuESNGKCkpSevWrVNgYKBGjRqlxYsX64UXXlBycrIaNWqk4cOHW5kiAAAAAAAOsxmGYVidxD8lJCQoKSlJ7u7uDu3v4url5IwAIL342B253sfNM9iETICcS06McXhfaisAs1FbcTvKSW21dEx3VooXLy53d3edPn1aERERVqcDAAAAAIBDCmTTneby5cuKjIy0Og0AAAAAABxi6ZjuzZs333L9yZMn8ykTAAAAAACcz9Ix3f7+/rLZbLpVCjabTdHR0bk6LuPOABQmjHGDszCmGwD+Rm2Fs+Sktlp6pdvDw0NjxoxR27ZtM10fHR2tLl265HNWAAAAAAA4h6VjumvXrq0DBw5kuT67q+AAAAAAABRkll7p7tevn65fv57l+urVq2vRokX5mBEAAAAAAM5jadMdFBR0y/UlS5ZU48aN8ykbAAAAAACcq0A/MgwAAAAAgNsZTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0AwAAAABgEppuAAAAAABMQtMNAAAAAIBJaLoBAAAAADCJzTAMw+oknM3F1cvqFADAUvGxO3K9j5tnsAmZoCBJToxxeF9qK4A7HbUVmclJbXXJhzyytX//fu3bt0/nz5+XJFWsWFGBgYEKCAiwODMAAAAAABxnadN94cIFDRkyRD/++KM8PT1VoUIF+/IJEyaoQYMGmj59un05AAAAAAC3E0ub7rFjxyo1NVXr1q2Tt7d3unVHjhzRa6+9prFjx2ratGkWZQgAAAAAgOMsbbp37NihTz/9NEPDLUne3t4aNWqUevfubUFmAAAAAADknaWzl7u6uiouLi7L9deuXZOrq2s+ZgQAAAAAgPNY2nS3b99er776qjZu3Jiu+Y6Li9PGjRsVERGhDh06WJghAAAAAACOs/T28oiICKWmpurll19WSkqKihUrJklKSkpS0aJF9eSTTyo8PNzKFAEAAAAAcFiBeE53XFycfvnll3SPDKtTp47c3d0dOh7PEgVwp+NZosgMz+kGAMdRW5GZ2+Y53e7u7nrggQesTgMAAAAAAKeydEy3JN24cUN79+7V4cOHM6xLSEhQZGRk/icFAAAAAIATWNp0Hz16VO3bt1doaKg6duyo0NBQnTlzxr7+6tWrioiIsDBDAAAAAAAcZ2nTPWXKFN13333auXOnNmzYoFKlSqlnz56KjY21Mi0AAAAAAJzC0onUmjVrpvnz58vPz0+SZBiG3njjDW3fvl2LFi2Sm5ubgoODFR0dnavjMtkLAOSeIxPESEwSczthIjUAyF/U1sIvJ7XV0ivdN27ckIvL/83lZrPZNHbsWLVq1UqhoaE6duyYdckBAAAAAJBHls5e7u3traioKPn4+KRbPnr0aEnSwIEDrUgLAAAAAACnsPRKd7t27bR27dpM140ePVohISEqAI8RBwAAAADAIZaO6TYL484AIPcYd1b4MaYbAPIXtbXwK/BjugEAAAAAKMxougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElshmEYVifhbC6uXlanAAB3jPjYHbnex80z2IRMkJ3kxBiH96W2AkD+obbePnJSWwvEle7U1NQsl8fGxuZzNgAAAAAAOIelTXdcXJyGDh2q+vXrq1mzZnr//feVkpJiX3/x4kW1adPGwgwBAAAAAHCci5XB33vvPR06dEiTJk3S1atXNWvWLB08eFDTp0+Xq6urJKkQ3v0OAAAAALhDWHqle/PmzRo3bpweeeQRde3aVStWrNDFixc1cOBAJSYmSpJsNpuVKQIAAAAA4DBLm+6LFy/K09PT/rp8+fKaP3++rl27pueee07x8fEWZgcAAAAAQN5Y2nTffffdOnLkSLpl7u7u+uijj5SQkKDBgwdblBkAAAAAAHlnadPdokULffHFFxmWlypVSvPmzVPx4sUtyAoAAAAAAOewdCK1IUOG6OzZs5muc3d318cff6yDBw/mc1YAAAAAADiHpU13mTJlVKZMmSzXu7u7q3HjxvmYEQAAAAAAzmPp7eWSdOPGDe3du1eHDx/OsC4hIUGRkZH5nxQAAAAAAE5gadN99OhRtW/fXqGhoerYsaNCQ0PT3W5+9epVRUREWJghAAAAAACOs7TpnjJliu677z7t3LlTGzZsUKlSpdSjRw/FxsZamRYAAAAAAE5hMwzDsCp4s2bNNH/+fPn5+UmSDMPQG2+8oe3bt2vRokVyc3NTcHCwoqOjc3VcF1cvM9IFAOC2lpwY4/C+1FYAADLKSW219Er3jRs35OLyf3O52Ww2jR07Vq1atVJoaKiOHTtmXXIAAAAAAOSRpbOXe3t7KyoqSj4+PumWjx49WpI0cOBAK9ICAAAAAMApLL3S3a5dO61duzbTdaNHj1ZISIgsvPsdAAAAAIA8sXRMt1kYdwYAQEaM6QYAwLkK/JhuAAAAAAAKM5puAAAAAABMQtMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk7hYnQAA4M4TH7vDof3cPIOdnAkAAIUDtbXg4ko3AAAAAAAmKZBNd58+fRQTE2N1GgAAAAAA5Imlt5dv3rw50+V79+7VN998oypVqkiS2rRpk59pAQAAAADgFDbDMAyrgvv7+8tms+lWKdhsNkVHR+fquC6uXnlNDQBgIsadWSM50fG7yKitAFCwUVutkZPaaumV7hYtWqho0aJ66623VKFCBfvy2rVr67///a9q1qxpYXYAAAAAAOSNpWO6582bp6ZNm+qJJ57Q1q1brUwFAAAAAACns3witaefflqzZs3SlClTNHr0aMXHx1udEgAAAAAATmF50y1JtWrV0hdffCFJ6tSp0y3HeAMAAAAAcLuwdEz3zUqUKKFx48Zp8+bN2rNnj8qVK2d1SgAAAAAA5EmBabrTtGnThkeEAQAAAAAKBctvL79x44b27t2rw4cPZ1iXkJCgyMjI/E8KAAAAAAAnsLTpPnr0qNq3b6/Q0FB17NhRoaGhOnv2rH391atXFRERYWGGAAAAAAA4ztKme8qUKbrvvvu0c+dObdiwQaVKlVKPHj0UGxtrZVoAAAAAADiFzbBwqvBmzZpp/vz58vPzkyQZhqE33nhD27dv16JFi+Tm5qbg4GBFR0fn6rgurl5mpAsAwG0tOTHG4X2prQAAZJST2mrple4bN27IxeX/5nKz2WwaO3asWrVqpdDQUB07dsy65AAAAAAAyCNLZy/39vZWVFSUfHx80i0fPXq0JGngwIFWpAUAAAAAgFNYeqW7Xbt2Wrt2babrRo8erZCQEFl49zsAAAAAAHli6ZhuszDuDACAjBjTDQCAcxX4Md0AAAAAABRmNN0AAAAAAJiEphsAAAAAAJPQdAMAAAAAYBKabgAAAAAATELTDQAAAACASWi6AQAAAAAwCU03AAAAAAAmcbE6AQAAcio+dkeu93HzDDYhEwAACgdqq/ksbboTExNls9lUrFgxSdKJEyf0xRdfKDY2Vp6ennryySdVrVo1K1MEAAAAAMBhlt5eHhYWps2bN0uSfvjhB4WEhGjr1q1KTk7W9u3b1bFjR+3bt8/KFAEAAAAAcJilV7oPHjwof39/SdJ7772nnj17KiIiwr7+vffe06RJk7RkyRKrUgQAAAAAwGGWXulOTU1VSkqKJOnIkSPq3LlzuvVdunTRoUOHrEgNAAAAAIA8s7TpDggI0NatWyVJ1apVy9BgR0dHq2zZshZkBgAAAABA3ll6e/lLL72k5557TvHx8erQoYMmTpyoY8eOycfHR0ePHtXixYvVv39/K1MEAAAAAMBhNsMwDCsT2LdvnyZOnKiff/453fJKlSopLCxMffv2zfUxXVy9nJUeAKAA4bEmeZOcGOPwvtRWACicqK15k5PaavlzugMDA7Vs2TJdvHhRJ0+eVGpqqjw8PFS1alWrUwMAAAAAIE8sb7rTlC9fXuXLl7c6DQAAAAAAnMbSidQk6caNG9q7d68OHz6cYV1CQoIiIyPzPykAAAAAAJzA0qb76NGjat++vUJDQ9WxY0eFhobq7Nmz9vVXr15N99xuAAAAAABuJ5Y23VOmTNF9992nnTt3asOGDSpVqpR69Oih2NhYK9MCAAAAAMApLG269+3bp2HDhql8+fKqUaOGZs+erRYtWqhXr146efKklakBAAAAAJBnlk6kduPGDbm4/F8KNptNY8eO1bhx4xQaGqqpU6damB0AoKBx5BEljjwKxdFYAADcbvKrtt7JddXSptvb21tRUVHy8fFJt3z06NGSpIEDB1qRFgAAAAAATmHp7eXt2rXT2rVrM103evRohYSEyDCMfM4KAAAAAADnsBmFsKt1cfWyOgUAQAHB7eX/JzkxxuF9qa0AgDTcXv5/clJbLX9ONwAAAAAAhRVNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMInNMAzD6iSczcXVy+oUAAC3ufjYHbnex80z2IRMnCc5McbhfamtAIC8cKSuSoWjtrrkQx63dOjQIf3yyy9q0qSJqlWrpt9//12ffvqpUlNT1a5dOwUHF+wPGQAAAACArFh6e/nXX3+tLl26aPLkyXrssce0c+dO9ezZU8ePH1dMTIyef/55rV692soUAQAAAABwmKVXumfPnq0hQ4Zo4MCBWrt2rYYOHaqnn35agwYNkiR9/PHH+uijj9SxY0cr0wQAAAAAwCGWXuk+evSovaFu37694uPj1bZtW/v6du3a6fjx41alBwAAAABAnljadJcqVUqXLl2SJF25ckXJycn215L0119/qVSpUtYkBwAAAABAHlnadDdt2lTjxo3Tl19+qfDwcDVv3lzvvPOO/vjjDx05ckSTJ09WgwYNrEwRAAAAAACHWdp0h4eHy93dXWPGjFFSUpLee+891alTRyEhIQoJCdHZs2c1YsQIK1MEAAAAAMBhBfI53SdPnlR8fLy8vb3l4pL7ud54ligAIK94Tnd61FYAQF7wnO4Cplq1alanAAAAAABAnll6e7kk3bhxQ3v37tXhw4czrEtISFBkZGT+JwUAAAAAgBNY/siw9u3bKzQ0VB07dlRoaKjOnj1rX3/16lVFRERYmCEAAAAAAI6ztOmeMmWK7rvvPu3cuVMbNmxQqVKl1KNHD8XGxlqZFgAAAAAATmHpRGrNmjXT/Pnz5efnJ0kyDENvvPGGtm/frkWLFsnNzU3BwcGKjo7O1XGZ7AUAYIWCPvkaE6kBAG43haG2Wnql+8aNG+lmJ7fZbBo7dqxatWql0NBQHTt2zLrkAAAAAADII0tnL/f29lZUVJR8fHzSLR89erQkaeDAgVakBQAAAACAU1h6pbtdu3Zau3ZtputGjx6tkJAQFcDHiAMAAAAAkCOWjuk2C+POAABWKAzjzrJCbQUAWKEw1FbLn9MNAAAAAEBhRdMNAAAAAIBJaLoBAAAAADAJTTcAAAAAACah6QYAAAAAwCQ03QAAAAAAmISmGwAAAAAAk9B0AwAAAABgEppuAAAAAABMYjMMw7A6CWdzcfWyOgUAAHIkPnaHQ/u5eQbnep/kxBiHYknUVgDA7aOg1VaudAMAAAAAYBIXqxOQpP3792vfvn06f/68JKlixYoKDAxUQECAxZkBAAAAAOA4S5vuCxcuaMiQIfrxxx/l6empChUq2JdPmDBBDRo00PTp0+3LAQAAAAC4nVjadI8dO1apqalat26dvL290607cuSIXnvtNY0dO1bTpk2zKEMAAAAAABxnadO9Y8cOffrppxkabkny9vbWqFGj1Lt3bwsyAwAAAAAg7yydSM3V1VVxcXFZrr927ZpcXV3zMSMAAAAAAJzH0qa7ffv2evXVV7Vx48Z0zXdcXJw2btyoiIgIdejQwcIMAQAAAABwnKW3l0dERCg1NVUvv/yyUlJSVKxYMUlSUlKSihYtqieffFLh4eFWpggAAAAAgMNshmEYVicRFxenX375Jd0jw+rUqSN3d3eHjufi6uXM9AAAME187A6H9nPzDM71PsmJMQ7FkqitAIDbR0GrrZbeXi5Jf/zxh7766it5eHioQ4cOqlWrltavX6///Oc/2rVrl9XpAQAAAADgMEub7u3bt+vxxx/XpEmT1LlzZ23fvl2hoaE6ceKEYmNjFRYWRuMNAAAAALhtWdp0f/DBBwoLC9OePXs0YcIEjRgxQt26ddP8+fO1cOFChYWFae7cuVamCAAAAACAwywd092wYUOtXLlSNWrUUGpqqurWravPP/9c999/vyTpt99+0zPPPKNvv/02V8dl3BkAoLBzZLxasYreDsejtgIACjuzaqvlY7ptNpskqUiRInJ1dVXp0qXt60qVKqWrV69alRoAAAAAAHliadPt5eWlY8eO2V8vW7ZMd999t/316dOn5eHhYUFmAAAAAADknaXP6e7Ro4dSU1Ptr319fdOt3759ux544IH8TgsAAAAAAKcoEM/pdjbGnQEACjvGdAMA4FyFdkw3AAAAAACFFU03AAAAAAAmoekGAAAAAMAkNN0AAAAAAJiEphsAAAAAAJPQdAMAAAAAYBKabgAAAAAATELTDQAAAACASWi6AQAAAAAwic0wDMPqJAAAAAAAKIy40g0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMMkd03R/+umnat26terWrauuXbtq//79To8xZ84cPfHEEwoMDFTTpk31wgsv6MiRI06Pk5kPP/xQfn5++s9//mPK8c+cOaMRI0aoSZMmCggIUMeOHRUVFeX0OCkpKXrvvffUunVrBQQEqG3btpo5c6YMw8jzsb///nsNGDBALVq0kJ+fnzZt2pRuvWEYev/999WiRQsFBATo6aef1rFjx5weKykpSZMnT1bHjh1Vv359tWjRQiNHjtSZM2ec/p5uNnr0aPn5+WnBggWOvKUcxfrjjz80YMAANWzYUPXr19cTTzyh2NhYp8e6du2axo0bp5YtWyogIEDt27fXkiVLch0nJz+zCQkJGjt2rJo0aaLAwEANGTJE58+fd3qsS5cu6c0339TDDz+sgIAA/etf/9L48eN19epVU95XGsMw1K9fv2zPn7zG2rdvn/r06aP69eurQYMG6tWrl27cuOH0WOfOndMrr7yi5s2bq379+urcubO++uqrXL+vzz77TB07dlSDBg3UoEEDde/eXdu2bbOvd9Z5cas4zjwnzEJtzRtqq3PiOLOu5uQ93YzamhG19faprYWxrmYXy6raekc03evWrdOECRM0aNAgrVq1Sv7+/goLC9OFCxecGue7775Tr169tHz5cs2fP1/JyckKCwvT9evXnRrnn/bv36+lS5fKz8/PlONfvnxZPXr0ULFixTR37lytXbtW4eHhKlOmjNNjzZ07V0uWLNHo0aO1bt06jRgxQvPmzdPixYvzfOzr16/Lz89PY8aMyTL24sWL9cYbb2j58uVyc3NTWFiYEhISnBrrxo0bOnjwoAYOHKiVK1dqxowZOnr0qAYOHOj095Rm48aN+vnnn1WpUqVcx8hprBMnTqhnz57y9vbW4sWL9eWXX+qFF15Q8eLFnR5r4sSJ2rFjhyZPnqx169apb9++evPNN7V58+ZcxcnJz+xbb72lrVu36r333tPixYt19uxZDR48ONfvKbtYZ8+e1dmzZxUeHq41a9ZowoQJ2rFjh15//XWnx7rZwoULZbPZch0jN7H27dunfv36qUWLFvr888+1YsUK9erVS0WK5K4E5SRWeHi4jh49qlmzZmn16tVq166dXnrpJR08eDBXsapUqaIRI0Zo5cqV+uKLL/TAAw9o0KBB+v333yU577y4VRxnnhNmoLbmDbU1d7U1v+pqTt5TGmpr5qitt09tLYx1NbtYltVW4w7w5JNPGmPHjrW/TklJMVq0aGHMmTPH1LgXLlwwfH19je+++860GHFxccZDDz1kfPvtt0ZoaKgxfvx4p8eYPHmy0aNHD6cfNzP9+/c3IiIi0i0bPHiwMXz4cKfG8fX1NTZu3Gh/nZqaajRv3tyYN2+efdmVK1eMOnXqGGvWrHFqrMz8/PPPhq+vrxETE+P0OH/++acRHBxs/Pbbb0arVq2M+fPnOxzjVrFeeuklY8SIEXk+dk5ihYSEGDNmzEi3rHPnzsY777yTp1j//Jm9cuWKUbt2bWP9+vX2bQ4fPmz4+voa+/btc2qszKxbt86oXbu2kZSUZEqsgwcPGsHBwcbZs2dzdJ46Gqtr167Gu+++m+dj5yRW/fr1jVWrVqXbrnHjxsby5cvzHK9Ro0bG8uXLTT0vbo6TGWedE85Abc0baqvjtTW/6uqtYlFbc47aevvU1sJaV2+OlZn8qK2F/kp3YmKiDhw4oGbNmtmXFSlSRM2aNdO+fftMjZ12m4IZ31qnGTdunB588MF078/ZtmzZojp16ujFF19U06ZN1alTJy1fvtyUWIGBgdq9e7eOHj0qSTp06JB++OEHtWzZ0pR4aU6dOqVz586l+xxLly6tevXqmX6eSFJcXJxsNpvuuusupx43NTVVr7zyisLCwnTfffc59dj/jPPNN9/onnvuUVhYmJo2baquXbs6dEtVTgQGBmrLli06c+aMDMOwnzMtWrTI03H/+TP7yy+/KCkpKd154ePjI09PT/30009OjZWZuLg4ubu7y8XFxemx4uPjNXz4cI0ePVoeHh55Ov6tYl24cEE///yzKlSooKeeekrNmjVTaGio9u7d6/RY0t/nxvr163Xp0iWlpqZq7dq1SkhIUOPGjR2Ok5KSorVr1+r69esKDAw07bz4Z5zMOOucyCtqa95RW809T8yqqxK1NbeorXmXX7W1sNXVzGJlJj9qq7VVOx/89ddfSklJUYUKFdItr1ChgqljwlJTU/XWW2+pQYMG8vX1NSXG2rVrdfDgQa1YscKU46c5efKklixZomeeeUYDBgxQVFSUxo8fr2LFiqlz585OjdW/f3/FxcXp0UcfVdGiRZWSkqKXX35Zjz32mFPj/NO5c+ckKdPzxNHxJDmVkJCgKVOmKCQkRO7u7k499ty5c+Xi4qI+ffo49bj/dOHCBV2/fl1z587VSy+9pBEjRmjHjh0aPHiwFi1alKdfzJn597//rX//+99q2bKlXFxcZLPZNH78eDVq1MjhY2b2M3v+/HkVK1Yswx9tFSpUsJ8zzor1TxcvXtQHH3yg7t27OxznVrEmTJigwMBAtW3bNk/Hzy7WyZMnJUkzZszQyJEjVatWLUVGRurpp5/WmjVrdM899zgtliS99957evnll9WkSRO5uLioRIkSmjFjhmrUqJHrGL/++queeuopJSQkqGTJkpo5c6Zq1qyp6Ohop54XWcX5J2edE85Abc07aqt5tdXMuipRW3OD2pp3+VVbC1NdvVWsf8qv2lrom26rjB07Vr///rs+++wzU45/+vRp/ec//9HHH3/s0Lie3DAMQ3Xq1NGwYcMkSffff79+//13LV261Ol/GKxfv16rV6/W1KlT7T+EEyZMUKVKlZweqyBISkrS0KFDZRiGxo4d69Rj//LLL1q0aJFWrlyZp3FFOZGamipJatOmjZ5++mlJUq1atfTjjz9q6dKlTv/DYPHixfrpp580a9YseXp6au/evRo7dqwqVark8JUps39mcxMrLi5Ozz//vHx8fBwez3SrWJs3b9bu3bu1atWqPB07J7HSzo3u3bvriSeekPT375Bdu3bpiy++0PDhw50WS5Lef/99XblyRQsWLFC5cuW0adMmvfTSS/r0009zPTb33nvvVWRkpK5evaqvvvpK4eHh+uSTTxzK15E4N/9x4Mxz4nZGbXXMnVRbzayrErU1t6iteZdftbUw1dVbxbKqthb6prtcuXIqWrRoholdLly4oIoVK5oSc9y4cfrmm2/0ySefqEqVKqbEOHDggC5cuKAuXbrYl6WkpOj777/Xp59+qqioKBUtWtQpsTw8POTj45Numbe3t0MzF2Zn0qRJ6t+/v0JCQiRJfn5+io2N1Zw5c0z9wyDtFqALFy6kmxDlwoUL8vf3NyVmUlKSXnrpJcXGxmrhwoVO/zZ+7969unDhglq1amVflpKSorfffluLFi3Sli1bnBarXLlycnFxyXCe+Pj46IcffnBaHOnvCXPeffddzZgxQ//6178kSf7+/oqOjtZHH33k0B8GWf3MVqxYUUlJSbpy5Uq6b18vXLjg8G1j2f1+iIuLU79+/VSqVCnNnDlTxYoVcyjOrWLt3r1bJ06cyHD1YsiQIQoKCnJocqWsYqV9TpmdG47MvnurWCdOnNAnn3yiNWvW2G/59Pf31969e/Xpp59q3LhxuYrj6upq/ya/Tp06ioqK0qJFi/Too4869bzIKk5avs48J5yF2pp31Fbn11az66pEbc0NauvfbofaWtjq6q1iWVVbC33T7erqqtq1a2vXrl32Wz1SU1O1a9cuhYaGOjWWYRh68803tXHjRi1evFjVqlVz6vFv9sADD2j16tXplkVERMjb21vPPfec0/4okKQGDRrYx4GlOXbsmLy8vJwWI82NGzcyfHNctGhRpzzW5FaqVq0qDw8P7dq1S7Vq1ZL09w/jzz//rB49ejg9XtofBsePH9eiRYtUrlw5p8d4/PHHMxTJsLAwPf744+n+oHQGV1dX1a1bN1/Ok+TkZCUlJTnlPMnuZ7ZOnToqVqyYdu3apYcffliSdOTIEcXGxqp+/fpOjSX9fc6FhYXJ1dVVs2bNcvhKW3ax+vfvr65du6Zb1rFjR0VERKT7Q9IZsapWrapKlSplem7kdjxpdrHi4+MlKcPMrc76HZKamqrExESnnhe3iiM575xwNmpr3lFbnVtb86OuStTWnKC2/p+CXlvvlLp6cyzJmtpa6JtuSXrmmWcUHh6uOnXqKCAgQAsXLlR8fLzTfzmOHTtWa9as0QcffKBSpUrZxyCULl1aJUqUcGosd3f3DONVSpYsqbJlyzp9nFvfvn3Vo0cPzZ49W48++qj279+v5cuX5/rbrZxo1aqVZs+eLU9PT/stcPPnz7ffOpMX165d04kTJ+yvT506pejoaJUpU0aenp7q06ePZs2apRo1aqhq1ap6//33ValSJYfG5dwqloeHh1588UUdPHhQc+bMUUpKiv1cKVOmjFxdXZ32nv75R0exYsVUsWJFeXt7O/U9eXp6KiwsTC+//LIaNWqkJk2aaMeOHdq6dasWLVrk9FiNGzfW5MmTVaJECXl6eur7779XZGSkXn311VzFye5ntnTp0nriiSc0ceJElSlTRu7u7ho/frwCAwNzXQSyixUXF6dnn31W8fHxmjx5suLi4hQXFydJKl++fK7+2M8uloeHR6bfHHt6eua6ockuls1mU1hYmKZPny5/f3/VqlVLq1at0pEjRzRt2jSnxvL29laNGjU0evRohYeHq2zZstq0aZO+/fZbzZkzJ1expk6dqpYtW+ruu+/WtWvXtGbNGn333Xf66KOPnHpe3CqOM88JM1Bb84bamrvaml91NSfvidp6a9TW9ApybS2MdTW7WFbVVpth9tecBcQnn3yijz76SOfOnVOtWrU0atQo1atXz6kxshrXMGHCBKf/EZKZ3r17y9/f35TnzG3dulXvvPOOjh07pqpVq+qZZ55Rt27dnB4nLi5O77//vjZt2mS/HS0kJESDBg3KddH8pz179mQ66Unnzp01ceJEGYahadOmafny5bpy5YoaNmyoMWPG6N5773VqrMGDB6tNmzaZ7rdo0SI1adLEKXEmTpyYYXnr1q3Vp08f+9iw3MhJrBUrVujDDz/Un3/+qXvvvVdDhgxx6EuL7GKdO3dO77zzjv73v//p8uXL8vT0VPfu3fX000/naoxdTn5mExISNHHiRK1du1aJiYlq0aKFxowZk+vbnbKLldV7lv4eJ1a1alWnxcpqn5kzZ+b63yunsT788EN9+umnunz5svz9/TVixAgFBQU5PdaxY8c0depU/fDDD7p+/bqqV6+uZ599Vp06dcpVrNdee027d+/W2bNnVbp0afn5+em5555T8+bNJTnvvLhVHGeeE2ahtuYNtdU5cZxZV7OLRW3NHrU1/T4FubYWxrqaXSyrausd03QDAAAAAJDfCv1zugEAAAAAsApNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMImL1QkAsM65c+c0Z84cbdu2TX/++adKly6t6tWr67HHHlPnzp3l5uZmdYoAANxWqK0A/ommG7hDnTx5Uj169FDp0qX18ssvy8/PT66urvr111+1fPlyVa5cWW3atMn1cRMTE+Xq6mpCxgAAFGzUVgCZsRmGYVidBID8FxYWpsOHD2v9+vUqWbJkhvWGYchms+nKlSt6++23tXnzZiUmJqpOnTp67bXX5O/vL0maPn26Nm3apNDQUM2aNUuxsbE6dOiQ/Pz8NHbsWG3dulW7d++Wp6en3nrrLZUvX16jRo1SVFSU/P39NWnSJFWvXl2SdOLECU2YMEE///yz4uPj5e3treHDh6tZs2b2vFq3bq1u3brp+PHj2rBhg8qUKaOBAweqe/fu+fPBAQCQBWorgMwwphu4A/3111/69ttv1atXr0z/KJAkm80mSRo6dKguXLiguXPnauXKlapdu7b69u2rS5cu2bc9ceKEvvrqK82YMUORkZH25R988IEef/xxRUZG2ov86NGj1b9/f33xxRcyDEPjxo2zb3/9+nU9+OCDWrBggVatWqXg4GANGDBAsbGx6XKbP3++6tSpo8jISPXs2VNvvPGGjhw54rwPCACAXKK2AsgKTTdwBzpx4oQMw9C9996bbnmTJk0UGBiowMBATZ48WXv37tX+/fs1bdo01a1bV/fcc4/Cw8N111136auvvrLvl5SUpEmTJun++++3f0svSV26dFH79u1177336rnnnlNMTIw6duyo4OBg+fj4qE+fPvruu+/s2/v7++upp56Sr6+v7rnnHr300kuqXr26tmzZki7Pli1bqlevXqpRo4aee+45lStXTnv27DHp0wIAIHvUVgBZYUw3ALsVK1YoNTVVI0aMUGJion799Vddv35dTZo0SbfdjRs3dOLECftrT09PlS9fPsPx/Pz87P9foUIFSZKvr2+6ZQkJCYqLi5O7u7uuXbumGTNm6JtvvtG5c+eUkpKiGzduZPg2/ubj2mw2VaxYURcuXMjbmwcAwATUVgA03cAdqHr16rLZbDp69Gi65dWqVZMklShRQpJ07do1eXh4aPHixRmOUbp0afv/ZzUTa7Fixez/n3ZLXWbLUlNTJUlvv/22du7cqfDwcFWvXl0lSpTQiy++qKSkpHTHdXFJ/6vLZrOJ6SkAAFaitgLICk03cAcqV66cmjdvrk8++UShoaFZjj2rXbu2zp8/r6JFi6pq1aqm57Vv3z517txZ7dq1k/T3HyYxMTGmxwUAIK+orQCywphu4A41ZswYpaSk6IknntC6dev0xx9/6MiRI/rvf/+rI0eOqGjRomrWrJnq16+vQYMG6X//+59OnTqlH3/8Ue+++66ioqKcnlONGjW0ceNGRUdH69ChQxo+fLj9m3oAAAo6aiuAzHClG7hDVa9eXatWrdKcOXM0depUnTlzRsWKFVPNmjX17LPPqmfPnrLZbPrwww/13nvvKSIiQn/99ZcqVqyooKAgVaxY0ek5vfrqq3rttdf01FNPqVy5cnruued07do1p8cBAMAM1FYAmeE53QAAAAAAmITbywEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJDTdAAAAAACYhKYbAAAAAACT0HQDAAAAAGASmm4AAAAAAExC0w0AAAAAgElougEAAAAAMAlNNwAAAAAAJqHpBgAAAADAJP8P9KgRnTL/AgcAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "sns.set_style(\"whitegrid\")\n",
    "\n",
    "\n",
    "def transform(x):\n",
    "    x = x - np.mean(x)\n",
    "    return x / np.var(x)\n",
    "\n",
    "\n",
    "def calculate_alignment_matrix(first: np.ndarray, second: np.ndarray, threshold: float = 1e-3) -> np.ndarray:\n",
    "    similarity = first @ second.T  # 1\n",
    "    similarity_en_to_de = np.apply_along_axis(transform, -1, similarity)  # 2\n",
    "    similarity_de_to_en = np.apply_along_axis(transform, -2, similarity)  # 2\n",
    "\n",
    "    both_one = (similarity_en_to_de > threshold) * (similarity_de_to_en > threshold)  # 3 and 4\n",
    "    return both_one\n",
    "\n",
    "\n",
    "threshold = 0.028\n",
    "\n",
    "alignment_matrix = calculate_alignment_matrix(embeddings_en, embeddings_de, threshold)\n",
    "alignment_matrix_pt = calculate_alignment_matrix(\n",
    "    embeddings_en_pt.detach().numpy(),\n",
    "    embeddings_de_pt.detach().numpy(),\n",
    "    threshold,\n",
    ")\n",
    "\n",
    "graph, axis = plt.subplots(1, 2, figsize=(10, 5), sharey=True)\n",
    "\n",
    "for matrix, ax, title in zip((alignment_matrix, alignment_matrix_pt), axis, (\"OpenVINO\", \"PyTorch\")):\n",
    "    plot = sns.heatmap(matrix, cbar=False, square=True, ax=ax)\n",
    "    plot.set_title(f\"Sentence Alignment Matrix {title}\")\n",
    "    plot.set_xlabel(\"German\")\n",
    "    if title == \"OpenVINO\":\n",
    "        plot.set_ylabel(\"English\")\n",
    "\n",
    "graph.tight_layout()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "8eab75a4-a847-4ddc-8766-1ec7dea9720c",
   "metadata": {},
   "source": [
    "After visualizing and comparing the alignment matrices, let's transform them into a dictionary to make it more convenient to work with alignment in Python. Dictionary keys will be English sentence numbers and values will be lists of German sentence numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "eefd16c7-f732-45fa-89f6-2c4f8eccb01a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{0: [0],\n",
       " 1: [2],\n",
       " 2: [3],\n",
       " 3: [4],\n",
       " 4: [5],\n",
       " 5: [6],\n",
       " 6: [7],\n",
       " 7: [8],\n",
       " 8: [9, 10],\n",
       " 9: [11],\n",
       " 10: [13, 14],\n",
       " 11: [15],\n",
       " 12: [16],\n",
       " 13: [17],\n",
       " 14: [],\n",
       " 15: [18],\n",
       " 16: [19],\n",
       " 17: [20],\n",
       " 18: [21],\n",
       " 19: [23],\n",
       " 20: [24],\n",
       " 21: [25],\n",
       " 22: [26],\n",
       " 23: [],\n",
       " 24: [27],\n",
       " 25: [],\n",
       " 26: [28],\n",
       " 27: [29],\n",
       " 28: [30],\n",
       " 29: [31],\n",
       " 30: [32],\n",
       " 31: [33]}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def make_alignment(alignment_matrix: np.ndarray) -> Dict[int, List[int]]:\n",
    "    aligned = {idx: [] for idx, sent in enumerate(sentences_en)}\n",
    "    for en_idx, de_idx in zip(*np.nonzero(alignment_matrix)):\n",
    "        aligned[en_idx].append(de_idx)\n",
    "    return aligned\n",
    "\n",
    "\n",
    "aligned = make_alignment(alignment_matrix)\n",
    "aligned"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a36bbbfb-6bd4-45bc-9d32-32f511396f1f",
   "metadata": {},
   "source": [
    "## Postprocess Sentence Alignment\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "There are several gaps in the resulting alignment, such as English sentence #14 not mapping to any German sentence. Here are some possible reasons for this:\n",
    "\n",
    "1. There are no equivalent sentences in the other book, and in such cases, the model is working correctly.\n",
    "1. The sentence has an equivalent sentence in another language, but the model failed to identify it. The `threshold` might be too high, or the model is not sensitive enough. To address this, lower the `threshold` value or try a different model.\n",
    "1. The sentence has an equivalent text part in another language, meaning that either the sentence splitters are too fine or too coarse. Try tuning the text cleaning and splitting steps to fix this issue.\n",
    "1. Combination of 2 and 3, where both the model's sensitivity and text preparation steps need adjustments.\n",
    "\n",
    "Another solution to address this issue is by applying heuristics. As you can see, English sentence 13 corresponds to German 17, and 15 to 18. Most likely, English sentence 14 is part of either German sentence 17 or 18. By comparing the similarity using the model, you can choose the most suitable alignment.\n",
    "\n",
    "## Visualize Sentence Alignment\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "To evaluate the final alignment and choose the best way to improve the results of the pipeline, we will create an interactive table with HTML and JS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "e682adac-48ce-487c-8842-7a31e9515abe",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, HTML\n",
    "from itertools import zip_longest\n",
    "from io import StringIO\n",
    "\n",
    "\n",
    "def create_interactive_table(list1: List[str], list2: List[str], mapping: Dict[int, List[int]]) -> str:\n",
    "    def inverse_mapping(mapping):\n",
    "        inverse_map = {idx: [] for idx in range(len(list2))}\n",
    "\n",
    "        for key, values in mapping.items():\n",
    "            for value in values:\n",
    "                inverse_map[value].append(key)\n",
    "\n",
    "        return inverse_map\n",
    "\n",
    "    inversed_mapping = inverse_mapping(mapping)\n",
    "\n",
    "    table_html = StringIO()\n",
    "    table_html.write('<table id=\"mappings-table\"><tr><th>Sentences EN</th><th>Sentences DE</th></tr>')\n",
    "    for i, (first, second) in enumerate(zip_longest(list1, list2)):\n",
    "        table_html.write(\"<tr>\")\n",
    "        if i < len(list1):\n",
    "            table_html.write(f'<td id=\"list1-{i}\">{first}</td>')\n",
    "        else:\n",
    "            table_html.write(\"<td></td>\")\n",
    "        if i < len(list2):\n",
    "            table_html.write(f'<td id=\"list2-{i}\">{second}</td>')\n",
    "        else:\n",
    "            table_html.write(\"<td></td>\")\n",
    "        table_html.write(\"</tr>\")\n",
    "\n",
    "    table_html.write(\"</table>\")\n",
    "\n",
    "    hover_script = (\n",
    "        \"\"\"\n",

    "    <script type=\"module\">\n",

    "      const highlightColor = '#0054AE';\n",

    "      const textColor = 'white'\n",

    "      const mappings = {\n",

    "        'list1': \"\"\"\n",
    "        + str(mapping)\n",
    "        + \"\"\",\n",

    "        'list2': \"\"\"\n",
    "        + str(inversed_mapping)\n",
    "        + \"\"\"\n",

    "      };\n",

    "\n",

    "      const table = document.getElementById('mappings-table');\n",

    "      let highlightedIds = [];\n",

    "\n",

    "      table.addEventListener('mouseover', ({ target }) => {\n",

    "        if (target.tagName !== 'TD' || !target.id) {\n",

    "          return;\n",

    "        }\n",

    "\n",

    "        const [listName, listId] = target.id.split('-');\n",

    "        const mappedIds = mappings[listName]?.[listId]?.map((id) => `${listName === 'list1' ? 'list2' : 'list1'}-${id}`) || [];\n",

    "        const idsToHighlight = [target.id, ...mappedIds];\n",

    "\n",

    "        setBackgroud(idsToHighlight, highlightColor, textColor);\n",

    "        highlightedIds = idsToHighlight;\n",

    "      });\n",

    "\n",

    "      table.addEventListener('mouseout', () => setBackgroud(highlightedIds, ''));\n",

    "\n",

    "      function setBackgroud(ids, color, text_color=\"unset\") {\n",

    "        ids.forEach((id) => {\n",

    "            document.getElementById(id).style.backgroundColor = color;\n",

    "            document.getElementById(id).style.color = text_color\n",

    "        });\n",

    "      }\n",

    "    </script>\n",

    "    \"\"\"\n",
    "    )\n",
    "    table_html.write(hover_script)\n",
    "    return table_html.getvalue()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "7bf34413-ae12-49d2-bf36-62d445d01a16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table id=\"mappings-table\"><tr><th>Sentences EN</th><th>Sentences DE</th></tr><tr><td id=\"list1-0\">Happy families are all alike; every unhappy family is unhappy in its own way.</td><td id=\"list2-0\">Alle glücklichen Familien sind einander ähnlich; jede unglückliche Familie ist auf hr Weise unglücklich.</td></tr><tr><td id=\"list1-1\">Everything was in confusion in the Oblonskys’ house.</td><td id=\"list2-1\">--</td></tr><tr><td id=\"list1-2\">The wife had discovered that the husband was carrying on an intrigue with a French girl, who had been a governess in their family, and she had announced to her husband that she could not go on living in the same house with him.</td><td id=\"list2-2\">Im Hause der Oblonskiy herrschte allgemeine Verwirrung.</td></tr><tr><td id=\"list1-3\">This position of affairs had now lasted three days, and not only the husband and wife themselves, but all the members of their family and household, were painfully conscious of it.</td><td id=\"list2-3\">Die Dame des Hauses hatte in Erfahrung gebracht, daß ihr Gatte mit der im Hause gewesenen französischen Gouvernante ein Verhältnis unterhalten, und ihm erklärt, sie könne fürderhin nicht mehr mit ihm unter einem Dache bleiben.</td></tr><tr><td id=\"list1-4\">Every person in the house felt that there was no sense in their living together, and that the stray people brought together by chance in any inn had more in common with one another than they, the members of the family and household of the Oblonskys.</td><td id=\"list2-4\">Diese Situation währte bereits seit drei Tagen und sie wurde nicht allein von den beiden Ehegatten selbst, nein auch von allen Familienmitgliedern und dem Personal aufs Peinlichste empfunden.</td></tr><tr><td id=\"list1-5\">The wife did not leave her own room, the husband had not been at home for three days.</td><td id=\"list2-5\">Sie alle fühlten, daß in ihrem Zusammenleben kein höherer Gedanke mehr liege, daß die Leute, welche auf jeder Poststation sich zufällig träfen, noch enger zu einander gehörten, als sie, die Glieder der Familie selbst, und das im Hause geborene und aufgewachsene Gesinde der Oblonskiy.</td></tr><tr><td id=\"list1-6\">The children ran wild all over the house; the English governess quarreled with the housekeeper, and wrote to a friend asking her to look out for a new situation for her; the man-cook had walked off the day before just at dinner time; the kitchen-maid, and the coachman had given warning.</td><td id=\"list2-6\">Die Herrin des Hauses verließ ihre Gemächer nicht, der Gebieter war schon seit drei Tagen abwesend.</td></tr><tr><td id=\"list1-7\">Three days after the quarrel, Prince Stepan Arkadyevitch Oblonsky—Stiva, as he was called in the fashionable world—woke up at his usual hour, that is, at eight o’clock in the morning, not in his wife’s bedroom, but on the leather-covered sofa in his study.</td><td id=\"list2-7\">Die Kinder liefen wie verwaist im ganzen Hause umher, die Engländerin schalt auf die Wirtschafterin und schrieb an eine Freundin, diese möchte ihr eine neue Stellung verschaffen, der Koch hatte bereits seit gestern um die Mittagszeit das Haus verlassen und die Köchin, sowie der Kutscher hatten ihre Rechnungen eingereicht.</td></tr><tr><td id=\"list1-8\">He turned over his stout, well-cared-for person on the springy sofa, as though he would sink into a long sleep again; he vigorously embraced the pillow on the other side and buried his face in it; but all at once he jumped up, sat up on the sofa, and opened his eyes.</td><td id=\"list2-8\">Am dritten Tage nach der Scene erwachte der Fürst Stefan Arkadjewitsch Oblonskiy -- Stiwa hieß er in der Welt -- um die gewöhnliche Stunde, das heißt um acht Uhr morgens, aber nicht im Schlafzimmer seiner Gattin, sondern in seinem Kabinett auf dem Saffiandiwan.</td></tr><tr><td id=\"list1-9\">\"Yes, yes, how was it now?\" he thought, going over his dream.</td><td id=\"list2-9\">Er wandte seinen vollen verweichlichten Leib auf den Sprungfedern des Diwans, als wünsche er noch weiter zu schlafen, während er von der andern Seite innig ein Kissen umfaßte und an die Wange drückte.</td></tr><tr><td id=\"list1-10\">\"Now, how was it? To be sure! Alabin was giving a dinner at Darmstadt; no, not Darmstadt, but something American. Yes, but then, Darmstadt was in America. Yes, Alabin was giving a dinner on glass tables, and the tables sang, l mio tesor—not l mio tesor though, but something better, and there were some sort of little decanters on the table, and they were women, too,\" he remembered.</td><td id=\"list2-10\">Plötzlich aber sprang er empor, setzte sich aufrecht und öffnete die Augen.</td></tr><tr><td id=\"list1-11\">Stepan Arkadyevitch’s eyes twinkled gaily, and he pondered with a smile.</td><td id=\"list2-11\">\"Ja, ja, wie war doch das?\" sann er, über seinem Traum grübelnd.</td></tr><tr><td id=\"list1-12\">\"Yes, it was nice, very nice. There was a great deal more that was delightful, only there’s no putting it into words, or even expressing it in one’s thoughts awake.\"</td><td id=\"list2-12\">\"Wie war doch das?</td></tr><tr><td id=\"list1-13\">And noticing a gleam of light peeping in beside one of the serge curtains, he cheerfully dropped his feet over the edge of the sofa, and felt about with them for his slippers, a present on his last birthday, worked for him by his wife on gold-colored morocco.</td><td id=\"list2-13\">Richtig; Alabin gab ein Diner in Darmstadt; nein, nicht in Darmstadt, es war so etwas Amerikanisches dabei.</td></tr><tr><td id=\"list1-14\">And, as he had done every day for the last nine years, he stretched out his hand, without getting up, towards the place where his dressing-gown always hung in his bedroom.</td><td id=\"list2-14\">Dieses Darmstadt war aber in Amerika, ja, und Alabin gab das Essen auf gläsernen Tischen, ja, und die Tische sangen: Il mio tesoro -- oder nicht so, es war etwas Besseres, und gewisse kleine Karaffen, wie Frauenzimmer aussehend,\" -- fiel ihm ein.</td></tr><tr><td id=\"list1-15\">And thereupon he suddenly remembered that he was not sleeping in his wife’s room, but in his study, and why: the smile vanished from his face, he knitted his brows.</td><td id=\"list2-15\">Die Augen Stefan Arkadjewitschs blitzten heiter, er sann und lächelte.</td></tr><tr><td id=\"list1-16\">\"Ah, ah, ah! Oo!...\" he muttered, recalling everything that had happened.</td><td id=\"list2-16\">\"Ja, es war hübsch, sehr hübsch. Es gab viel Ausgezeichnetes dabei, was man mit Worten nicht schildern könnte und in Gedanken nicht ausdrücken.\"</td></tr><tr><td id=\"list1-17\">And again every detail of his quarrel with his wife was present to his imagination, all the hopelessness of his position, and worst of all, his own fault.</td><td id=\"list2-17\">Er bemerkte einen Lichtstreif, der sich von der Seite durch die baumwollenen Stores gestohlen hatte und schnellte lustig mit den Füßen vom Sofa, um mit ihnen die von seiner Gattin ihm im vorigen Jahr zum Geburtstag verehrten gold- und saffiangestickten Pantoffeln zu suchen; während er, einer alten neunjährigen Gewohnheit folgend, ohne aufzustehen mit der Hand nach der Stelle fuhr, wo in dem Schlafzimmer sonst sein Morgenrock zu hängen pflegte.</td></tr><tr><td id=\"list1-18\">\"Yes, she won’t forgive me, and she can’t forgive me. And the most awful thing about it is that it’s all my fault—all my fault, though I’m not to blame. That’s the point of the whole situation,\" he reflected.</td><td id=\"list2-18\">Hierbei erst kam er zur Besinnung; er entsann sich jäh wie es kam, daß er nicht im Schlafgemach seiner Gattin, sondern in dem Kabinett schlief; das Lächeln verschwand von seinen Zügen und er runzelte die Stirn.</td></tr><tr><td id=\"list1-19\">\"Oh, oh, oh!\" he kept repeating in despair, as he remembered the acutely painful sensations caused him by this quarrel.</td><td id=\"list2-19\">\"O, o, o, ach,\" brach er jammernd aus, indem ihm alles wieder einfiel, was vorgefallen war.</td></tr><tr><td id=\"list1-20\">Most unpleasant of all was the first minute when, on coming, happy and good-humored, from the theater, with a huge pear in his hand for his wife, he had not found his wife in the drawing-room, to his surprise had not found her in the study either, and saw her at last in her bedroom with the unlucky letter that revealed everything in her hand.</td><td id=\"list2-20\">Vor seinem Innern erstanden von neuem alle die Einzelheiten des Auftritts mit seiner Frau, erstand die ganze Mißlichkeit seiner Lage und -- was ihm am peinlichsten war -- seine eigene Schuld.</td></tr><tr><td id=\"list1-21\">She, his Dolly, forever fussing and worrying over household details, and limited in her ideas, as he considered, was sitting perfectly still with the letter in her hand, looking at him with an expression of horror, despair, and indignation.</td><td id=\"list2-21\">\"Ja wohl, sie wird nicht verzeihen, sie kann nicht verzeihen, und am Schrecklichsten ist, daß die Schuld an allem nur ich selbst trage -- ich bin schuld -- aber nicht schuldig!</td></tr><tr><td id=\"list1-22\">\"What’s this? this?\" she asked, pointing to the letter.</td><td id=\"list2-22\">Und hierin liegt das ganze Drama,\" dachte er, \"o weh, o weh!\"</td></tr><tr><td id=\"list1-23\">And at this recollection, Stepan Arkadyevitch, as is so often the case, was not so much annoyed at the fact itself as at the way in which he had met his wife’s words.</td><td id=\"list2-23\">Er sprach voller Verzweiflung, indem er sich alle die tiefen Eindrücke vergegenwärtigte, die er in jener Scene erhalten.</td></tr><tr><td id=\"list1-24\">There happened to him at that instant what does happen to people when they are unexpectedly caught in something very disgraceful.</td><td id=\"list2-24\">Am unerquicklichsten war ihm jene erste Minute gewesen, da er, heiter und zufrieden aus dem Theater heimkehrend, eine ungeheure Birne für seine Frau in der Hand, diese weder im Salon noch im Kabinett fand, und sie endlich im Schlafzimmer antraf, jenen unglückseligen Brief, der alles entdeckte, in den Händen.</td></tr><tr><td id=\"list1-25\">He did not succeed in adapting his face to the position in which he was placed towards his wife by the discovery of his fault.</td><td id=\"list2-25\">Sie, die er für die ewig sorgende, ewig sich mühende, allgegenwärtige Dolly gehalten, sie saß jetzt regungslos, den Brief in der Hand, mit dem Ausdruck des Entsetzens, der Verzweiflung und der Wut ihm entgegenblickend.</td></tr><tr><td id=\"list1-26\">Instead of being hurt, denying, defending himself, begging forgiveness, instead of remaining indifferent even—anything would have been better than what he did do—his face utterly involuntarily (reflex spinal action, reflected Stepan Arkadyevitch, who was fond of physiology)—utterly involuntarily assumed its habitual, good-humored, and therefore idiotic smile.</td><td id=\"list2-26\">\"Was ist das?\" frug sie ihn, auf das Schreiben weisend, und in der Erinnerung hieran quälte ihn, wie das oft zu geschehen pflegt, nicht sowohl der Vorfall selbst, als die Art, wie er ihr auf diese Worte geantwortet hatte.</td></tr><tr><td id=\"list1-27\">This idiotic smile he could not forgive himself.</td><td id=\"list2-27\">Es ging ihm in diesem Augenblick, wie den meisten Menschen, wenn sie unerwartet eines zu schmählichen Vergehens überführt werden.</td></tr><tr><td id=\"list1-28\">Catching sight of that smile, Dolly shuddered as though at physical pain, broke out with her characteristic heat into a flood of cruel words, and rushed out of the room.</td><td id=\"list2-28\">Er verstand nicht, sein Gesicht der Situation anzupassen, in welche er nach der Entdeckung seiner Schuld geraten war, und anstatt den Gekränkten zu spielen, sich zu verteidigen, sich zu rechtfertigen und um Verzeihung zu bitten oder wenigstens gleichmütig zu bleiben -- alles dies wäre noch besser gewesen als das, was er wirklich that -- verzogen sich seine Mienen (\"Gehirnreflexe\" dachte Stefan Arkadjewitsch, als Liebhaber von Physiologie) unwillkürlich und plötzlich zu seinem gewohnten, gutmütigen und daher ziemlich einfältigen Lächeln.</td></tr><tr><td id=\"list1-29\">Since then she had refused to see her husband.</td><td id=\"list2-29\">Dieses dumme Lächeln konnte er sich selbst nicht vergeben.</td></tr><tr><td id=\"list1-30\">\"It’s that idiotic smile that’s to blame for it all,\" thought Stepan Arkadyevitch.</td><td id=\"list2-30\">Als Dolly es gewahrt hatte, erbebte sie, wie von einem physischen Schmerz, und erging sich dann mit der ihr eigenen Leidenschaftlichkeit in einem Strom bitterer Worte, worauf sie das Gemach verließ.</td></tr><tr><td id=\"list1-31\">\"But what’s to be done? What’s to be done?\" he said to himself in despair, and found no answer.</td><td id=\"list2-31\">Von dieser Zeit an wollte sie ihren Gatten nicht mehr sehen.</td></tr><tr><td></td><td id=\"list2-32\">\"An allem ist das dumme Lächeln schuld,\" dachte Stefan Arkadjewitsch.</td></tr><tr><td></td><td id=\"list2-33\">\"Aber was soll ich thun, was soll ich thun?\" frug er voll Verzweiflung sich selbst, ohne eine Antwort zu finden.</td></tr></table>\n",
       "    <script type=\"module\">\n",
       "      const highlightColor = '#0054AE';\n",
       "      const textColor = 'white'\n",
       "      const mappings = {\n",
       "        'list1': {0: [0], 1: [2], 2: [3], 3: [4], 4: [5], 5: [6], 6: [7], 7: [8], 8: [9, 10], 9: [11], 10: [13, 14], 11: [15], 12: [16], 13: [17], 14: [], 15: [18], 16: [19], 17: [20], 18: [21], 19: [23], 20: [24], 21: [25], 22: [26], 23: [], 24: [27], 25: [], 26: [28], 27: [29], 28: [30], 29: [31], 30: [32], 31: [33]},\n",
       "        'list2': {0: [0], 1: [], 2: [1], 3: [2], 4: [3], 5: [4], 6: [5], 7: [6], 8: [7], 9: [8], 10: [8], 11: [9], 12: [], 13: [10], 14: [10], 15: [11], 16: [12], 17: [13], 18: [15], 19: [16], 20: [17], 21: [18], 22: [], 23: [19], 24: [20], 25: [21], 26: [22], 27: [24], 28: [26], 29: [27], 30: [28], 31: [29], 32: [30], 33: [31]}\n",
       "      };\n",
       "\n",
       "      const table = document.getElementById('mappings-table');\n",
       "      let highlightedIds = [];\n",
       "\n",
       "      table.addEventListener('mouseover', ({ target }) => {\n",
       "        if (target.tagName !== 'TD' || !target.id) {\n",
       "          return;\n",
       "        }\n",
       "\n",
       "        const [listName, listId] = target.id.split('-');\n",
       "        const mappedIds = mappings[listName]?.[listId]?.map((id) => `${listName === 'list1' ? 'list2' : 'list1'}-${id}`) || [];\n",
       "        const idsToHighlight = [target.id, ...mappedIds];\n",
       "\n",
       "        setBackgroud(idsToHighlight, highlightColor, textColor);\n",
       "        highlightedIds = idsToHighlight;\n",
       "      });\n",
       "\n",
       "      table.addEventListener('mouseout', () => setBackgroud(highlightedIds, ''));\n",
       "\n",
       "      function setBackgroud(ids, color, text_color=\"unset\") {\n",
       "        ids.forEach((id) => {\n",
       "            document.getElementById(id).style.backgroundColor = color;\n",
       "            document.getElementById(id).style.color = text_color\n",
       "        });\n",
       "      }\n",
       "    </script>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "html_code = create_interactive_table(sentences_en, sentences_de, aligned)\n",
    "display(HTML(html_code))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f374e42d-51c5-4fd6-a922-c66fb31f879b",
   "metadata": {},
   "source": [
    "You can see that the pipeline does not fully clean up the German text, resulting in issues like the second sentence consisting of only `--`. On a positive note, the split sentences in the German translation line up correctly with the single English sentence. Overall, the pipeline already works well, but there is still room for improvement.\n",
    "\n",
    "Save the OpenVINO model to disk for future use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "bf4c127a-02bd-4488-8797-ea4ca413ee82",
   "metadata": {},
   "outputs": [],
   "source": [
    "from openvino.runtime import serialize\n",
    "\n",
    "\n",
    "ov_model_path = \"ov_model/model.xml\"\n",
    "serialize(ov_model, ov_model_path)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "fb56f746-ff32-4a9e-be5c-0a4fc59719c9",
   "metadata": {},
   "source": [
    "To read the model from disk, use the `read_model` method of the `Core` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "20e14ba6-70c1-43b1-8f27-7dae9ca32245",
   "metadata": {},
   "outputs": [],
   "source": [
    "ov_model = core.read_model(ov_model_path)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3c9c1a74-b4c6-4b65-8a92-346633d12540",
   "metadata": {},
   "source": [
    "## Speed up Embeddings Computation\n",
    "[back to top ⬆️](#Table-of-contents:)\n",
    "\n",
    "Let's see how we can speed up the most computationally complex part of the pipeline - getting embeddings. You might wonder why, when using OpenVINO, you need to compile the model after reading it. There are two main reasons for this:\n",
    "1. Compatibility with different devices. The model can be compiled to run on a [specific device](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes.html), like CPU, GPU or GNA. Each device may work with different data types, support different features, and gain performance by changing the neural network for a specific computing model. With OpenVINO, you do not need to store multiple copies of the network with optimized for different hardware. A universal OpenVINO model representation is enough.\n",
    "1. Optimization for different scenarios. For example, one scenario prioritizes minimizing the _time between starting and finishing model inference_ ([**latency-oriented optimization**](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/optimizing-latency.html)). In our case, it is more important _how many texts per second the model can process_ ([**throughput-oriented optimization**](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/optimizing-throughput.html)).\n",
    "\n",
    "To get a throughput-optimized model, pass a [performance hint](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/high-level-performance-hints.html#performance-hints-latency-and-throughput) as a configuration during compilation. Then OpenVINO selects the optimal parameters for execution on the available hardware."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b0599ae7-6d1c-437d-9b4e-04f073defdf2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Any\n",
    "\n",
    "\n",
    "compiled_throughput_hint = core.compile_model(\n",
    "    ov_model,\n",
    "    device_name=device.value,\n",
    "    config={\"PERFORMANCE_HINT\": \"THROUGHPUT\"},\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "27ca9dfc-9d0d-4d24-b7db-c6958e15c3a8",
   "metadata": {},
   "source": [
    "To further optimize hardware utilization, let's change the inference mode from synchronous (Sync) to asynchronous (Async). While the synchronous API may be easier to start with, it is [recommended](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/general-optimizations.html#prefer-openvino-async-api) to use the asynchronous (callbacks-based) API in production code. It is the most general and scalable way to implement flow control for any number of requests.\n",
    "\n",
    "To work in asynchronous mode, you need to define two things:\n",
    "\n",
    "1. Instantiate an `AsyncInferQueue`, which can then be populated with inference requests.\n",
    "1. Define a `callback` function that will be called after the output request has been executed and its results have been processed.\n",
    " \n",
    "In addition to the model input, any data required for post-processing can be passed to the queue. We can create a zero embedding matrix in advance and fill it in as the inference requests are executed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "61371a41-e59d-48ac-9529-86f4852e7817",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_embeddings_async(sentences: List[str], embedding_model: OVModel) -> np.ndarray:\n",
    "    def callback(infer_request: ov.InferRequest, user_data: List[Any]) -> None:\n",
    "        embeddings, idx, pbar = user_data\n",
    "        embedding = infer_request.get_output_tensor(0).data[0, 0]\n",
    "        embeddings[idx] = embedding\n",
    "        pbar.update()\n",
    "\n",
    "    infer_queue = ov.AsyncInferQueue(embedding_model)\n",
    "    infer_queue.set_callback(callback)\n",
    "\n",
    "    embedding_dim = embedding_model.output(0).get_partial_shape().get_dimension(2).get_length()\n",
    "    embeddings = np.zeros((len(sentences), embedding_dim))\n",
    "\n",
    "    with tqdm(total=len(sentences), disable=disable_tqdm) as pbar:\n",
    "        for idx, sent in enumerate(sentences):\n",
    "            tokenized = tokenizer(sent, return_tensors=\"np\").data\n",
    "\n",
    "            infer_queue.start_async(tokenized, [embeddings, idx, pbar])\n",
    "\n",
    "        infer_queue.wait_all()\n",
    "\n",
    "    return embeddings"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4b786808-8cb2-413b-85b0-63291b563743",
   "metadata": {},
   "source": [
    "Let's compare the models and plot the results.\n",
    "\n",
    "> **Note**: To get a more accurate benchmark, use the [Benchmark Python Tool](https://docs.openvino.ai/2024/learn-openvino/openvino-samples/benchmark-tool.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "7b37e15f-df1c-4233-b600-adcb8cf11def",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11a0d5efa2ec4aaabcae095c0ce43e8d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "112"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "number_of_chars = 15_000\n",
    "more_sentences_en = splitter_en.segment(clean_text(anna_karenina_en[:number_of_chars]))\n",
    "len(more_sentences_en)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "95cd83c2-814b-4e36-9267-5102049f8635",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/3 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Benchmark Runs:   0%|          | 0/11 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Benchmark Runs:   0%|          | 0/11 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Benchmark Runs:   0%|          | 0/11 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "from time import perf_counter\n",
    "\n",
    "\n",
    "benchmarks = [\n",
    "    (pt_model, get_embeddings, \"PyTorch\"),\n",
    "    (compiled_model, get_embeddings, \"OpenVINO\\nSync\"),\n",
    "    (\n",
    "        compiled_throughput_hint,\n",
    "        get_embeddings_async,\n",
    "        \"OpenVINO\\nThroughput Hint\\nAsync\",\n",
    "    ),\n",
    "]\n",
    "\n",
    "number_of_sentences = 100\n",
    "benchmark_data = more_sentences_en[: min(number_of_sentences, len(more_sentences_en))]\n",
    "\n",
    "benchmark_results = {name: [] for *_, name in benchmarks}\n",
    "\n",
    "benchmarks_iterator = tqdm(benchmarks, leave=False, disable=disable_tqdm)\n",
    "for model, func, name in benchmarks_iterator:\n",
    "    printable_name = name.replace(\"\\n\", \" \")\n",
    "    benchmarks_iterator.set_description(f\"Run benchmark for {printable_name} model\")\n",
    "    for run in tqdm(range(10 + 1), leave=False, desc=\"Benchmark Runs: \", disable=disable_tqdm):\n",
    "        with disable_tqdm_context():\n",
    "            start = perf_counter()\n",
    "            func(benchmark_data, model)\n",
    "            end = perf_counter()\n",
    "        benchmark_results[name].append(len(benchmark_data) / (end - start))\n",
    "\n",
    "benchmark_dataframe = pd.DataFrame(benchmark_results)[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "0df48072-8c81-42e8-9c0c-690b716edd28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHoCAYAAACvlC5HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/OQEPoAAAACXBIWXMAAA9hAAAPYQGoP6dpAAByxElEQVR4nO3dd1hT598G8Js9VFAEF25s4gBkFWW4BQfu1YpKq1SrdVeroK2Ce1v3FmeLm9a9a927Igq4BVQQQUGGMnLeP3g5P2NACASBeH+ui6vNc9Y3yTG585znnKMhCIIAIiIiIjWlWdwFEBERERUlhh0iIiJSaww7REREpNYYdoiIiEitMewQERGRWmPYISIiIrXGsENERERqjWGHiIiI1BrDDhEREak1hh0iKnIDBgxAp06dPsu2pFIpli1blud8y5Ytg1QqlWtr3bo1fHx8iqo0tRcVFQWpVIoNGzYUdyn5snfvXkilUty+fbu4S6Eipl3cBRApKzw8HCtWrMDt27fx6tUrlC9fHvXq1UPr1q0xYMCAIttuTEwMdu7cibZt26JBgwZFtp3P5fLly/Dy8sp1+qJFi+Dh4fEZK6JPyen9MjY2Ru3atdG/f3906dKlmCojKvkYdqhUuXHjBry8vFCtWjX07t0bZmZmePHiBW7duoUtW7YUadh5+fIlli9fDnNzc7UIO9kGDBgAKysrhXYbG5vPX0wxO3LkCDQ0NIq7jE/68P168+YNDh8+jF9++QVv375Fv379irk6opKJYYdKldWrV6NcuXLYvXs3jIyM5KbFxcUVU1Wlm4ODA9q3b1/cZZQIurq6xV1Cnj5+v/r27Yu2bdti//79DDv59P79e+jo6BR3GfQZccwOlSoRERGoV6+eQtABgIoVKyq0/fXXX+jRowesra3h6OiIsWPH4sWLF3LzZI8nefDgAQYMGIDGjRujWbNmWLdunTjP5cuX0atXLwCAr68vpFIppFIp9u7dK85z69YteHt7w97eHo0bN0b//v1x/fp1uW1ljxN5+vQpfHx84ODgAHt7e/j6+iI1NTXH+nv16oXGjRvj66+/Rr9+/XDu3Dm5ec6cOQNPT0/Y2NjA1tYWQ4YMwf379/PxauafVCrFtGnTcPjwYXTs2BHW1tb45ptvEB4eDgAIDAyEm5sbrKysMGDAAERFReW4npCQEHz77bewtrZG69at8eeffyrMk5aWhqVLl8LNzQ2WlpZo0aIF5s2bh7S0NIX5Zs2ahaZNm8LW1hZDhw5FdHR0jtu9du0aevbsCSsrK7Rt2xaBgYE5zvfxmJ3sMR3Xr1/H7Nmz0bRpU9jY2GD48OGIj4+XW1Ymk2HZsmVwdXVF48aNMWDAADx48EBhnenp6Vi+fDnc3d1hZWWFJk2aoG/fvjh//nyONeVFV1cXxsbG0NZW/O2qqv0/2/v377Fs2TK0a9cOVlZWcHV1xYgRIxAREaEw744dO9C2bVtYWlqiZ8+eCA4Olpvu4+MDW1tbPH/+HD/++CNsbW3RrFkzbN++HUDW4WovLy/Y2NigVatW2L9/v9zyb968wdy5c9G5c2fY2trCzs4OP/zwA8LCwuTmu3z5MqRSKQ4ePIjFixejWbNmaNy4MZKSknJ8PRMSEtCrVy80b94cjx49ynEeKn3Ys0Olirm5OW7evIl79+5BIpF8ct5Vq1ZhyZIl6NChA3r16oX4+Hhs27YN/fr1Q1BQkFxgSkhIwA8//AA3Nzd06NABR48exYIFCyCRSNCiRQtYWFhg1KhRWLp0Kb755hvY29sDAOzs7AAAFy9exODBg2FpaYkRI0ZAQ0MDe/fuxXfffYc//vgD1tbWcrWNGTMG1atXx88//4y7d+9i165dMDExwS+//CLOs3z5cixbtgy2trYYNWoUdHR0cOvWLVy6dAmurq4AgKCgIPj4+MDV1RXjx49Hamoq/vzzT3h6emLfvn2oXr16nq9pcnKywhc3AFSoUEHukM61a9dw6tQpeHp6AgDWrl2LoUOH4ocffsAff/wBT09PJCQkYP369Zg0aRK2bNkit76EhAQMGTIEHTp0gIeHBw4fPgw/Pz/o6OiIQVImk2HYsGG4fv06+vTpAwsLC9y7dw+bN2/GkydPsHLlSnF9kydPxt9//41OnTrBzs4Oly5dwpAhQxSeR3h4OLy9vWFiYoKRI0ciIyMDy5YtyzEc52bGjBkwMjLCiBEj8OzZM2zevBnTpk3D77//Ls6zcOFCrF+/Hq1atUKzZs0QFhYGb29vvH//Xm5dy5cvx5o1a9C7d29YW1sjKSkJISEhuHPnDlxcXPKs5cP3KyEhAQcOHMC9e/cwc+ZMuflUuf8DQGZmJn788UdcvHgRHh4e8PLyQnJyMs6fP4979+6hZs2a4voOHDiA5ORkfPPNN9DQ0MD69esxcuRInDhxQq5HJTMzE4MHD4aDgwPGjx+P/fv3Y9q0aTAwMMDixYvRuXNnuLu7IzAwEBMnToSNjQ1q1KgBAIiMjMSJEyfQvn17VK9eHa9evcKOHTvQv39/HDx4EJUrV5Z7PVauXAkdHR14e3sjLS0tx56d+Ph4DBo0CAkJCdi2bZvcc6JSTiAqRc6dOyc0aNBAaNCggfDNN98I8+bNE86ePSukpaXJzRcVFSU0aNBAWLVqlVx7eHi40LBhQ7n2/v37CxKJRNi3b5/Y9v79e8HFxUUYOXKk2BYcHCxIJBJhz549cuuUyWSCu7u7MGjQIEEmk4ntqampQuvWrYWBAweKbUuXLhUkEong6+srt47hw4cLjo6O4uMnT54I9evXF4YPHy5kZmYqbE8QBCEpKUlwcHAQfv31V7npsbGxgr29vUL7xy5duiRIJJJc/16+fCnOK5FIBEtLSyEyMlJsCwwMFCQSieDi4iK8fftWbF+4cKEgkUjk5s1+jTdu3Ci2vX//Xujatavg5OQkvn9BQUFC/fr1hatXr8rV+ueffwoSiUS4fv26IAiCEBoaKkgkEsHPz09uvp9//lmQSCTC0qVLxbaffvpJsLKyEp49eya2PXjwQGjQoIEgkUjklm/VqpUwceJE8fGePXsEiUQifP/993Lv7axZs4QGDRoIiYmJgiBkveYNGzYUfvrpJ7n1LVu2TJBIJHLr7NKlizBkyBBBWbm9X/Xr11fYz4ti/9+9e7cgkUiEgIAAhdqyX5vIyEhBIpEIjo6Owps3b8TpJ06cECQSiXDq1CmxbeLEiYJEIhFWr14ttiUkJAjW1taCVCoVDh48KLY/fPhQ4X19//69wr+NyMhIwdLSUli+fLnC69amTRshNTVVbv7s9zc4OFh4+fKl4OHhIbRp00aIiopSeI5UuvEwFpUqLi4uCAwMROvWrREWFob169fD29sbzZs3x8mTJ8X5jh8/DplMhg4dOiA+Pl78MzU1Ra1atXD58mW59RoaGqJr167iY11dXVhZWSEyMjLPmkJDQ/HkyRN07twZr1+/FreVkpICJycnXL16FTKZTG6Zb7/9Vu6xg4MD3rx5I3atnzhxAjKZDMOHD4empvw/0+zelgsXLiAxMREeHh5yz1FTUxONGzdWeI65GT58OAICAhT+jI2N5eZzcnKS6ylq3LgxAMDd3R1ly5YV27N7sT5+7bS1tfHNN9+Ij3V1dfHNN98gLi4Od+7cAZA1QNjCwgJ169aVe05NmzYFAPE5nTlzBgAUBqR/9913co8zMzNx7tw5tG3bFtWqVRPbLSwsxN6x/OjTp49cL5eDgwMyMzPx7NkzAFk9exkZGWKvV7b+/fsrrMvIyAj379/HkydP8r39D334fi1evBgeHh5YvHgxNm/eLM5TFPv/sWPHUKFChRyf08eDujt27Ci3/zg4OABQ3CcAoHfv3uL/GxkZoU6dOjAwMECHDh3E9rp168LIyEhueV1dXfHfRmZmJl6/fg1DQ0PUqVMHd+/eVdhOt27doK+vr9AOZJ1p2b9/f6Snp2P79u0wNzfPcT4qvXgYi0oda2trLF++HGlpaQgLC8OJEyewadMmjB49GkFBQahXrx6ePHkCQRDg7u6e4zo+Ht9QpUoVhQ9sY2NjcUzKp2R/aU2cODHXed6+fSv34f/hFy8A8ZBCQkICypYti4iICGhqasLCwiLP7X78BZ/twwDyKRKJBM7OznnOV7Vq1RzXX6VKFbn2cuXKAQASExPl2itVqgRDQ0O5ttq1awMAnj17BhsbGzx9+hQPHz6Ek5NTjjVkD0J/9uwZNDU1FQ4z1K1bV+5xfHw83r17h1q1aimsq06dOmJoyktu71f2c3z+/DkAKNRTvnx5hdA4atQo/PTTT2jXrh0kEglcXV3RtWtX1K9fP1+1fPx+dezYEUlJSVi4cCE6d+4MExOTItn/IyIiUKdOnRzHBn3s430l+zX4eJ/Q09ODiYmJXFu5cuVyrKdcuXJyy8tkMmzZsgV//PEHoqKikJmZKU4rX768Qk2fOqT7yy+/QFtbG4cOHYKZmdmnnxyVSgw7VGrp6urC2toa1tbWqF27Nnx9fXHkyBGMGDECMpkMGhoaWLduHbS0tBSW/fhLN6d58ksQBADAhAkTcj0l/ePtfdxb8/G6lNnuvHnzcvyALsxzyklu68utXZnnkk0mk0EikcDX1zfH6R8Hq89FFe9Xtq+//hrHjx/HyZMncf78eezevRubN2+Gv7+/XC+HMpo2bYrTp08jODgYLVu2/Kz7f07yu08UZp9avXo1lixZgp49e2L06NEwNjaGpqYmZs2aleP7kluvDpDVOxkUFIQtW7Zg3Lhxuc5HpRfDDqkFS0tLAFnXwgGyfmELgoDq1aujTp06KtlGbtdfyR4wWbZs2Xz1kORHzZo1IZPJ8PDhw1wDVPZ2K1asqLLtFqWXL18iJSVF7os2u3cq+7BBzZo1ERYWBicnp09e78bc3BwymQwRERFyvTkfnz1jYmICfX19PH36VGEdjx8/LszTkZPd8xMRESG+LwDw+vVrJCQkKMxfvnx59OzZEz179kRycjL69++PZcuWFTjsZPdqpKSkACia/b9mzZq4desW0tPTS8Rp20ePHkWTJk0wa9YsufbExERUqFBBqXX1798fNWvWxNKlS1GuXLkcB7pT6cYxO1SqXLp0KcdfbdmHI7K/+Nzd3aGlpYXly5crzC8IAl6/fq30tg0MDAAodsVbWlqiZs2a2LhxI5KTkxWWy+lMp7y0bdsWmpqaWLFihcJ4n+zn06xZM5QtWxZr1qxBenq6SrZblDIyMrBjxw7xcVpaGnbs2AETExM0atQIANChQwfxStUfe/funfhl3rx5cwDA1q1b5eb5cNwKkNVD4OrqihMnToiHmgDg4cOHCqfwF4aTkxO0tbUVTqXPPo36Qx/ve2XKlEHNmjUVTq1Xxj///AMA4u0vimL/d3d3x+vXr3N8TgXp4SosLS0the0ePnwYMTExBVrf8OHDMWjQICxcuBB//PGHKkqkEoQ9O1SqzJgxA6mpqXBzc0PdunWRnp6OGzdu4PDhwzA3N0ePHj0AZP0KHTNmDBYuXIhnz56hbdu2KFOmDKKionDixAn06dMH3t7eSm27Zs2aMDIyQmBgIMqUKQNDQ0NYW1ujRo0amDFjBgYPHoxOnTqhR48eqFy5MmJiYnD58mWULVsWq1evVmpbtWrVwtChQ7Fy5Up4enrC3d0durq6uH37NipVqoRx48ahbNmy8PPzw4QJE9CjRw907NgRJiYmeP78Oc6cOQM7OztMmTIlz21du3ZN4fRoIOuLM7/jSPKjUqVKWLduHZ49e4batWvj0KFDCA0NxfTp08Wegq5du+Lw4cOYOnUqLl++DDs7O2RmZuLRo0c4cuQI1q9fDysrKzRo0ACdOnXCH3/8gbdv38LW1haXLl3KsQdn5MiROHv2LPr164e+ffsiMzMT27ZtQ7169fI1Jis/TE1N4eXlhY0bN2Lo0KFo1qwZwsPD8e+//yqcwu/h4QFHR0c0atQI5cuXx+3bt3H06NEcB/7m5MP3KyEhAadOncKVK1fg4eEhjvEqiv2/W7duCAoKwuzZsxEcHAx7e3ukpqbi4sWL4oUNP6eWLVtixYoV8PX1ha2tLe7du4f9+/fL9awpa+LEiUhKSsK0adNQpkwZuUHbVLox7FCpMmHCBBw5cgRnzpzBjh07kJ6ejmrVqsHT0xPDhg2Tu3bIkCFDULt2bWzatAkrVqwAkDXmw8XFBa1bt1Z62zo6OpgzZw4WLVoEPz8/ZGRkYPbs2ahRowaaNGmCHTt2YOXKldi2bRtSUlJgZmYmXnyvIEaPHo3q1atj27ZtWLx4MQwMDCCVSuU+gDt37oxKlSph7dq12LBhA9LS0lC5cmU4ODiIwS8vH/eOZBsxYoRKw46xsTHmzJmDGTNmYOfOnTA1NcWUKVPQp08fcZ7s3qxNmzbhr7/+wvHjx2FgYIDq1atjwIABcodkZs2ahQoVKmD//v04efIkmjRpgrVr14rXhclWv359bNiwAbNnz8bSpUtRpUoVjBw5ErGxsSoLOwAwfvx46OvrY9euXbh48SJsbGywYcMGeHp6yl2ZecCAATh16hTOnz+PtLQ0VKtWDWPGjMl3+Pjw/dLR0UGNGjUwduxYheVVvf9raWlh3bp1WLVqFQ4cOIBjx46hfPnysLOzU7ih6ucwdOhQpKamYv/+/Th06BAaNmyINWvWYOHChYVar7+/P1JSUjBp0iSUKVPms4c4KhoaQnH0PxIRfQESExPx9ddfY8yYMRg2bFhxl0P0xeKYHSIiFXj37p1CW/YYIkdHx89dDhF9gIexiIhU4NChQ9i3bx+aN28OQ0ND3LhxAwcOHICrq6t4exEiKh4MO0REKiCVSqGlpYX169cjOTkZFStWhJeXF8aMGVPcpRF98Thmh4iIiNQax+wQERGRWmPYISIiIrXGsEOf3eXLlyGVSnO8K/fgwYPx66+/FnjdY8eOxejRowtTXoni5+eHgQMHftZtLliwQKnbFgwYMEDh7uNERCUJw84XaO/evZBKpbh9+7bSy6ampmLZsmU5BpXCun79Os6fP4/BgweLbdnBKPuvQYMGcHJywqhRo/Dw4UOFdQwePBjHjh1DWFhYvrebmZmJPXv2YMCAAXB0dISlpSVat24NX1/fAr1GqhIZGYndu3fjxx9/BJAVKj58LXL7W7ZsGQCgdevWkEql+P7773Nc/86dO8VlPnye3333HcLCwnDy5MkC1/7q1Sv4+vrCyckJ1tbW6N69Ow4fPpzv5YODg+Hn54cePXqgUaNGeV60bteuXejQoQOsrKzg7u6e64USDx48iO7du8PKygpNmzbFpEmTcrytxtu3bzFv3jy4u7vD2toarVq1wqRJk+RuOZEtJiYGo0ePhoODA+zs7DBs2DBERkYq1CeVSrFv3z6F5W/evIn69etj7ty5Ytun3uv27dt/8rXIlpSUhOXLl6NLly6wtbWFtbU1OnXqhPnz58vdUsHHx0du/XZ2dujSpQs2btwodwsLHx8f2Nra5ro9W1tb+Pj45Ku2nLx//z7H257kx7t37zBp0iR06tQJ9vb2sLW1RZcuXbB58+Z8r1Mmk2HdunVo3bo1rKys0LlzZxw4cCDHeR8+fAhvb2/Y2trC0dERv/zyS663Z4mLi8OCBQvQuXNn2NrawsrKCm5ubvD19cW1a9fk5s3rs3nAgAHo1KlTvp4PyePZWKSU1NRULF++HCNGjECTJk1Uuu4NGzbAyckJtWrVUpg2YMAAWFlZISMjA+Hh4QgMDMTly5dx4MABuTt+N2zYEJaWlti4cSPmzZuX5zbfvXuHESNG4OzZs/j666/x448/wtjYGM+ePcPhw4exb98+/PPPP8Vyt+0tW7bA3NwcTZs2BZB1xdhevXqJ02/fvo2tW7di6NChcjfD/DAY6Onp4fLly4iNjVW4M/r+/fuhp6encKsIMzMztGnTBhs3bkSbNm3yrHPDhg1yj5OSkuDp6YlXr17By8sLZmZmOHz4MMaMGYOMjAx07tw5z3WeOXMGu3fvhkQiQfXq1cUbhuYkMDAQU6dORbt27TBw4EBcu3ZNvK3Ihzd0/OOPP+Dv7w8nJyf4+PggJiYGW7ZsQUhICHbt2gU9PT0AWV96AwcOxMOHD9G3b1/UqVMHT58+xR9//IFz587h0KFDKFu2LAAgOTkZXl5eePv2LX788Ufo6Ohg06ZN6N+/P4KCgsQbUvbq1Qv79u3D3Llz0bJlS7E9PT0dU6ZMQdWqVTFy5Ei551WlShX8/PPPCs+3XLlyeb5+kZGR+P777/HixQu0b98e33zzDXR0dBAeHo7du3fjxIkTOHr0qDi/rq4uZsyYASAr6B09ehRz587F7du3sXjx4jy3V1DBwcHYunUrzp8/j7i4OGhoaKBy5cpwc3PDgAEDcvwsyMm7d+/w4MEDNG/eHObm5tDU1MTNmzfFW1vk56rKixcvxtq1a9GnTx9YWVnh5MmTGDduHDQ0NODh4SHOFx0djX79+qFcuXIYO3YsUlJSsHHjRty7dw+7du2Su1p2cHAwhgwZguTkZHh4eODbb7+Frq6ueNuOvXv3Ytu2bfj666+Vf/FIOQJ9cfbs2SNIJBIhODhY6WXj4uIEiUQiLF26tMDbv3TpkiCRSIRLly6Jba9evRIaNmwo7Ny5M8d5Dx8+LNf+xx9/CBKJRFi7dq3C+jds2CDY2NgISUlJedbi7+8vSCQSISAgQGFaRkaGsH79euHFixf5fGa5S05OVmr+tLQ0oUmTJsLixYtznefw4cMKr+OHWrVqJXz33XeCnZ2dsGnTJrlpL168EOrXry+MHDkyx33hyJEjglQqFSIiIpSqWxAEYd26dYJEIhEuXLggtmVmZgo9e/YUXFxchPfv3+e5jtjYWCE1NVUQhP+9RzlJTU0VHB0dhSFDhsi1jxs3TrCxsRHevHkjCIIgvH//XnBwcBD69esnyGQycb5Tp04JEolE2LJli9h2/fp1QSKRCNu2bZNb5+7duwWJRCIcO3ZMbFu7dq0gkUiEW7duiW0PHjwQGjRoICxcuFBu+fv37wuNGjUSfHx8xLY1a9YIEolEOHnypNy8/fv3Fzw8PHJ/gT4hPT1d6NKli9C4cWPh6tWrCtPfvn0rLFq0SHw8ceJEwcbGRm6ezMxMoUePHoJEIhGio6Nzne9DNjY2wsSJE/Ndo7+/vyCVSgVPT09hw4YNwqlTp4SjR48Kq1evFrp27SpYWVkpvAfKmjZtmiCRSISXL19+cr7o6GihUaNGgr+/v9gmk8kET09PoXnz5kJGRobYPnXqVMHa2lp49uyZ2Hb+/HlBIpEIgYGBYtubN28EFxcXwcXFRXjw4IHCNmUymbB//365fSevz+bC7BdfOh7GIgD/66KOiYnBTz/9BFtbWzRt2hRz585FZmYmACAqKgpOTk4AgOXLlyscNgGyundHjRoFR0dHWFlZoUePHvk6HPLPP/8gIyMDzs7O+arXwcEBABQOFwCAs7MzUlJScOHChU+uIzo6Gjt27ICLi0uOh3q0tLTg7e0t16tz9+5d/PDDD7Czs4OtrS2+++47/Pfff3LLZXdFX7lyBX5+fnBycpK7X9OZM2fg6ekJGxsb2NraYsiQIbh//77cOq5fv47Xr1/n+/XIjZ6eHtzd3RW64w8cOAAjIyO4urrmuFz2dvPz3n08ZufatWswMTER9xUg655XHTp0QGxsLK5evZrnOk1NTaGvr5/nfJcvX8abN2/g6ekp196vXz+kpKSIdwO/f/8+EhMT0aFDB7mbcrZq1QqGhoY4ePCg2JaUlAQAqFixotw6s3vGsnuAAODo0aOwsrKCtbW12GZhYQEnJyeFw3b16tWDt7c39u7diytXriAyMhIrVqyAu7t7ge5VlZvsw7hDhw4V/518qGzZshg7duwn16GpqSle9fnZs2cqqy2br68vDhw4gLVr12L79u0YNGgQWrVqBXd3d/z4448ICgqCv78/5s6dq3AneWWYm5sDyLptx6ecOHEC6enpcvuRhoYG+vbti+joaNy8eVNsP3bsGFq2bIlq1aqJbc7Ozqhdu7bcex4YGIjY2FhMmjRJvEHrhzQ0NNCpUye5fUdZHx+CzOlwNmXhYSwSZWZmwtvbG9bW1pgwYQIuXryIjRs3okaNGvD09ISJiQn8/Pzg5+cHNzc3uLm5AfjfYZP79++jb9++qFy5MgYPHgxDQ0McPnwYw4cPx7Jly8T5c3Lz5k2UL19e/HDKS/YH8Ic3/sxWr1496Ovr48aNG5/c5r///ouMjAx06dIlX9u8f/8++vXrhzJlyuCHH36AtrY2duzYgQEDBmDbtm1o3Lix3Pz+/v4wMTHB8OHDkZKSAgAICgqCj48PXF1dMX78eKSmpuLPP/+Ep6cn9u3bh+rVq4uvh4aGBho2bJiv2j6lU6dOGDRoECIiIlCzZk0AWWGnXbt20NbO+SOgXLlyqFmzJm7cuJHrmJ/cpKen5xhUstvu3LkDFxcX5Z5ELu7evQsAsLS0lGtv1KgRNDU1ERoaiq5du4pjT3KrKzQ0FDKZDJqamrC0tIShoSGWLFkCY2Nj1K1bF0+fPsX8+fNhZWUlBkGZTIbw8HD07NlTYZ1WVlY4d+4ckpKSxENeADBs2DAcPHgQU6ZMgbm5ObS0tHIdkJ+ZmZnjOBB9fX0YGhrm+ppkB9TC3rE7+4dE+fLlC7WejwUFBeHEiRPYuXMnvvrqKwCAIAhISUlBmTJlAADx8fHo2rUrKlSogNGjR4uHp/KSlpaGpKQkvH//HiEhIdi4cSPMzc3zPBwWGhoKQ0NDhVCSHURCQ0Ph4OCAmJgYxMXFKexv2fP++++/4uPTp09DX1//k59BuUlKSsrxvf94/NE333wj96MCAM6ePYv9+/fDxMRE6e2qM4YdEr1//x4dOnTA8OHDAQB9+/ZF9+7dsXv3bnh6esLQ0BDt2rWDn5+fwt23AWDmzJmoWrUq9uzZIx639vT0RN++fbFgwYJP/qN/9OjRJz/MkpOTER8fL47ZmTVrFjQ0NODu7q4wr7a2NqpUqYIHDx588vlmD3DO7x2bf//9d6Snp+PPP/9EjRo1AADdunVD+/btMX/+fGzbtk1ufmNjY2zatAlaWlric5g5cyZ69+6N6dOni/N1794d7du3x5o1a8T2R48ewdjYWO6LsqCaNm0KMzMzHDhwAD/99BMePnyI0NBQTJ48OceesWw1atTI8zXMSZ06dXDhwgU8e/ZM7j29fv06AMgNji2s2NhYaGlpKfTC6Orqonz58nj58iUAoFatWtDQ0MCNGzfkwsmjR4/EL5WEhARUqFABJiYmWLx4MX799Ve5oOfq6oqlS5eKAfHNmzdIS0tTGAsF/K8X6OXLl3Lvob6+Pvz8/ODt7Y3Hjx9j0qRJqFy5co7P7dGjRwpfZEDWF9y0adNyfU0ePXqEcuXKoWrVqrnOk5Ps1yEpKQmHDx/GiRMnIJVK5caDFZYgCFi2bBnGjBkjBp2TJ0/Cz88PL1++RLVq1TB9+nR4e3vj5MmTaNmyJZo1a4adO3fm2RsFAMePH5cb52RpaYlZs2blGuqzxcbGomLFinK9foD8+/jhf3N7z7P3CV1dXTx69Ah16tSBjo6O3HxJSUlyA79zCq+f+oGR/boBWYPCPxw0/vTpU0yfPh0uLi749ttvP/WUvzgMOySnb9++co/t7e3x999/57ncmzdvcOnSJYwaNUo8DJDN1dUVy5YtQ0xMTK4f7G/evMl1GgBMmjRJ7rGJiQnmzZuXaxewsbExXr9+/cmas+vM/jX5KZmZmTh//jzatm0rBh0AqFSpEjp16oRdu3Yp/Irv06ePGHQA4MKFC0hMTISHh4fcrzZNTU00btxY7gy3N2/ewNjYOM+68kNLSwvt27fHwYMH8dNPP+Hvv/9G1apV4eDg8MmwY2RkJPacKKNXr14IDAzEmDFj4OvrC1NTUxw+fBjHjx8HkPMNMwvq3bt3Cl8m2fT09MRtmZiYoEOHDggKCoKFhQXc3NwQExOD6dOnQ0dHB+np6XIDtU1MTNCwYUPY2dmhXr16CAsLw/r16+Hr64ulS5cCgDj/hwNSP9z2h/N8yNjYGJqampDJZLkeRgSyDsFkDxr+0Kf+nQBZ+3V+9ukPpaSkKAQrW1tbzJ8/X6n15CUkJATx8fHiQPuYmBiMGzcOLi4u6Nq1K54/f67wb71t27YIDAzM1/qbNGmCgIAAJCYm4uLFiwgPD0dqamqey7179+6T72P2fpSf9zx7XUlJSTn2wE2YMEHu8HC/fv0wZcoUuXmmTJmCOnXqKCw7Z84cyGSyHJ9DSkoKRowYASMjIyxcuFDus4cYdugDenp6Cl2fxsbGSEhIyHPZiIgICIKAJUuWYMmSJTnOExcX98kPauETdy4ZPnw4HBwckJKSguPHj+PgwYPQ1Mx9yJkgCAq/0j724Rk1eYmPj0dqamqOH0AWFhaQyWR48eKF3K+u7ENS2bLPKPruu+8+Wc+Hz0FVOnfujK1btyIsLAwHDhxAx44d83x98vMa5qR+/fpYsGABpk6dKoZnMzMzTJo0CX5+fuIXQHJysnh4D8gKZcp2vevr6+d6avH79+/lDltNmzYN7969w9y5c8XTvLt06YKaNWvi2LFjYl2RkZHw8vLC3Llz0a5dOwBZX7jm5ubw8fHBmTNn0KJFC/HL7cNf6R9uG5Af3wNkheYpU6agUqVKSElJwYwZMxAQEJBj/YaGhgUas1W2bNlPhtic6OnpYfXq1QCyvsirV69eoDMQ89pf7ty5A0tLSzGM/f3336hcuTKWLl0qfjkbGRnB19dXXKZixYp5/nDJZmpqClNTUwBA+/btsXr1agwcOBDHjh3LsTcmm76+/iffx+z9KD/vefa8ZcqUkdu/s40aNQr9+/cHgFyvoWVtbQ0rKyuF9k/9iPvtt98QERGBwMBA8Ww/+h+GHRIV5pdA9q+NQYMGoVmzZjnOkz1eJCfly5f/5CBCiUQifvC3bdsWqamp+O2332Bvb59jd31iYmKex+mzu+fDw8PRoEGDT85bEB9/0WWHl3nz5uX4wfvh65/X66Gsxo0bo2bNmpg5cyaioqLydfp3YmJigT8027dvj9atWyMsLAwymQwNGzbElStXAAC1a9cGAGzcuBHLly8XlzE3N8epU6eU2o6ZmRkyMzMRFxcndygrLS0Nb968QaVKlcS2cuXKYdWqVXj+/DmePXuGatWqwdzcHN9++y1MTEzE8V979+7F+/fv0apVK7ltZQ8ivnHjBlq0aIHy5ctDV1cXsbGxCnVlt324fSDrcgJ3797FihUrEBMTg2nTpmH//v35ej/yq27durh79y5evHiR70NZWlpaeQYrXV1dpKWl5RiCBUHA+/fvc+zx+NDH78mzZ8/QoEEDuX3/497a6OjoAo8bateuHRYvXoyTJ09+8rCOmZkZLl++rPDcPn4fs/+b23uevU8AWe9DWFgY0tPT5Xof69evX6Dn8imbN2/GgQMHMH/+/CL5LFMHDDuklNx+uWUf2tHR0SnQr9G6devi2LFj+Z5//PjxOHHiBFatWqUwfiEjIwMvXrzI8wyX5s2bQ0tLC/v370e3bt0+Oa+JiQkMDAzw+PFjhWmPHj2CpqZmnl8s2a9RxYoV83yN6tati/379+Pt27f5uq5Kfnh4eGDVqlWwsLDI1wdiVFRUoT6YdXV15b64ss+Oy37u3bp1g729vTj943CYH9nPIyQkRO6Mt5CQEMhkshzrr1atmngmTWJiIkJCQsQeHCCrB1IQBPEsxGwZGRkAILZrampCIpEgJCREYRvBwcGoUaOGXG/dixcvsHTpUrRp0wZt27aFTCZDUFAQ5syZg5YtW6rsfW7VqhUOHDiAv//+W7wgpSqYm5sjIyMDERERCj8knj59iszMzDwHEZctWxZv374VH5uamiI4OFhung97pQRBwO7du3Mcu5Qf2b0tH24zJw0aNMCuXbvw8OFD1KtXT2y/deuWOB3IOoRoYmKS63v+4f7WsmVL/Pfffzh+/Dg6duxYoPrz49q1a5g3bx6+++67fJ9s8SXiqeekFAMDAwCKp3JWrFgRjo6O2LFjhziI70O5XV00m42NDRISEvLd/V6zZk24u7tj3759Cr+yHjx4gPfv33/yaq8AULVqVfTu3Rvnzp3L8Yq7MpkMGzduRHR0NLS0tODi4oKTJ08iKipKnOfVq1c4cOAA7O3t8xxM3KxZM5QtWxZr1qzJ8dDLh6+RjY0NBEHI8UO1oHr37o0RI0Zg4sSJec779u1bRERE5Pka5teTJ08QGBiIVq1aiYcCa9SoAWdnZ/Hvw+CTX02bNkX58uUVTk/+888/YWBggJYtW35y+YULFyIzM1Pu0GLt2rUhCILCqePZp+9/eIZcu3btcPv2bbkr3j569AiXLl1SuNJx9uDz3377DUBWWPL398fr16+xaNGifD7jvLVr1w4SiQSrV6+WO2U6W1JSUoEuFNi8eXMAUBiIDwDbt2+Xmyc3FhYWuH37ttgT3LZtW4SGhmLJkiWIjIwUv7iBrDOgRo4ciejoaHh5eX1yvfHx8Tke9t21axcA+bP13r59i4cPH8oFoDZt2kBHRwd//PGH2CYIAgIDA1G5cmW5fwfu7u74559/8OLFC7Ht4sWLePLkidx73rdvX5iammL27Nk5/khSxWHqly9fYsyYMbCzs8OECRMKvT51xp4dUoq+vj7q1auHw4cPo3bt2ihfvjy++uorSCQSTJ06FZ6enujcuTP69OmDGjVq4NWrV/jvv/8QHR39yYHOLVu2hLa2Ni5cuIBvvvkmX7V4e3vj8OHD2Lx5M8aPHy+2X7hwAQYGBvnqYfLx8UFkZCRmzJiBY8eOoVWrVjAyMsKLFy9w5MgRPHr0SLx66pgxY3DhwgV4enrC09MTWlpa2LFjB9LS0vDLL7/kua2yZcvCz88PEyZMQI8ePdCxY0eYmJjg+fPnOHPmDOzs7MSBivb29ihfvjwuXrxY4F+1HzM3N1e4Sm9uLly4AEEQ8nUF5Zx07NgR7du3R9WqVREVFYXAwECUL18e/v7++Vr+2bNn+OuvvwBADHwrV64EkNUzk90Tp6+vj1GjRmHatGkYNWoUmjVrhmvXruHvv//G2LFj5Q5/rF27Fvfu3UPjxo2hpaWFkydP4ty5cxgzZoxcD1T37t2xceNGTJkyBXfv3sVXX32FO3fuYPfu3fjqq6/Qtm1bcV5PT0/s2rULP/74IwYNGgRtbW1s2rQJFStWxKBBg8T5jh8/jpMnT8LHx0euB7Bhw4bw9PTE9u3b0b17d7k63r59K74GH/vUaeU6OjpYvnw5Bg4ciP79+6N9+/aws7ODjo4O7t+/L15jKT9nN32oQYMG6N27N7Zs2YKnT5+K/74uXLiAM2fOoHfv3nn2BNrb2yM9PR2nTp1C27ZtUb9+fYwZMwa///47Vq5cCW1tbUycOBEzZ87EiBEj4Orqim3btuU5luvvv/9GYGCgeAJBcnIyzp07h/Pnz6NVq1Zy/4aOHz8OX19fzJ49Gz169ACQdbVqLy8vbNiwARkZGbCyssKJEydw7do1LFiwQO4w29ChQ3HkyBF4eXnBy8sLKSkp2LBhAyQSidyZfuXLl8fy5csxdOhQdO3aFR4eHrC0tISOjo74+QJA6bPmPjRjxgzEx8fjhx9+kLtWFJB1lmlRHDIrrRh2SGkzZszA9OnTMXv2bKSnp2PEiBGQSCSoV68e9uzZg+XLl2Pfvn148+aNeFZL9unsuTE1NUXz5s1x+PDhfIcdKysrODo64s8//8SPP/4oHgY4cuQI3Nzc8nXatoGBAdatW4e9e/ciKCgIK1euxLt371CpUiU0adIECxYsEAdVf/XVV9i+fTsWLlyINWvWQBAEWFtbY/78+QrX2MlN586dUalSJaxduxYbNmxAWloaKleuDAcHB/GDF8g6BNS5c2ccOXIkx1sGFLUjR47A3t7+k+OsPqV+/frYu3cvXr16hQoVKqB9+/YYNWqUwiniuYmKilIY6J792NHRUe6wY79+/aCjo4ONGzfi1KlTqFq1Knx9fRUGgkskEhw/fhynTp2CTCaDVCrF77//jg4dOsjNV6FCBezZswdLlizB6dOnxaDWs2dPjB07Vm5cStmyZbF161bMmjULq1atgkwmQ5MmTeDr6yt+QScnJ2PGjBlo0KBBjj0UY8aMwdGjR+Hn54ddu3aJX6zR0dG5/lrP6xo6tWrVQlBQEDZt2iQGLZlMhlq1aqF3794FvnHrtGnTIJFIsGfPHrE3qk6dOvj111/Rr1+/PJfX1dXFd999hzlz5uDrr7+GsbExfvzxR3Tr1g2RkZGoXbs2TE1N0ahRI9SpUyffA9bt7e1x8+ZNHDx4EK9evYK2tjbq1KkDX19fcTBwXsaPHw9jY2Ps2LEDe/fuRe3atTF//nyF8VRVq1bFtm3bMGfOHCxcuBA6Ojpo0aIFfHx8FMYs2dra4sCBAwgICMCZM2dw6NAhyGQyVK5cGfb29pg+fXqOF37Mr9evXyMzMxOzZ89WmDZixAiGnQ9oCKo85YOoEK5du4YBAwaIvUYFERoaiu7du2Pfvn2lfqBeZGQkOnTogHXr1qmsdyc/YmNj0aZNGyxatEiuF4NIFd6/f4++fftCU1MTK1euVBjEnS37RwtPoSZV4JgdKjEcHBzg4uKC9evXF3gda9euRbt27Up90AGyxrT07NkTa9eu/azb3bx5MyQSCYMOFQk9PT2sXbsWGhoa4gU5r169imfPnuHhw4cICgrCN998Ax8fH9y5c6e4yyU1wZ4dIiL67NLS0rB9+3Zs27ZNbtC/np4e3NzcMHLkyAL38BJ9jGGHiIiKVVRUFGJiYqCnpwcLCwvxrE8iVWHYISIiIrXGMTtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiL6jFasWAE3NzesWLGiuEv5YmgIgiAUdxFERERfgnfv3qFz586QyWTQ1NTE/v37oa+vX9xlqT327BAREX0mGRkZkMlkAACZTIaMjIxirujLwLBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBHRZ5MpyyzuEqgE+Vz7Ay8qSEREn5X/UX88ef2kuMsoFrL3MkRtiRIfV/eqDk29L7PfoXaF2pjabupn2Zb2Z9lKLq5evYoNGzYgJCQEsbGxWLFiBdq2bZvjvFOmTMGOHTvg6+uL77//Xmx/8+YNpk+fjtOnT0NTUxPu7u6YPHkyypQp85meBRERKePJ6ye4F3uvuMsoHmmAPv53xeQHrx4AusVYzxeiWONkSkoKpFIppk79dLI7fvw4bt26hUqVKilMGz9+PB48eICAgACsXr0a165dw5QpU4qqZCIiIiplijXstGjRAmPHjoWbm1uu88TExGD69OlYsGABdHR05KY9fPgQZ8+exYwZM9C4cWM4ODjg119/xcGDBxETE1PU5RMREVEpUKyHsfIik8nwyy+/wNvbG1999ZXC9Js3b8LIyAhWVlZim7OzMzQ1NREcHPzJEJXTtjh8iYioaGlpaRV3CVTCZGYWbpByfvapEh121q1bB21tbXh5eeU4/dWrVzAxMZFr09bWhrGxMWJjY5XaVkhICNLT0wtcKxERfZqBgQEaNmxY3GVQCRMeHo7U1NQCL29vb5/nPCU27ISEhGDLli3Yu3cvNDQ0inx7lpaW7NkhIiL6zKRSaZFvo8SGnWvXriEuLg6tWrUS2zIzMzF37lxs2bIFp06dgqmpKeLj4+WWy8jIQEJCAszMzJTanqbml3nqHxERfUaagAABGtCAoCHwanf4PIc2S2zY6dq1K5ydneXavL290bVrV/To0QMAYGtri8TERISEhMDS0hIAcOnSJchkMlhbW3/2momIiD5JG8ismwmtx1rIrJNZgr+F1UuxvszJycmIiIgQH0dFRSE0NBTGxsaoVq0aKlSoIDe/jo4OTE1NUbduXQCAhYUFmjVrht9++w3+/v5IT0/H9OnT4eHhgcqVK3/W50JERJQfGdYZyLDOKO4yvijFGnZCQkLkBh/Pnj0bANC9e3fMmTMnX+tYsGABpk+fju+++068qOCvv/5aJPUSERFR6cPbRRAR0Wc1MHDgl3sFZRJJzCQI+Dbgs2yLQ6OIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNRasYadq1evYujQoXB1dYVUKsWJEyfEaenp6Zg/fz46d+4MGxsbuLq6YsKECYiJiZFbx5s3bzBu3DjY2dnBwcEBkyZNQnJy8ud+KkRERFRCFWvYSUlJgVQqxdSpUxWmvXv3Dnfv3sWwYcOwd+9eLF++HI8fP8awYcPk5hs/fjwePHiAgIAArF69GteuXcOUKVM+11MgIiKiEk67ODfeokULtGjRIsdp5cqVQ0BAgFzbb7/9ht69e+P58+eoVq0aHj58iLNnz2L37t2wsrICAPz6668YMmQIJkyYgMqVKxf5cyAiIqKSrVjDjrKSkpKgoaEBIyMjAMDNmzdhZGQkBh0AcHZ2hqamJoKDg+Hm5pbvdctkMgiCoPKaiYjof7S0tIq7BCphMjMzC7V8fvapUhN23r9/jwULFsDDwwNly5YFALx69QomJiZy82lra8PY2BixsbFKrT8kJATp6ekqq5eIiOQZGBigYcOGxV0GlTDh4eFITU0t8PL29vZ5zlMqwk56ejpGjx4NQRDg7+9fJNuwtLRkzw4REdFnJpVKi3wbJT7spKenY8yYMXj+/Dk2b94s9uoAgKmpKeLj4+Xmz8jIQEJCAszMzJTajqYmz8InIiL63D7Hoc0S/Q2fHXSePn2KTZs2oUKFCnLTbW1tkZiYiJCQELHt0qVLkMlksLa2/tzlEhERUQlUrD07ycnJiIiIEB9HRUUhNDQUxsbGMDMzw6hRo3D37l2sWbMGmZmZ4jgcY2Nj6OrqwsLCAs2aNcNvv/0Gf39/pKenY/r06fDw8OCZWERERAQA0BCKcaDK5cuX4eXlpdDevXt3jBgxAm3atMlxuS1btqBJkyYAsi4qOH36dJw6dQqamppwd3fHr7/+ijJlyhRp7UREVDADAwfiXuy94i6DipnETIKAbwPynlEFijXsEBHRl4dhh4DPG3ZK9JgdIiIiosJi2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrWnnZ6bZs2fne4W+vr4FLoaIiIhI1fIVdu7evavwODMzE3Xq1AEAPHnyBJqammjUqJHqKyQiIiIqhHyFna1bt4r/HxAQgDJlymDu3LkwNjYGACQkJMDX1xcODg5FUyURERFRASk9Zmfjxo0YN26cGHQAwNjYGGPGjMHGjRtVWhwRERFRYSkddpKSkhAfH6/QHh8fj+TkZJUURURERKQqSocdNzc3+Pr64tixY4iOjkZ0dDSOHj2KyZMnw93dvShqJCIiIiqwfI3Z+ZC/vz/mzp2LcePGISMjAwCgpaWFXr16YcKECSovkIiIiKgwlA47BgYG8PPzw4QJExAREQEAqFmzJgwNDVVeHBEREVFhKR12shkaGqJ+/fqqrIWIiIhI5ZQOOykpKVi7di0uXbqEuLg4yGQyueknT55UWXFEREREhaV02Pn1119x5coVdO3aFWZmZtDQ0CiKuoiIiIhUQumw8++//2LNmjWwt7cvinqIiIiIVErpU8+NjIxQvnz5IiiFiIiISPWUDjujR4/GkiVLkJqaWhT1EBEREamU0oexAgICEBERAWdnZ1SvXh3a2vKr2LdvX77XdfXqVWzYsAEhISGIjY3FihUr0LZtW3G6IAhYunQpdu3ahcTERNjZ2cHPzw+1a9cW53nz5g2mT5+O06dPQ1NTE+7u7pg8eTLKlCmj7FMjIiIiNaR02PkwjBRWSkoKpFIpevbsiREjRihMX7duHbZu3Yo5c+agevXqWLJkCby9vXHo0CHo6ekBAMaPH4/Y2FgEBAQgPT0dkyZNwpQpU7Bw4UKV1UlERESll9JhJ6dQUlAtWrRAixYtcpwmCAK2bNmCYcOGiQFr3rx5cHZ2xokTJ+Dh4YGHDx/i7Nmz2L17N6ysrABknS02ZMgQTJgwAZUrV853LTKZDIIgFP5JERFRrrS0tIq7BCphMjMzC7V8fvapAl9UMCQkBA8fPgQAfPXVV2jYsGFBV5WjqKgoxMbGwtnZWWwrV64cGjdujJs3b8LDwwM3b96EkZGRGHQAwNnZGZqamggODoabm1u+txcSEoL09HSVPgciIvofAwMDlX9XUOkXHh5eqHHA+Tk7XOmwExcXh7Fjx+LKlSswMjICACQmJqJJkyZYvHgxTExMlK80B7GxsQCAihUryrVXrFgRr169AgC8evVKYXva2towNjYWl88vS0tL9uwQERF9ZlKptMi3oXTYmT59OpKTk3Hw4EFYWFgAAB48eICJEydixowZWLRokcqL/Bw0NZU+MY2IiIgK6XMc2lT6G/7s2bOYOnWqGHQAoF69epg6dSr+/fdflRVmZmYGIKsn6UNxcXEwNTUFAJiamiI+Pl5uekZGBhISEsTliYiI6MumdNiRyWTQ0dFRaNfW1la4T1ZhVK9eHWZmZrh48aLYlpSUhFu3bsHW1hYAYGtri8TERISEhIjzXLp0CTKZDNbW1iqrhYiIiEovpcNO06ZNMXPmTMTExIhtMTExmD17NpycnJRaV3JyMkJDQxEaGgoga1ByaGgonj9/Dg0NDXh5eWHVqlU4efIkwsPDMWHCBFSqVEk8O8vCwgLNmjXDb7/9huDgYFy/fh3Tp0+Hh4eHUmdiERERkfrSEJQclfvixQsMGzYMDx48QJUqVQAA0dHR+Oqrr7Bq1SqxLT8uX74MLy8vhfbu3btjzpw54kUFd+7cicTERNjb22Pq1KmoU6eOOG/2RQVPnTolXlTw119/5UUFiYhKqIGBA3Ev9l5xl0HFTGImQcC3AZ9lW0qHHSDrGjgXLlzAo0ePAGT1sHx4ijgREVFuGHYI+Lxhp0DX2dHQ0ICLiwtcXFxUXQ8RERGRSik9ZmfGjBnYsmWLQvu2bdswc+ZMlRRFREREpCpKh52jR4/Czs5Ood3W1hZHjx5VSVFEREREqqJ02Hnz5g3KlSun0F62bFm8fv1aJUURERERqYrSYadWrVo4e/asQvu///6LGjVqqKQoIiIiIlVReoDy999/j+nTpyM+Ph5NmzYFAFy8eBEBAQGYNGmSygskIiIiKgylw06vXr2QlpaG1atXY+XKlQAAc3Nz+Pn5oVu3bqquj4iIiKhQCnTquaenJzw9PREfHw89PT1ewI+IiIhKrALd6jsjIwMXLlzAsWPHkH1NwpiYGCQnJ6u0OCIiIqLCUrpn59mzZ/jhhx/w4sULpKWlwcXFBWXLlsW6deuQlpaGadOmFUWdRERERAWidM/OzJkzYWlpiStXrkBPT09sd3Nzw6VLl1RaHBEREVFhKd2zc/36dfz555/Q1dWVazc3N5e7EzoRERFRSaB0z45MJoNMJlNoj46O5kBlIiIiKnGUDjsuLi7YvHmzXFtycjKWLVuGFi1aqKwwIiIiIlVQOuz4+Pjgxo0b6NixI9LS0jB+/Hi0bt0aMTExGD9+fFHUSERERFRgSo/ZqVKlCv766y8cOnQIYWFhSElJQa9evdC5c2fo6+sXRY1EREREBVagiwpqa2ujS5cu6NKli6rrISIiIlKpfB/Gevz4MYKDg+XaLl68iAEDBqBXr15YvXq1yosjIiIiKqx8h50FCxbg9OnT4uPIyEgMHToUurq6sLGxwZo1a7Bp06aiqJGIiIiowPJ9GCskJAQ//PCD+Hj//v2oXbs2NmzYAACQSqXYtm0bvv/+e5UXSURERFRQ+e7Zef36NapUqSI+vnz5Mlq1aiU+btKkCZ49e6ba6oiIiIgKKd9hx9jYGLGxsQCyLiwYEhICGxsbcXp6erp4U1AiIiKikiLfYcfR0RErV67EixcvsHnzZshkMjg6OorTHzx4AHNz8yIpkoiIiKig8j1mZ+zYsRg0aBBatWoFLS0tTJ48GYaGhuL0v/76C02bNi2SIomIiIgKKt9hp3r16jh06BAePHiAChUqoHLlynLTR40apdBGREREVNyUuqigtrY26tevn+O03NqJiIiIipPS98YiIiIiKk0YdoiIiEitMewQERGRWlMq7GRkZGD58uWIjo4uqnqIiIiIVEqpsKOtrY0NGzYgIyOjqOohIiIiUimlD2M1bdoUV69eLYpaiIiIiFROqVPPAaB58+ZYuHAh7t27h0aNGsHAwEBueps2bVRWHBEREVFhKR12/P39AQABAQEK0zQ0NBAaGlr4qoiIiIhUROmwExYWVhR1EBERERWJQp16/v79e1XVQURERFQklA47mZmZWLFiBZo1awZbW1tERkYCAH7//Xfs2rVL5QUSERERFYbSYWfVqlXYt28ffvnlF+jo6IjtEokEu3fvVmlxmZmZ+P3339G6dWtYW1ujbdu2WLFiBQRBEOcRBAFLliyBq6srrK2t8f333+PJkycqrYOIiIhKL6XDzl9//YXp06ejS5cu0NT83+JSqRSPHj1SaXHr1q3Dn3/+iSlTpuDQoUMYP3481q9fj61bt8rNs3XrVvj5+WHnzp0wMDCAt7c3D7ERERERgAKEnZiYGNSsWVOhXRAElV9s8ObNm2jTpg1atmyJ6tWro3379nB1dUVwcLC4zS1btmDYsGFo27Yt6tevj3nz5uHly5c4ceKESmshIiKi0knps7Hq1auHa9euwdzcXK79yJEjaNCggcoKAwBbW1vs3LkTjx8/Rp06dRAWFobr16/Dx8cHABAVFYXY2Fg4OzuLy5QrVw6NGzfGzZs34eHhke9tyWQyucNjRESkelpaWsVdApUwmZmZhVo+P/uU0mHnp59+go+PD2JiYiAIAo4dO4bHjx8jKCgIa9asKVChuRkyZAiSkpLQoUMHaGlpITMzE2PHjkWXLl0AALGxsQCAihUryi1XsWJFvHr1SqlthYSEID09XTWFExGRAgMDAzRs2LC4y6ASJjw8HKmpqQVe3t7ePs95lA47bdu2xerVq7FixQoYGBhg6dKlaNiwIVavXg0XF5cCFZqbw4cPY//+/Vi4cCHq1auH0NBQzJ49G5UqVUL37t1Vui1LS0v27BAREX1mUqm0yLehVNgRBAFPnz6FsbEx1q1bB21tpbOSUubNm4chQ4aIh6OkUimeP3+ONWvWoHv37jAzMwMAxMXFoVKlSuJycXFxqF+/vlLb+nCwNREREX0en+PQZr6/4SMjI9GlSxd06NABXbp0gZubG27fvl2UteHdu3fQ0NCQa9PS0hJ7YKpXrw4zMzNcvHhRnJ6UlIRbt27B1ta2SGsjIiKi0iHfXTPz589HRkYG5s+fDz09PWzYsAFTpkzBvn37iqy4Vq1aYfXq1ahWrZp4GCsgIAA9e/YEkHUvLi8vL6xatQq1atVC9erVsWTJElSqVAlt27YtsrqIiIio9Mh32Ll+/TqWLFkCBwcHAEDjxo3RokULpKSkwNDQsEiK+/XXX7FkyRL4+/uLh6q++eYbDB8+XJxn8ODBSE1NxZQpU5CYmAh7e3usX78eenp6RVITERERlS4aQj5H5davXx/nzp2Dqamp2GZra4u///4bNWrUKLICiYhIvQwMHIh7sfeKuwwqZhIzCQK+Dfgs28p3z46GhgZSUlKQlJQk15acnCzXVrZsWdVWSERERFQI+Q47giCgXbt2Cm3Zp4ALggANDQ2EhoaqtkIiokJYsWIFgoKC0K1bN7lD4ET05ch32NmyZUtR1kFEpHLv3r1DUFAQZDIZgoKC4O3tDX19/eIui4g+s3yHHUdHx6Ksg4hI5TIyMiCTyQBk3RJG1ffvI6LSgVfSI1Jjgqxw95wh9cL9gb5URXsJZCIqVhqaWni11wfprx4VdynFIjlNJvc4etP3KKP7Zf7G0zGtC9Mec4q7DKJiwbBDpObSXz1CevSXeeKAkAlowAwCNKAJAUJsONJ5022iL86X+ROHiL4IelpAW/NUaEJAG/NU6DHoEH2RCt2zk5SUhEuXLqFOnTqwsLBQRU1ERCrjWS8JnvWS8p6RiNSW0j07o0ePxrZt2wBkndbZs2dPjBkzBl26dMHRo0dVXiARERFRYSgddq5duybeH+v48eMQBAFXr17F5MmTsWrVKpUXSERERFQYSoedt2/fwtjYGABw9uxZuLu7w8DAAC1btsTTp09VXiARERFRYSgddqpWrYqbN28iJSUFZ8+ehYuLCwAgMTERurq6Ki+QiIiIqDCUHqDs5eWFX375BYaGhqhatSqaNGkCALh69SokEonKCyQiIiIqDKXDTr9+/WBtbY3o6Gg4OztDUzOrc6hGjRoYM2aMqusjIiIiKpQCnXpuZWUFqVSKqKgo1KxZE9ra2mjZsqWKSyMiIiIqPKXH7KSmpmLSpEmwsbFBp06d8OLFCwDA9OnTsXbtWpUXSERERFQYSoedhQsXIiwsDFu2bIGenp7Y7uTkhEOHDqm0OCIiIqLCUvow1smTJ7F48WLY2NjItX/11VeIiIhQVV1EREREKqF0z058fDwqVqyo0J6amgoNDQ2VFEVERESkKkqHHUtLS/zzzz8K7bt27VLo7SEiIiIqbkofxho7diwGDx6MBw8eIDMzE1u2bMHDhw9x8+ZNbN26tShqJCIiIiowpXt2HBwc8NdffyEzMxMSiQTnz5+HiYkJAgMDYWlpWRQ1EhERERVYga6zU7NmTcyYMUPVtRARERGpnNI9O2fOnMHZs2cV2s+ePYszZ86opCgiIiIiVVE67CxYsAAymUyhXRAELFy4UCVFEREREamK0mHn6dOnsLCwUGivW7cur7NDREREJY7SYadcuXKIjIxUaI+IiICBgYFKiiIiIiJSFaXDTps2bTBr1iy5XpynT59izpw5aN26tUqLIyIiIiospc/G+uWXX/DDDz+gQ4cOqFy5MgAgJiYG9vb2mDhxosoLJCIiIioMpcNOuXLlEBgYiPPnzyMsLAz6+vqQSqX4+uuvi6I+IiIiokIp0HV2NDQ04OrqCldXV1XXQ0RERKRSBQo7Fy9exMWLFxEXF6dwGvrs2bNVUhgRERGRKigddpYvX44VK1bA0tISZmZmvNM5ERERlWhKh53AwEDMnj0b3bp1K4JyiIiIiFRL6VPP09PTYWdnVxS1EBEREamc0mGnV69e2L9/f1HUQkRERKRySh/Gev/+PXbu3ImLFy9CKpVCW1t+Fb6+viorDsi6hs/8+fNx9uxZpKamolatWpg1axasrKwAZN2Ta+nSpdi1axcSExNhZ2cHPz8/1K5dW6V1EBERUemkdNgJDw9H/fr1AQD37t2Tm6bqwcoJCQno27cvmjRpgnXr1qFChQp4+vQpjI2NxXnWrVuHrVu3Ys6cOahevTqWLFkCb29vHDp0CHp6eiqth4iIiEofpcPO1q1bi6KOHK1btw5VqlSRO529Ro0a4v8LgoAtW7Zg2LBhaNu2LQBg3rx5cHZ2xokTJ+Dh4fHZaiUiIqKSqUDX2QGy7ocVERGBr7/+Gvr6+hAEQeU9O6dOnYKrqytGjRqFq1evonLlyvD09ESfPn0AAFFRUYiNjYWzs7O4TLly5dC4cWPcvHlTqbAjk8kgCIJK6ycqblpaWsVdApUwmZmZxbp97pP0scLuk/nZp5QOO69fv8aYMWNw+fJlaGho4NixY6hRowYmTZoEY2Nj+Pj4FKjYnERGRuLPP//EwIEDMXToUNy+fRszZsyAjo4OunfvjtjYWABAxYoV5ZarWLEiXr16pdS2QkJCkJ6errLaiYqbgYEBGjZsWNxlUAkTHh6O1NTUYtk290nKSWH3SXt7+zznUTrszJ49G9ra2vjnn3/QoUMHsb1jx46YM2eOSsOOIAiwtLTEzz//DABo2LAh7t+/j8DAQHTv3l1l2wEAS0tL9uwQkdqTSqXFXQKRnM+xTyodds6fP48NGzagSpUqcu21a9fG8+fPVVYYAJiZmcHCwkKurW7dujh69Kg4HQDi4uJQqVIlcZ64uDhxEHV+aWoqfRY+EVGpw8NIVNJ8jn1S6W/4lJQU6OvrK7S/efMGurq6Kikqm52dHR4/fizX9uTJE5ibmwMAqlevDjMzM1y8eFGcnpSUhFu3bsHW1laltRAREVHppHTYcXBwQFBQkFybTCbD+vXr0aRJE1XVBQD47rvvcOvWLaxevRpPnz7F/v37sXPnTnh6egLIOtXdy8sLq1atwsmTJxEeHo4JEyagUqVK4tlZRERE9GVT+jDWL7/8gu+//14c0Dt//nw8ePAACQkJ+PPPP1VanLW1NZYvX45FixZhxYoVqF69OiZNmoQuXbqI8wwePBipqamYMmUKEhMTYW9vj/Xr1/MaO0RERAQA0BAKMCr37du32LZtG8LCwpCSkoKGDRuiX79+cuNmiKhkeLG2D9KjQ4u7DCpmOlUaoOqQncVdBgBgYOBA3Iu9l/eMpNYkZhIEfBvwWbaldM/O8+fPUbVqVQwbNizHadWqVVNJYURERESqoPSYnTZt2iA+Pl6h/fXr12jTpo1KiiIiIiJSFaXDTm5XSk5JSeE4GSIiIipx8n0YK/v+VBoaGvj9999hYGAgTsvMzERwcLDS17YhIiIiKmr5Djt3794FkNWzc+/ePejo6IjTdHV1Ub9+fQwaNEj1FRIREREVQr7DTvbdzn19fTF58mSULVu2yIoiIiIiUpUC3RuLiIiIqLRQOuykpKRg7dq1uHTpEuLi4iCTyeSmnzx5UmXFERERERWW0mHn119/xZUrV9C1a1eYmZnleGYWERERUUmhdNj5999/sWbNGtjb2xdFPUREREQqpfR1doyMjFC+fPkiKIWIiIhI9ZQOO6NHj8aSJUuQmppaFPUQERERqZTSh7ECAgIQEREBZ2dnVK9eHdra8qvYt2+fyoojIiIiKiylw07btm2Log4iIiKiIqF02BkxYkRR1EFERERUJJQeswMAiYmJ2LVrFxYuXIg3b94AAO7cuYOYmBhV1kZERERUaEr37ISFhWHgwIEoV64cnj17hj59+qB8+fI4duwYXrx4gXnz5hVFnUREREQFonTPzpw5c9C9e3ccO3YMurq6YnuLFi1w7do1lRZHREREVFhKh53bt2/j22+/VWivXLkyYmNjVVIUERERkaooHXZ0dXWRlJSk0P7kyROYmJiopCgiIiIiVVE67LRu3RorVqxAenq62Pb8+XMsWLAA7u7uKi2OiIiIqLCUDjs+Pj5ISUmBs7Mz3r9/jwEDBsDd3R1lypTB2LFji6JGIiIiogJT+myscuXKISAgANevX0dYWBhSUlLQqFEjODs7F0V9RERERIWidNjJZm9vzzufExERUYmX78NYN2/exOnTp+XagoKC0Lp1azg5OeG3335DWlqaygskIiIiKox8h50VK1bg/v374uPw8HBMnjwZzs7OGDJkCE6fPo01a9YUSZFEREREBZXvsBMWFgYnJyfx8aFDh2BtbY0ZM2Zg4MCBmDx5Mg4fPlwkRRIREREVVL7DTkJCAkxNTcXHV65cQfPmzcXHVlZWePHihWqrIyIiIiqkfIcdU1NTREVFAQDS0tJw9+5d2NjYiNOTk5Oho6Oj8gKJiIiICiPfYad58+ZYuHAhrl27hkWLFkFfX1/ubKzw8HDUqFGjSIokIiIiKqh8h53Ro0dDS0sL/fv3x86dOzFjxgy5G4Hu2bMHrq6uRVIkERERUUHl+zo7JiYm2L59O96+fQtDQ0NoaWnJTV+yZAkMDQ1VXiARERFRYRToCso5KV++fGFrISIiIlI5pe+NRURERFSaMOwQERGRWmPYISIiIrXGsENERERqjWGHiIiI1FqpCjtr166FVCrFzJkzxbb379/D398fTZo0ga2tLUaOHIlXr14VY5VERERUkpSasBMcHIzAwEBIpVK59lmzZuH06dP4/fffsXXrVrx8+RIjRowopiqJiIiopFH6OjvFITk5Gb/88gtmzJiBVatWie1v377Fnj17sGDBAvGO7LNmzULHjh3x33//yd27Ky8ymQyCIKi6dKJi9fHFP4kyMzOLdfvcJ+ljhd0n87NPlYqwM23aNLRo0QLOzs5yYSckJATp6elwdnYW2ywsLFCtWjWlw072uojUhYGBARo2bFjcZVAJEx4ejtTU1GLZNvdJyklh98kP79OZmxIfdg4ePIi7d+9i9+7dCtNevXoFHR0dGBkZybVXrFgRsbGxSm3H0tKSPTtEpPY+HgpAVNw+xz5ZosPOixcvMHPmTGzcuBF6enpFui1NzVIzfImIqMB4GIlKms+xT5bosHPnzh3ExcWhR48eYltmZiauXr2K7du3Y8OGDUhPT0diYqJc705cXBzMzMyKo2QiIiIqYUp02GnatCn2798v1+br64u6deti8ODBqFq1KnR0dHDx4kW0a9cOAPDo0SM8f/5cqfE6REREpL5KdNgpW7YsJBKJXJuhoSHKly8vtvfs2RNz5syBsbExypYtixkzZsDW1pZhh4iIiACUouvs5GbSpElo2bIlRo0ahf79+8PU1BTLli0r7rK+SCtWrICbmxtWrFhR3KUQERGJNASegkQq8O7dO3Tu3BkymQyamprYv38/9PX1i7ssAvBibR+kR4cWdxlUzHSqNEDVITuLuwwAwMDAgbgXe6+4y6BiJjGTIODbgM+yrVLfs0MlQ0ZGBmQyGYCsCzRmZGQUc0VERERZGHaIiIhIrTHsEBERkVpj2CEiIiK1xrCjIpn/P16FCOD+QERUkpTo6+yUJlqamvj1j7N4/DKhuEspFrL0d3KPB686Ak2dL/NsrDqVjDHDs1lxl0FERP+PYUeFHr9MQNiz+OIuo1hoZKTB+IPH95+/gaCtW2z1EBERZeNhLCIiIlJrDDtERESk1hh2SCUETU0I0Mj6f2hA0OSuRUREJQO/kUg1NLWRZlYfAjSQZlYf0ORwMCIiKhn4jUQqk1rDEak1HIu7DCIiIjns2SEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1xrBDREREao1hh4iIiNQaww4RERGpNYYdIiIiUmsMO0RERKTWGHaIiIhIrTHsEBERkVpj2CEiIiK1VqLDzpo1a9CzZ0/Y2trCyckJP/30Ex49eiQ3z/v37+Hv748mTZrA1tYWI0eOxKtXr4qpYiIiIippSnTYuXLlCvr164edO3ciICAAGRkZ8Pb2RkpKijjPrFmzcPr0afz+++/YunUrXr58iREjRhRj1URERFSSaBd3AZ+yYcMGucdz5syBk5MT7ty5g6+//hpv377Fnj17sGDBAjg5OQHICj8dO3bEf//9Bxsbm2KomoiIiEqSEh12Pvb27VsAgLGxMQAgJCQE6enpcHZ2FuexsLBAtWrVlA47MpkMgiAUuDYtLa0CL0vqKTMzs7hL4H5JCop7v+Q+SR8r7D6Zn32q1IQdmUyGWbNmwc7ODhKJBADw6tUr6OjowMjISG7eihUrIjY2Vqn1ZwengjAwMEDDhg0LtCypr/DwcKSmphbb9rlfUk6Kc7/kPkk5Kew+aW9vn+c8pSbs+Pv74/79+/jjjz+KZP2WlpaF6tkh+phUKi3uEogUcL+kkuZz7JOlIuxMmzYN//zzD7Zt24YqVaqI7aampkhPT0diYqJc705cXBzMzMyU2oamZokeq02lELvrqSTifkklzefYJ0v0N7wgCJg2bRqOHz+OzZs3o0aNGnLTLS0toaOjg4sXL4ptjx49wvPnzzk4mYiIiACU8J4df39/HDhwACtXrkSZMmXEcTjlypWDvr4+ypUrh549e2LOnDkwNjZG2bJlMWPGDNja2jLsEBEREYASHnb+/PNPAMCAAQPk2mfPno0ePXoAACZNmgRNTU2MGjUKaWlpcHV1xdSpUz97rURERFQyleiwEx4enuc8enp6mDp1KgMOERER5ahEj9khIiIiKiyGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJrDDtERESk1hh2iIiISK0x7BAREZFaU5uws337drRu3RpWVlbo3bs3goODi7skIiIiKgHUIuwcOnQIs2fPxvDhw7Fv3z7Ur18f3t7eiIuLK+7SiIiIqJipRdgJCAhAnz590LNnT9SrVw/+/v7Q19fHnj17irs0IiIiKmbaxV1AYaWlpeHOnTv48ccfxTZNTU04Ozvj5s2b+VqHIAjIyMiAIAgFrkNLSwtfVTGGrpZGgddB6qGWmREyMzORmZlZ3KVAS0sLWmYSyDR1i7sUKmZaFWuXiP1SS0sL9UzqQUdTp1jroOJXq3wtleyTWlpa0NTUhIZG7t+/pT7svH79GpmZmahYsaJce8WKFfHo0aN8rUMmk6lkjE/nrwyBrwwLvR4q/f7777/iLuF/anYHahZ3EVQSRJaQ/bJ9xfZAxbznI/Wnqs9KGxsbaGlp5Tq91IcdVdDU1ISNjU1xl0FEREQFoKn56VE5pT7sVKhQAVpaWgqDkePi4mBqapqvdWhoaHwyERIREVHpVeoHKOvq6qJRo0a4ePGi2CaTyXDx4kXY2toWY2VERERUEpT6nh0AGDhwICZOnAhLS0tYW1tj8+bNSE1NRY8ePYq7NCIiIipmahF2OnbsiPj4eCxduhSxsbFo0KAB1q9fn+/DWERERKS+NITCnG9NREREVMKV+jE7RERERJ/CsENERERqjWGHiIiI1BrDDpVIe/fuhYODQ3GXQUREaoBh5wvi4+MDqVQKqVQKS0tLuLm5Yfny5cjIyPjkcnv37hWXy+0vKirqMz0LKm1evHgBX19fuLq6wtLSEq1atcKMGTPw+vXrz1rH0KFD4e3tneO0a9euQSqVIiwsDFFRUZBKpQgNDQUA8bGTkxOSkpLkluvatSuWLVsm13b//n2MHj0aTZs2haWlJdq1a4clS5YgNTW1aJ4YKY375Je3TzLsfGGaNWuGc+fO4ejRoxg4cCCWL1+ODRs2fHKZjh074ty5c+Kfra0t+vTpI9dWtWrVfNeQlpZW2KdBpURkZCR69uyJp0+fYtGiRTh27Bj8/Pxw6dIlfPvtt3jz5s1nq6VXr164cOECoqOjFabt2bMHlpaWqF+/fq7LJycnY+PGjZ/cxn///Yc+ffogPT0da9euxdGjRzFmzBjs27cPAwcO5L5fAnCf/DL3SYadL4yuri7MzMxgbm4OT09PODs74/Dhw7Czs8ORI0fk5j1x4gRsbGyQkZEBMzMz8U9HRwf6+vri4/T0dIwYMQK2traws7PD6NGj8erVK3E9y5YtQ9euXbFr1y60bt0a1tbWAIDExERMmTIFzs7OsLKyQqdOnXD69Gm5Gs6ePYsOHTrA1tYW3t7eePnyZdG/SKQy/v7+0NHRwcaNG+Ho6Ihq1aqhRYsWCAgIQExMDBYvXgwAaN26NVasWIGff/4ZNjY2aNasGbZv3y63rsTEREyePBlNmzaFnZ0dvLy8EBYWJk7P3s+CgoLQunVr2NvbY+zYseIv35YtW8LExAR79+6VW29ycjKOHDmCXr16ffK59O/fHwEBAQq3pskmCAImT56MunXrYvny5bC2toa5uTk6dOiA1atX47///sOmTZuUfQlJxbhPfpn7JMPOF05PTw+amprw8PBQ+Ae3Z88etGvXDmXLls11eZlMhp9++gkJCQnYunUrAgICEBkZibFjx8rNFxERgaNHj2L58uUICgqCTCbD4MGDcePGDcyfPx+HDh3CuHHj5G7m9u7dO2zcuBHz5s3Dtm3b8OLFC8ydO1e1LwAVmTdv3uDcuXPw9PSEvr6+3DQzMzN07twZhw8fRvalvjZs2ID69etj3759GDJkCGbOnInz58+Ly4wePRpxcXFYt24d9u7di0aNGuG7776T+yUeERGBkydPYvXq1VizZg2uXr2KdevWAQC0tbXRtWtX7Nu3Dx9eXuzIkSOQyWTo1KnTJ59Pp06dUKtWLaxYsSLH6aGhoXjw4AEGDhyocFPC+vXrw9nZGQcPHsz7haMiw33yf760fZJh5wslCAIuXLiAc+fOoUmTJujduzfOnTsn9pzExcXh33//Rc+ePT+5nosXL+LevXtYuHAhLC0t0bhxY8ybNw9XrlxBcHCwOF96ejrmzZuHhg0bon79+rhw4QKCg4OxbNkyuLi4oEaNGmjVqhVatGght4y/vz+srKzQqFEj9OvXD5cuXSqaF4RU7unTpxAEARYWFjlOt7CwQEJCAuLj4wEAdnZ2GDJkCOrUqYMBAwagXbt24q/Oa9euITg4GEuXLoWVlRVq166NiRMnwsjICEePHhXXKQgCZs+eDYlEAgcHB3Tp0kXuvnk9e/ZEREQErly5Irbt3bsX7u7uKFeu3Cefj4aGBsaNG4edO3ciIiJCYfqTJ0/E55WTunXrivNQ8eA+Ke9L2icZdr4w//zzD2xtbWFlZYXBgwejY8eOGDlyJKytrVGvXj0EBQUBAP7++29Uq1YNX3/99SfX9/DhQ1SpUkVuzE69evVgZGSER48eiW3VqlWDiYmJ+Dg0NBRVqlRBnTp1cl23gYEBatasKT6uVKlSrt21VHLl9yLtNjY2Co8fPnwIAAgPD0dKSgqaNGkCW1tb8S8qKkruQ97c3FyuJ/LjfcbCwgK2trbYs2cPgKwvv2vXruV5uCBbs2bNYGdnhyVLluQ6Dy9KX/Jxn/zyqMW9sSj/mjRpAj8/P+jo6KBSpUrQ1v7fLtC7d29s374dQ4YMwd69e9GjRw9oaGioZLsGBgZyjz/uQs7Jh7UBWb9i+I+29KhZsyY0NDTw8OFDuLm5KUx/+PAhjI2N5UJwbpKTk2FmZoatW7cqTPvw1+/H+wyg+EHfq1cvzJgxA1OmTMHevXtRs2ZNODo65ucpAQDGjx+Pb775RuEsmtq1awPIel4NGzZUWO7Ro0fiPFQ8uE/K+5L2SfbsfGEMDAxQq1YtVKtWTeEfYZcuXfD8+XNs2bIFDx48QPfu3fNcn4WFBaKjo/HixQux7cGDB0hMTMy16xQApFIpoqOj8fjx44I/GSrRKlSoABcXF/zxxx949+6d3LTY2Fjs378fHTp0EAP1rVu35Oa5deuWuA81atQIr169gpaWFmrVqiX3l58vpg9lb/PAgQMICgpCz549lQr11tbWcHNzw8KFC+XaGzRogLp162LTpk2QyWRy08LCwnDhwgV4eHgoVSupFvfJ//nS9kmGHRIZGxvDzc0N8+bNg4uLC6pUqZLnMs7OzpBIJBg/fjzu3LmD4OBgTJgwAY6OjrCyssp1OUdHRzg4OGDUqFE4f/48IiMjcebMGfz777+qfEpUzH777TekpaXB29sbV69exYsXL/Dvv/9i0KBBqFy5stxA9hs3bmDdunV4/Pgxtm/fjiNHjsDLywtA1n5mY2OD4cOH49y5c4iKisKNGzewePFi3L59W6maypQpg44dO2LRokWIjY3NV6j/2NixY3Hp0iW5sK6hoYGZM2fi4cOHGDlyJIKDg/H8+XMcPnwYQ4cOhY2NDb7//nult0WqxX3yy9wnGXZITq9evZCenp7nwORsGhoaWLlyJYyMjNC/f398//33qFGjhnj65qcsW7YMlpaW+Pnnn+Hh4YEFCxYo/Pqg0q127drYs2cPatSogTFjxsDNzQ1TpkxBkyZNEBgYiPLly4vzDhw4ECEhIejevTtWrVoFHx8fNGvWDEDWfrZ27Vp8/fXX8PX1Rfv27fHzzz/j2bNnMDU1VbquXr16ISEhAa6urqhcubLSy9epUwc9e/bE+/fv5drt7OywY8cOaGlpYfDgwXB3d8eiRYvQrVs3BAQEQFdXV+ltkWpxn/wy90kNgYMg6ANBQUGYPXs2zp49+8X8I6Di17p1a3h5eX0xvzKp5OM+qV44QJkAAKmpqYiNjcW6devw7bffMugQEZHa4GEsAgCsX78eHTp0gKmpKYYMGVLc5RAREakMD2MRERGRWmPPDhEREak1hh0iIiJSaww7RERU6ly+fBlSqRSJiYnFXUquWrduXSrvKl4aXltlMewQUakQHx+PqVOnomXLlrC0tISLiwu8vb1x/fr14i6NVEwqlX7yb9myZcVdYomybNkydO3atcDzRUVFQSqVIjQ0FABga2uLc+fO5Xkj0g/5+Pjgp59+yn/RnxlPPSeiUmHkyJFIT0/HnDlzUKNGDcTFxeHixYt48+ZNcZdGKnbu3Dnx/w8dOoSlS5fiyJEjYpuhoSFCQkKUXm9aWhovq5EPurq6MDMzK+4yVIo9O0RU4iUmJuLatWsYP348mjZtCnNzc1hbW+PHH39EmzZt4Ovrix9//FFumfT0dDg5OWHXrl0AgAEDBmDGjBmYN28eHB0d4eLiotBDkJiYiClTpsDZ2RlWVlbo1KkTTp8+/dmeJ2UxMzMT/8qVKwcNDQ25tjJlyojz3rlzBz169EDjxo3x7bff4tGjR+K07J6MXbt2oXXr1rC2tgYAPH/+HMOGDYOtrS3s7OwwevRovHr1Slwup16KmTNnYsCAAeLjpKQkjBs3DjY2NnB1dcWmTZswYMAAzJw5U265d+/ewdfXF7a2tmjZsiV27NghTsvuUTl48CC+/fZbcZ+7cuWKOM/evXvh4OAgt84TJ05AKpWK05cvX46wsDCx52vv3r1Kv+Yf+vgwVnYNZ8+eRYcOHWBrawtvb2+8fPkSQNbrvG/fPpw8eVKs4fLly4WqQdUYdoioxDM0NIShoSFOnDiBtLQ0hem9e/fG2bNnxQ9fAPjnn3/w7t07dOzYUWzbt28fDA0NsXPnTvzyyy9YsWIFzp8/DwCQyWQYPHgwbty4gfnz5+PQoUMYN24cNDX5MVmSLV68GD4+PtizZw+0tLQwadIkuekRERE4evQoli9fjqCgIMhkMvz0009ISEjA1q1bERAQgMjISLl7YuXHnDlzcPPmTaxatQobN27EtWvXcOfOHYX5AgICYGlpiaCgIHh6esLPz08ukAHAvHnzMHDgQAQFBcHGxgZDhw7F69ev81VHx44dMWjQIHz11Vc4d+4czp07J7fPq8q7d++wceNGzJs3D9u2bcOLFy8wd+5cAMCgQYPQoUMHNGvWTKzB1tZW5TUUBg9jEVGJp62tjTlz5uC3335DYGAgGjZsCEdHR3Ts2BH169eHnZ0d6tSpg7/++guDBw8GAOzZswft27eX6wWQSqUYMWIEgKx7JG3btg0XL16Ei4sLLly4gODgYBw6dAh16tQBANSoUePzP1lSytixY+Ho6AgAGDJkCIYMGYL3799DT08PQFYP37x588Q7kZ8/fx737t3DyZMnUbVqVQBZYcPDwwPBwcFi78+nJCUlISgoCAsWLICTkxMAYPbs2eJ9sz7UvHlz9OvXDwAwePBgbNq0CZcvX0bdunXFefr164d27doBAPz8/HD27Fns3r1b3Jc/RV9fH4aGhtDS0srXoad79+4pBJH8XG4vPT0d/v7+qFmzpljzypUrAWTdyFRfXx9paWkl9vAXf7IQUanQrl07nD17FqtWrUKzZs1w5coV9OjRQ+yy7927t/j/r169wtmzZxVuaJvd9Z/NzMwMcXFxAIDQ0FBUqVJFDDpUOnz4nmZ/0Wa/pwBQrVo1MegAwMOHD1GlShUx6ABAvXr1YGRkpNDjkpuoqCikp6fLBaNy5crluO98WJ+GhgZMTU3l6gMgFz60tbVhaWmZ71qUVadOHQQFBcn9rV27Ns/lDAwMxKADAJUqVVJ4HiUZww4RlRp6enpwcXHB8OHDERgYiO7du4vjbrp27YrIyEjcvHkTf//9N6pXr64w1kFbW74zW0NDQ/xVq6+v/3meBKnUh++phoYGgKxDktkMDAyUXueH+0W2jIyMQteX27o/RVNTU2H+9PT0AtUCADo6OqhVq5bcX7Vq1fJcrrDPo7gx7BBRqVWvXj2kpKQAACpUqIC2bdti79692LdvH3r06KHUuqRSKaKjo/H48eOiKJVKCAsLC0RHR+PFixdi24MHD5CYmAgLCwsAgImJCWJjY+WWyz4tGwCqV68OHR0d3L59W2x7+/Ytnjx5UqCa/vvvP/H/MzIycOfOHfEwV4UKFZCcnCzu5wAQFhYmt7yOjo5cwCsOJaGGT2HYIaIS7/Xr1/Dy8sJff/2FsLAwREZG4vDhw1i/fj3atGkjzte7d2/s27cPDx8+RLdu3ZTahqOjIxwcHDBq1CicP38ekZGROHPmDP79918VPxsqTs7OzpBIJBg/fjzu3LmD4OBgTJgwAY6OjrCysgIANG3aFCEhIQgKCsKTJ0+wdOlS3L9/X1xH2bJl0a1bN8ybNw+XLl3C/fv3MXnyZGhoaIi9S8r4448/cPz4cTx8+BDTpk1DQkKCeAi2cePGMDAwwKJFixAREYH9+/crnG1lbm6OqKgohIaGIj4+PsdB/EXN3Nwc4eHhePToEeLj4wvV+1QUGHaIqMQrU6YMGjdujM2bN6N///7o3LkzlixZgt69e2PKlCnifM7OzqhUqRJcXV1RuXJlpbezbNkyWFpa4ueff4aHhwcWLFhQon+tkvI0NDSwcuVKGBkZoX///vj+++9Ro0YNLF68WJynWbNm+OmnnzB//nz06tULycnJCuHZx8dHPHNq4MCBsLOzg4WFhTgwWhnjxo3D2rVr0bVrV1y/fh2rVq0SxxmVL18e8+fPx7///ovOnTvj4MGDGDlypNzy7dq1Q7NmzeDl5QUnJyccOHBA+RemkPr06YM6deqgZ8+ecHJywo0bNz57DZ/Cu54TkdpITk5G8+bNMXv2bLi7uxd3OfQFSUlJQfPmzTFx4kT07t07X8tERUWhTZs2CAoKQoMGDYq4wi8bTz0nolJPJpPh9evX2LhxI4yMjNC6deviLonU3N27d/Ho0SNYW1vj7du3WLFiBQDIHValkoNhh4hKvefPn6NNmzaoUqUK5syZo3DmCFFR2LhxIx4/fgwdHR00atQI27dvlzvNnUoOHsYiIiIitcYBykRERKTWGHaIiIhIrTHsEBER5dPNmzfRoEEDDBkypLhLISUw7BAREeXT7t270b9/f1y9ehUxMTHFXQ7lE8MOERFRPiQnJ+PQoUPo27cvWrZsiX379onTEhISMG7cODRt2hTW1tZwd3fHnj17AABeXl6YNm2a3Lri4+NhaWmJixcvAgBat26N1atXw9fXF7a2tmjZsiV27Nght0x0dDR+/vlnODo6wsbGBj169MCtW7eK+FmrB4YdIiKifDh8+DDq1q2LunXrokuXLtizZ494M8wlS5bg4cOHWLduHQ4dOgQ/Pz9UqFABQNZtTA4cOCB3G4e///4blSpVQtOmTcW2gIAAWFpaIigoCJ6envDz8xPvfp6cnIz+/fsjJiYGK1euxF9//YUffviBV/jOJ4YdIiKifNi9eze6dOkCIOuWEm/fvsWVK1cAZF3rqUGDBrCyskL16tXh7OwsXtwy+2reJ06cENe1d+9e9OjRQ+5eWs2bN0e/fv1Qq1YtDB48GBUqVMDly5cBAAcOHEB8fDxWrFgBBwcH1KpVCx07doStre1nee6lHcMOERFRHh49eoTbt2+jU6dOAABtbW107NgRu3fvBgD07dsXhw4dQteuXTFv3jy5e0Pp6emJPUEAcOfOHdy/fx/du3eX24ZUKhX/X0NDA6ampoiLiwOQddf1hg0bonz58kX5NNUWLzNKRESUh927dyMjIwPNmjUT2wRBgK6uLqZMmYIWLVrg9OnTOHPmDM6fP4/vv/8e/fr1w8SJEwFkHcrq1q0boqOjsXfvXjRt2hTm5uZy2/j4yt8aGhriYTJ9ff0ifobqjT07REREn5CRkYG//voLPj4+CAoKEv/++usvVKpUSbzLuImJCbp3744FCxZg0qRJcgOMpVIpLC0tsXPnThw4cAA9e/ZUqgapVIrQ0FC8efNGlU/ti8GeHSIiok/4559/kJCQgF69eqFcuXJy09zd3bF79268fPkSjRo1wldffYW0tDT8888/sLCwkJu3d+/emDZtGgwNDeHm5qZUDR4eHli9ejWGDx+On3/+GZUqVcLdu3dRqVIljtvJB/bsEBERfcLu3bvh7OysEHQAoF27dggJCYGWlhYWLVqELl26oH///tDU1MSiRYvk5vXw8IC2tjY8PDygp6enVA26urrYuHEjKlasiCFDhqBz585Yu3YttLS0CvXcvhS8ESgREdFnEBUVBTc3N+zevRuNGjUq7nK+KDyMRUREVITS09Px5s0b/P7772jcuDGDTjHgYSwiIqIidOPGDbi6uuL27dvw9/cv7nK+SDyMRURERGqNPTtERESk1hh2iIiISK0x7BAREZFaY9ghIiIitcawQ0RERGqNYYeIiIjUGsMOERERqTWGHSIiIlJr/wc4QQY3ijZKIgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cpu_name = core.get_property(\"CPU\", \"FULL_DEVICE_NAME\")\n",
    "\n",
    "plot = sns.barplot(benchmark_dataframe, errorbar=\"sd\")\n",
    "plot.set(ylabel=\"Sentences Per Second\", title=f\"Sentence Embeddings Benchmark\\n{cpu_name}\")\n",
    "perf_ratio = benchmark_dataframe.mean() / benchmark_dataframe.mean()[0]\n",
    "plot.spines[\"right\"].set_visible(False)\n",
    "plot.spines[\"top\"].set_visible(False)\n",
    "plot.spines[\"left\"].set_visible(False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b9f566e3-1923-4f94-95c7-60fc5097c37b",
   "metadata": {},
   "source": [
    "On an Intel Core i9-10980XE CPU, the OpenVINO model processed 45% more sentences per second compared with the original PyTorch model. Using Async mode with throughput hint, we get ×3.21 (or 221%) performance boost.\n",
    "\n",
    "Here are useful links with information about the techniques used in this notebook:\n",
    "- [OpenVINO performance hints](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/high-level-performance-hints.html)\n",
    "- [OpenVINO Async API](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/general-optimizations.html#prefer-openvino-async-api)\n",
    "- [Throughput Optimizations](https://docs.openvino.ai/2024/openvino-workflow/running-inference/optimize-inference/optimizing-throughput.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "openvino_notebooks": {
   "imageUrl": "https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/cross-lingual-books-alignment/cross-lingual-books-alignment.png?raw=true",
   "tags": {
    "categories": [
     "Model Demos"
    ],
    "libraries": [],
    "other": [],
    "tasks": [
     "Feature Extraction",
     "Paraphrase Identification"
    ]
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}