class_docstr: Class to build and search annoy indices.
  
  TODO(b/162421415): Split the managing of indices and the index logic.
  
  Upon instantiation of this class, it will first create empty indices for
  every model/embedding/dataset triple. It will then search for existing indices
  in the data directory and fill those. If the flag `initialize_new_indices` is
  set, it will fill the remaining indices by iterating over the data.
  During the saving process, both the index and a mapping from index-row to
  example are saved. These are used during the nearest neighbor lookup to return
  the closest example.
  
  Attributes:
     models: specification akin to the LIT server.
     datasets: specification akin to the LIT server.
     data_dir: path for (de-)serialization of indices.
     initialize_new_indices: whether to build new indices or simply load
       existing ones.
code: def __init__(
      self,
      models: Mapping[str, lit_model.Model],
      datasets: Mapping[str, lit_data.IndexedDataset],
      data_dir: Optional[str],
      initialize_new_indices: Optional[bool] = False,
  ):
      self.datasets = datasets
      self._indices = collections.OrderedDict()
      self._example_lookup = collections.defaultdict(dict)
      # Indicator whether to build new indices. If False, only load existing ones.
      self._initialize_new_indices = initialize_new_indices
      # Ensure directory to save indices exists.
      if not os.path.isdir(data_dir):
          os.mkdir(data_dir)
      self._data_dir = data_dir
      self._models = models
  
      # Create/Load indices.
      for model_name, model in self._models.items():
          compatible_datasets = [
              dataset_name
              for dataset_name, dataset in self.datasets.items()
              if model.is_compatible_with_dataset(dataset)
          ]
          for dataset in compatible_datasets:
              self._create_empty_indices(model_name, dataset)
              self._fill_indices(model_name, dataset)
  
      # Update cache with all indices.
      self._save_lookups()
  
imports: import collections
  from collections.abc import Mapping
  import os
  import pickle
  from typing import Optional
  from absl import logging
  import annoy
  from lit_nlp.api import dataset
  from lit_nlp.api import model
  from lit_nlp.api import types
  from lit_nlp.lib import utils
label_desc: Use image node_id 1 to create a new Indexer object from inherited base classes: object with example: obj = Indexer(models, datasets, data_dir, initialize_new_indices)