class_docstr: Compressed Indexer.
  
  If a FAISS index factory is specified, we build that.
  (see https://github.com/facebookresearch/faiss/wiki/The-index-factory)
  
  The default is IVF4096_HNSW128,PQ128; this also translates directly to the
  default when index_factory is specified as ''.
  
  See https://github.com/facebookresearch/faiss/wiki/Faiss-indexes#cell-probe-methods-indexivf-indexes
  for more details.
docstr: Initialize IVFPQ FAISS Indexer.
  
  The IVFPQ Indexer is a great way to reduce memory footprint of dense embeddings.
docstr_tok: ["Initialize","IVFPQ","FAISS","Indexer",".","The","IVFPQ","Indexer","is","a","great","way","to","reduce","memory","footprint","of","dense","embeddings","."]
code: def __init__(self, opt: Opt):
      """
      Initialize IVFPQ FAISS Indexer.
  
      The IVFPQ Indexer is a great way to reduce memory footprint of dense embeddings.
      """
      super().__init__(opt)
      self.dim = opt["retriever_embedding_size"]
      self.use_gpu_train = (
          not opt["no_cuda"]
          and torch.cuda.is_available()
          and opt["compressed_indexer_gpu_train"]
      )
      self.hnsw_ef_search = opt["hnsw_ef_search"]
  
      self.index_factory = opt["compressed_indexer_factory"]
      if self.index_factory:
          logging.warning(
              f"Creating Index from Index Factory: {self.index_factory}"
          )
          self.is_ivf_index = "IVF" in self.index_factory
          self.index = self.faiss.index_factory(
              self.dim,
              self.index_factory,
              self.faiss.METRIC_INNER_PRODUCT,
          )
      else:
          self.is_ivf_index = True
          quantizer = self.faiss.IndexHNSWFlat(
              self.dim,
              opt["hnsw_indexer_store_n"],
              self.faiss.METRIC_INNER_PRODUCT,
          )
          quantizer.hnsw.efConstruction = opt["hnsw_ef_construction"]
          quantizer.hnsw.efSearch = opt["hnsw_ef_search"]
          ivf_index = self.faiss.IndexIVFPQ(
              quantizer,
              self.dim,
              4096,
              128,
              8,
              self.faiss.METRIC_INNER_PRODUCT,
          )
          ivf_index.nprobe = opt["compressed_indexer_nprobe"]
          self.index = ivf_index
  
      if self.is_ivf_index:
          self.index_ivf = self.faiss.extract_index_ivf(self.index)
          self.index_ivf.metric_type = self.faiss.METRIC_INNER_PRODUCT
          self.nlist = self.index_ivf.nlist
          self.index_ivf.verbose = True
          self.downcast_quantizer = self.faiss.downcast_index(
              self.index_ivf.quantizer
          )
          self.downcast_quantizer.verbose = True
          self.downcast_quantizer.metric_type = (
              self.faiss.METRIC_INNER_PRODUCT
          )
          if hasattr(self.downcast_quantizer, "hnsw"):
              self.downcast_quantizer.hnsw.efSearch = opt[
                  "hnsw_ef_search"
              ]
              self.downcast_quantizer.hnsw.efConstruction = opt[
                  "hnsw_ef_construction"
              ]
              self.downcast_quantizer.hnsw.metric_type = (
                  self.faiss.METRIC_INNER_PRODUCT
              )
  
          self.setup_gpu_train()
          self.index.nprobe = opt["compressed_indexer_nprobe"]
  
      self.nprobe = opt["compressed_indexer_nprobe"]
      self.span = 5  # arbitrarily chosen, from prior evidence
      self.random = random.Random(42)
  
imports: from parlai.core.build_data import modelzoo_path
  from parlai.core.opt import Opt
  import parlai.utils.logging
  from abc import ABC, abstractmethod
  import math
  import numpy
  import os
  import random
  import time
  import torch
  import torch.cuda
  from typing import List, Tuple, Optional
  from parlai.agents.rag.args import WIKIPEDIA_COMPRESSED_INDEX, WIKIPEDIA_EXACT_INDEX
label_desc: Use image node_id 1 to create a new CompressedIndexer object from inherited base classes: BaseIndexer with example: obj = CompressedIndexer(opt)