class_docstr: Unit tests for OIVF. Some of these unit tests first copy the required test data objects and puts them in the tempdir created by the context manager.
class_docstr_tok: ["Unit","tests","for","OIVF",".","Some","of","these","unit","tests","first","copy","the","required","test","data","objects","and","puts","them","in","the","tempdir","created","by","the","context","manager","."]
docstr: Test the case where the shard size is not a divisor of the database size and is greater than the first file size.
docstr_tok: ["Test","the","case","where","the","shard","size","is","not","a","divisor","of","the","database","size","and","is","greater","than","the","first","file","size","."]
code: def test_index_shard_unequal_file_sizes(self) -> None:
      """
      Test the case where the shard size is not a divisor of the database size and is greater than the first file size.
      """
      with tempfile.TemporaryDirectory() as tmpdirname:
          file_sizes = [20000, 15001, 13990]
          xb_ds_size = sum(file_sizes)
          index_shard_size = 30000
          data_creator = TestDataCreator(
              tempdir=tmpdirname,
              dimension=8,
              data_type=np.float16,
              index_factory=["IVF256,PQ4"],
              training_sample=9984,
              file_sizes=file_sizes,
              nprobe=2,
              k=2,
              metric="METRIC_L2",
              index_shard_size=index_shard_size,
              evaluation_sample=100,
          )
          data_creator.create_test_data()
          test_args = data_creator.setup_cli("train_index")
          process_options_and_run_jobs(test_args)
          test_args = data_creator.setup_cli("index_shard")
          cfg = load_config(test_args.config)
          process_options_and_run_jobs(test_args)
          num_shards = xb_ds_size // index_shard_size
          if xb_ds_size % index_shard_size != 0:
              num_shards += 1
          print(f"number of shards:{num_shards}")
          for i in range(num_shards):
              index_shard_file = (
                  cfg["output"]
                  + "/my_test_data/"
                  + cfg["index"]["prod"][-1].replace(",", "_")
                  + f".shard_{i}"
              )
              self.assert_file_exists(index_shard_file)
  
file_path: faiss/demos/offline_ivf/tests/test_offline_ivf.py
imports: import numpy
  import unittest
  from utils import load_config
  import pathlib
  import tempfile
  from typing import List
  from tests.testing_utils import TestDataCreator
  from run import process_options_and_run_jobs
label_desc: Use image node_id 5 for calling the TestOIVF obj's underlying member method code with example usage: obj.test_index_shard_unequal_file_sizes() without return types