class_docstr: Find duplicates using CNN and/or generate CNN encodings given a single image or a directory of images.
  
  The module can be used for 2 purposes: Encoding generation and duplicate detection.
  - Encodings generation:
  To propagate an image through a Convolutional Neural Network architecture and generate encodings. The generated
  encodings can be used at a later time for deduplication. Using the method 'encode_image', the CNN encodings for a
  single image can be obtained while the 'encode_images' method can be used to get encodings for all images in a
  directory.
  
  - Duplicate detection:
  Find duplicates either using the encoding mapping generated previously using 'encode_images' or using a Path to the
  directory that contains the images that need to be deduplicated. 'find_duplicates' and 'find_duplicates_to_remove'
  methods are provided to accomplish these tasks.
class_docstr_tok: ["Find","duplicates","using","CNN","and\/or","generate","CNN","encodings","given","a","single","image","or","a","directory","of","images",".","The","module","can","be","used","for","2","purposes",":","Encoding","generation","and","duplicate","detection",".","-","Encodings","generation",":","To","propagate","an","image","through","a","Convolutional","Neural","Network","architecture","and","generate","encodings",".","The","generated","encodings","can","be","used","at","a","later","time","for","deduplication",".","Using","the","method","'encode_image","'",",","the","CNN","encodings","for","a","single","image","can","be","obtained","while","the","'encode_images","'","method","can","be","used","to","get","encodings","for","all","images","in","a","directory",".","-","Duplicate","detection",":","Find","duplicates","either","using","the","encoding","mapping","generated","previously","using","'encode_images","'","or","using","a","Path","to","the","directory","that","contains","the","images","that","need","to","be","deduplicated",".","'find_duplicates","'","and","'find_duplicates_to_remove'","methods","are","provided","to","accomplish","these","tasks","."]
docstr: Give out a list of image file names to remove based on the similarity threshold. Does not remove the mentioned
  files.
  
  Args:
      image_dir: Path to the directory containing all the images or dictionary with keys as file names
                 and values as numpy arrays which represent the CNN encoding for the key image file.
      encoding_map: Optional, used instead of image_dir, a dictionary containing mapping of filenames and
                    corresponding CNN encodings.
      min_similarity_threshold: Optional, threshold value (must be float between -1.0 and 1.0). Default is 0.9
      outfile: Optional, name of the file to save the results, must be a json. Default is None.
      recursive: Optional, find images recursively in a nested image directory structure, set to False by default.
      num_enc_workers: Optional, number of cpu cores to use for multiprocessing encoding generation (supported only on linux platform), set to 0 by default. 0 disables multiprocessing.
      num_sim_workers: Optional, number of cpu cores to use for multiprocessing similarity computation, set to number of CPUs in the system by default. 0 disables multiprocessing.
  
  Returns:
      duplicates: List of image file names that should be removed.
  
  Example:
  ```
  from imagededup.methods import CNN
  myencoder = CNN()
  duplicates = myencoder.find_duplicates_to_remove(image_dir='path/to/images/directory'),
  min_similarity_threshold=0.85)
  
  OR
  
  from imagededup.methods import CNN
  myencoder = CNN()
  duplicates = myencoder.find_duplicates_to_remove(encoding_map=<mapping filename to cnn encodings>,
  min_similarity_threshold=0.85, outfile='results.json')
  ```
docstr_tok: ["Give","out","a","list","of","image","file","names","to","remove","based","on","the","similarity","threshold",".","Does","not","remove","the","mentioned","files",".","Args",":","image_dir",":","Path","to","the","directory","containing","all","the","images","or","dictionary","with","keys","as","file","names","and","values","as","numpy","arrays","which","represent","the","CNN","encoding","for","the","key","image","file",".","encoding_map",":","Optional",",","used","instead","of","image_dir",",","a","dictionary","containing","mapping","of","filenames","and","corresponding","CNN","encodings",".","min_similarity_threshold",":","Optional",",","threshold","value","(","must","be","float","between","-1.0","and","1.0",")",".","Default","is","0.9","outfile",":","Optional",",","name","of","the","file","to","save","the","results",",","must","be","a","json",".","Default","is","None",".","recursive",":","Optional",",","find","images","recursively","in","a","nested","image","directory","structure",",","set","to","False","by","default",".","num_enc_workers",":","Optional",",","number","of","cpu","cores","to","use","for","multiprocessing","encoding","generation","(","supported","only","on","linux","platform",")",",","set","to","0","by","default",".","0","disables","multiprocessing",".","num_sim_workers",":","Optional",",","number","of","cpu","cores","to","use","for","multiprocessing","similarity","computation",",","set","to","number","of","CPUs","in","the","system","by","default",".","0","disables","multiprocessing",".","Returns",":","duplicates",":","List","of","image","file","names","that","should","be","removed",".","Example",":","``","`","from","imagededup.methods","import","CNN","myencoder","=","CNN","(",")","duplicates","=","myencoder.find_duplicates_to_remove","(","image_dir='path\/to\/images\/directory","'",")",",","min_similarity_threshold=0.85",")","OR","from","imagededup.methods","import","CNN","myencoder","=","CNN","(",")","duplicates","=","myencoder.find_duplicates_to_remove","(","encoding_map=","<","mapping","filename","to","cnn","encodings",">",",","min_similarity_threshold=0.85",",","outfile='results.json","'",")","``","`"]
code: def find_duplicates_to_remove(
      self,
      image_dir: PurePath = None,
      encoding_map: Dict[str, np.ndarray] = None,
      min_similarity_threshold: float = 0.9,
      outfile: Optional[str] = None,
      recursive: Optional[bool] = False,
      num_enc_workers: int = 0,
      num_sim_workers: int = cpu_count(),
  ) -> List:
      """
      Give out a list of image file names to remove based on the similarity threshold. Does not remove the mentioned
      files.
  
      Args:
          image_dir: Path to the directory containing all the images or dictionary with keys as file names
                     and values as numpy arrays which represent the CNN encoding for the key image file.
          encoding_map: Optional, used instead of image_dir, a dictionary containing mapping of filenames and
                        corresponding CNN encodings.
          min_similarity_threshold: Optional, threshold value (must be float between -1.0 and 1.0). Default is 0.9
          outfile: Optional, name of the file to save the results, must be a json. Default is None.
          recursive: Optional, find images recursively in a nested image directory structure, set to False by default.
          num_enc_workers: Optional, number of cpu cores to use for multiprocessing encoding generation (supported only on linux platform), set to 0 by default. 0 disables multiprocessing.
          num_sim_workers: Optional, number of cpu cores to use for multiprocessing similarity computation, set to number of CPUs in the system by default. 0 disables multiprocessing.
  
      Returns:
          duplicates: List of image file names that should be removed.
  
      Example:
      ```
      from imagededup.methods import CNN
      myencoder = CNN()
      duplicates = myencoder.find_duplicates_to_remove(image_dir='path/to/images/directory'),
      min_similarity_threshold=0.85)
  
      OR
  
      from imagededup.methods import CNN
      myencoder = CNN()
      duplicates = myencoder.find_duplicates_to_remove(encoding_map=<mapping filename to cnn encodings>,
      min_similarity_threshold=0.85, outfile='results.json')
      ```
      """
      if image_dir or encoding_map:
          duplicates = self.find_duplicates(
              image_dir=image_dir,
              encoding_map=encoding_map,
              min_similarity_threshold=min_similarity_threshold,
              scores=False,
              recursive=recursive,
              num_enc_workers=num_enc_workers,
              num_sim_workers=num_sim_workers,
          )
  
      files_to_remove = get_files_to_remove(duplicates)
  
      if outfile:
          save_json(files_to_remove, outfile)
  
      return files_to_remove
  
imports: from pathlib import Path, PurePath
  import sys
  from typing import Dict, List, Optional, Union
  import warnings
  from multiprocessing import cpu_count
  import numpy
  from PIL import Image
  import torch
  from imagededup.handlers.search.retrieval import get_cosine_similarity
  from imagededup.utils.data_generator import img_dataloader
  from imagededup.utils.models import CustomModel, MobilenetV3, DEFAULT_MODEL_NAME
  from imagededup.utils.general_utils import generate_relative_names, get_files_to_remove, save_json
  from imagededup.utils.image_utils import expand_image_array_cnn, load_image, preprocess_image
  from imagededup.utils.logger import return_logger
label_desc: Use image node_id 12 for calling the CNN obj's underlying member method code with example usage: obj.find_duplicates_to_remove(image_dir, encoding_map, min_similarity_threshold, outfile, recursive, num_enc_workers, num_sim_workers) and returns: files_to_remove