class_docstr: This measurement returns the duplicate strings contained in the input(s).
docstr: Returns the duplicates contained in the input data and the number of times they are repeated.
code: def _compute(self, data, list_duplicates=False):
      """Returns the duplicates contained in the input data and the number of times they are repeated."""
      if list_duplicates == True:
          logger.warning(
              "This functionality can be memory-intensive for large datasets!"
          )
          n_dedup = len(set([get_hash(d) for d in data]))
          c = Counter(data)
          duplicates = {k: v for k, v in c.items() if v > 1}
          return {
              "duplicate_fraction": 1 - (n_dedup / len(data)),
              "duplicates_dict": duplicates,
          }
      else:
          n_dedup = len(set([get_hash(d) for d in data]))
          return {"duplicate_fraction": 1 - (n_dedup / len(data))}
  
file_path: evaluate/measurements/text_duplicates/text_duplicates.py
imports: import hashlib
  from collections import Counter
  import datasets
  import evaluate
label_desc: Use image node_id 2 for calling the TextDuplicates obj's underlying member method code with example usage: obj._compute(data, list_duplicates) and returns: dict, dict