class_docstr: Checks for frequent substrings in the dataset.
  
  Substrings of varying lengths (n-grams) are extracted from the dataset text samples.
  The frequencies of these n-grams are calculated and only substrings exceeding a defined minimum length are retained.
  The substrings are then sorted by their frequencies and the most frequent substrings are identified.
  Finally, the substrings with the highest frequency and those surpassing a significance level are displayed.
  
  Parameters
  ----------
  n_to_show : int, default: 5
      Number of most frequent substrings to show.
  n_samples : int, default: 10_000
      Number of samples to use for this check.
  random_state : int, default: 42
      Random seed for all check internals.
  n_sentences : int, default: 5
      The number of sentences to extract from the beginning and end of the text content.
  min_ngram_length: int, default: 4
      Minimum amount of words for a substring to be considered a frequent substring.
  min_substring_ratio: float, default: 0.05
      Minimum frequency required for a substring to be considered "frequent".
  significant_substring_ratio: float, default: 0.3
      Frequency above which samples are considered significant. Substrings
      meeting or exceeding this ratio will always be returned, regardless of
      other parameters and conditions.
  frequency_margin: float, default: 0.02
      Defines the tolerance level for selecting longer overlapping substrings. If a longer substring
      has a frequency that's less than a shorter overlapping substring but the difference is within
      the specified frequency_margin, the longer substring is still preferred.
  min_relative_change : float, optional, default=0.05
      Defines the threshold for relative change. If the computed relative
      change falls below this specified threshold, it is
      considered insignificant and is thus set to zero.
class_docstr_tok: ["Checks","for","frequent","substrings","in","the","dataset",".","Substrings","of","varying","lengths","(","n-grams",")","are","extracted","from","the","dataset","text","samples",".","The","frequencies","of","these","n-grams","are","calculated","and","only","substrings","exceeding","a","defined","minimum","length","are","retained",".","The","substrings","are","then","sorted","by","their","frequencies","and","the","most","frequent","substrings","are","identified",".","Finally",",","the","substrings","with","the","highest","frequency","and","those","surpassing","a","significance","level","are","displayed",".","Parameters","--","--","--","--","--","n_to_show",":","int",",","default",":","5","Number","of","most","frequent","substrings","to","show",".","n_samples",":","int",",","default",":","10_000","Number","of","samples","to","use","for","this","check",".","random_state",":","int",",","default",":","42","Random","seed","for","all","check","internals",".","n_sentences",":","int",",","default",":","5","The","number","of","sentences","to","extract","from","the","beginning","and","end","of","the","text","content",".","min_ngram_length",":","int",",","default",":","4","Minimum","amount","of","words","for","a","substring","to","be","considered","a","frequent","substring",".","min_substring_ratio",":","float",",","default",":","0.05","Minimum","frequency","required","for","a","substring","to","be","considered","``","frequent","''",".","significant_substring_ratio",":","float",",","default",":","0.3","Frequency","above","which","samples","are","considered","significant",".","Substrings","meeting","or","exceeding","this","ratio","will","always","be","returned",",","regardless","of","other","parameters","and","conditions",".","frequency_margin",":","float",",","default",":","0.02","Defines","the","tolerance","level","for","selecting","longer","overlapping","substrings",".","If","a","longer","substring","has","a","frequency","that","'s","less","than","a","shorter","overlapping","substring","but","the","difference","is","within","the","specified","frequency_margin",",","the","longer","substring","is","still","preferred",".","min_relative_change",":","float",",","optional",",","default=0.05","Defines","the","threshold","for","relative","change",".","If","the","computed","relative","change","falls","below","this","specified","threshold",",","it","is","considered","insignificant","and","is","thus","set","to","zero","."]
docstr: Remove overlapping n-grams from the results based on their lengths and frequencies.
  
  Given a dictionary of n-grams and their respective frequencies, this method
  filters out overlapping n-grams, preserving only the longest n-gram, unless
  a shorter n-gram has a frequency that exceeds the longer n-gram's frequency
  by a margin greater than `self.frequency_margin`.
  
  Parameters:
  -----------
  results : dict
      A dictionary where keys are n-grams and values are another dictionary
      containing frequency ('freq'), original indexes ('original_indexes'),
      and filtered indexes ('filtered_indexes') of the n-gram.
  
  Returns:
  --------
  dict
      A filtered dictionary where keys are non-overlapping n-grams and values are
      details about the n-grams similar to the input dictionary.
  
  Notes:
  ------
  The method employs a nested loop approach, comparing each n-gram with every other
  n-gram to identify and eliminate overlapping n-grams based on length and frequency criteria.
docstr_tok: ["Remove","overlapping","n-grams","from","the","results","based","on","their","lengths","and","frequencies",".","Given","a","dictionary","of","n-grams","and","their","respective","frequencies",",","this","method","filters","out","overlapping","n-grams",",","preserving","only","the","longest","n-gram",",","unless","a","shorter","n-gram","has","a","frequency","that","exceeds","the","longer","n-gram","'s","frequency","by","a","margin","greater","than","`","self.frequency_margin","`",".","Parameters",":","--","--","--","--","--","-","results",":","dict","A","dictionary","where","keys","are","n-grams","and","values","are","another","dictionary","containing","frequency","(","'freq","'",")",",","original","indexes","(","'original_indexes","'",")",",","and","filtered","indexes","(","'filtered_indexes","'",")","of","the","n-gram",".","Returns",":","--","--","--","--","dict","A","filtered","dictionary","where","keys","are","non-overlapping","n-grams","and","values","are","details","about","the","n-grams","similar","to","the","input","dictionary",".","Notes",":","--","--","--","The","method","employs","a","nested","loop","approach",",","comparing","each","n-gram","with","every","other","n-gram","to","identify","and","eliminate","overlapping","n-grams","based","on","length","and","frequency","criteria","."]
code: def _eliminate_overlapping_substrings(self, results):
      """
      Remove overlapping n-grams from the results based on their lengths and frequencies.
  
      Given a dictionary of n-grams and their respective frequencies, this method
      filters out overlapping n-grams, preserving only the longest n-gram, unless
      a shorter n-gram has a frequency that exceeds the longer n-gram's frequency
      by a margin greater than `self.frequency_margin`.
  
      Parameters:
      -----------
      results : dict
          A dictionary where keys are n-grams and values are another dictionary
          containing frequency ('freq'), original indexes ('original_indexes'),
          and filtered indexes ('filtered_indexes') of the n-gram.
  
      Returns:
      --------
      dict
          A filtered dictionary where keys are non-overlapping n-grams and values are
          details about the n-grams similar to the input dictionary.
  
      Notes:
      ------
      The method employs a nested loop approach, comparing each n-gram with every other
      n-gram to identify and eliminate overlapping n-grams based on length and frequency criteria.
      """
      ngram_strings = list(results.keys())
      ngram_strings.sort(key=len, reverse=True)
      for long_ngram in ngram_strings:
          if long_ngram not in results:
              continue
          final_key = long_ngram
          final_freq = results[long_ngram]["freq"]
          for ngram in ngram_strings:
              if ngram not in results:
                  continue
              if ngram in final_key:
                  if ngram == final_key:
                      continue
                  ngram_freq = results[ngram]["freq"]
                  if ngram_freq - final_freq <= self.frequency_margin:
                      del results[ngram]
                  else:
                      del results[final_key]
                      final_key = ngram
                      final_freq = ngram_freq
      return results
  
imports: import re
  from collections import defaultdict
  from typing import Dict
  import pandas
  from deepchecks.core import CheckResult, ConditionCategory, ConditionResult
  from deepchecks.core.errors import DeepchecksValueError
  from deepchecks.nlp import Context, SingleDatasetCheck
  from deepchecks.utils.strings import format_percent
label_desc: Use image node_id 7 for calling the FrequentSubstrings obj's underlying member method code with example usage: obj._eliminate_overlapping_substrings(results) and returns: results