code: def ngrams_for_evaluation(sequence, max_n, predict_first=False):
      """
      Produce each token with the appropriate context.
  
      The function produces as large N-grams as possible, so growing from
      unigrams/bigrams to max_n.
  
      E.G. when your model is a trigram model, you'll still only have one token
      of context (the start of sentence) for the first token.
  
      In general this is useful when evaluating an N-gram model.
  
      Arguments
      ---------
      sequence : iterator
          The sequence to produce tokens and context from.
      max_n : int
          The maximum N-gram length to produce.
      predict_first : bool
          To produce the first token in the sequence to predict (without
          context) or not. Essentially this should be False when the start of
          sentence symbol is the first in the sequence.
  
      Yields
      ------
      Any
          The token to predict
      tuple
          The context to predict conditional on.
  
      Example
      -------
      >>> for token, context in ngrams_for_evaluation("Brain", 3, True):
      ...     print(f"p( {token} |{' ' if context else ''}{' '.join(context)} )")
      p( B | )
      p( r | B )
      p( a | B r )
      p( i | r a )
      p( n | a i )
      """
      if max_n <= 0:
          raise ValueError("Max N must be >=1")
      iterator = iter(sequence)
      history = []
      if not predict_first:
          history.append(next(iterator))
      for token in iterator:
          if len(history) == max_n:
              del history[0]
          yield token, tuple(history)
          history.append(token)
      return
  
imports: import itertools
label_desc: Use image node_id 3 for calling a global function with example usage: ngrams_for_evaluation(sequence, max_n, predict_first) without return types