class_docstr: Data Processor for SGD dialogues.
  
  More information at https://arxiv.org/abs/1909.05855
  
  ***Downloading the dataset***
      #   git clone https://github.com/google-research-datasets/dstc8-schema-guided-dialogue.git
  
  ***Data format***
  SGD data comes with a JSON schema file and dialogue files for each dataset split. 
  
  In the following we will show an example for a service entry in the schema file.
  * service_name
  * description
  * slots
      * name
      * description
      * is_categorical
      * possible values
  * intents
      * name
      * description
      * required_slots (not used)
      * is_transactional (not used)
      * optional_slots (not used)
      * result_slots (not used)
  
  
  In the following we will show an example for a dialogue. 
  * dialogue_id
  * services
  * turns
      * frames
          * actions
              * act
              * slot
              * values
          * service
          * slots
              * exclusive_end
              * slot
              * start
          * state
              * active_intent
              * requeste_slots
              * slot_values 
      * speaker - [USER, SYSTEM]
      * utterance
docstr: Create examples for every turn in the dialogue.
  Args:
      dialog: dialogue example
      schemas: schema for all services of all datasets
      dataset_split: data split
      slot_carryover_candlist: a dictionary to keep and count the number of carry-over cases between two slots from two different services
      subsample: whether to balance postive and negative samples in the dataset
  Returns:
      examples: a list of `InputExample`s.
docstr_tok: ["Create","examples","for","every","turn","in","the","dialogue",".","Args",":","dialog",":","dialogue","example","schemas",":","schema","for","all","services","of","all","datasets","dataset_split",":","data","split","slot_carryover_candlist",":","a","dictionary","to","keep","and","count","the","number","of","carry-over","cases","between","two","slots","from","two","different","services","subsample",":","whether","to","balance","postive","and","negative","samples","in","the","dataset","Returns",":","examples",":","a","list","of","`","InputExample","`","s","."]
code: def _create_examples_from_dialog(
      self,
      dialog: dict,
      schemas: object,
      dataset_split: str,
      slot_carryover_candlist: dict,
      subsample: bool,
  ):
      """
      Create examples for every turn in the dialogue.
      Args:
          dialog: dialogue example
          schemas: schema for all services of all datasets
          dataset_split: data split
          slot_carryover_candlist: a dictionary to keep and count the number of carry-over cases between two slots from two different services
          subsample: whether to balance postive and negative samples in the dataset
      Returns:
          examples: a list of `InputExample`s.
      """
      dialog_id = dialog["dialogue_id"]
      prev_states = {}
      examples = []
      for turn_idx, turn in enumerate(dialog["turns"]):
          # Generate an example for every frame in every user turn.
          if turn["speaker"] == "USER":
              user_utterance = turn["utterance"]
              user_frames = {f["service"]: f for f in turn["frames"]}
              if self.cfg.system_utterance == "prev_turn":
                  if turn_idx > 0:
                      system_turn = dialog["turns"][turn_idx - 1]
                      system_utterance = system_turn["utterance"]
                      system_frames = {
                          f["service"]: f for f in system_turn["frames"]
                      }
                  else:
                      system_utterance = ""
                      system_frames = {}
              else:  # takes the system utterance of the next turn
                  system_turn = dialog["turns"][turn_idx + 1]
                  system_utterance = system_turn["utterance"]
                  system_frames = {
                      f["service"]: f for f in system_turn["frames"]
                  }
  
              turn_id = "{}-{}-{:02d}".format(
                  dataset_split, dialog_id, turn_idx
              )
              (
                  turn_examples,
                  prev_states,
                  slot_carryover_values,
              ) = self._create_examples_from_turn(
                  turn_id,
                  system_utterance,
                  user_utterance,
                  system_frames,
                  user_frames,
                  prev_states,
                  schemas,
                  subsample,
              )
              examples.extend(turn_examples)
  
              for value, slots_list in slot_carryover_values.items():
                  if value in ["True", "False"]:
                      continue
                  if len(slots_list) > 1:
                      for service1, slot1 in slots_list:
                          for service2, slot2 in slots_list:
                              if service1 == service2:
                                  continue
                              if service1 > service2:
                                  service1, service2 = (
                                      service2,
                                      service1,
                                  )
                                  slot1, slot2 = slot2, slot1
                              slot_carryover_candlist[
                                  (service1, slot1, service2, slot2)
                              ] += 1
      return examples
  
imports: import collections
  import json
  import os
  import pickle
  import re
  from typing import List
  from nemo.collections.nlp.data.dialogue.data_processor.data_processor import DialogueDataProcessor
  from nemo.collections.nlp.data.dialogue.input_example.input_example import DialogueInputExample
  from nemo.collections.nlp.data.dialogue.sgd.schema import Schema
  from nemo.utils import logging
  from nemo.utils.get_rank import is_global_rank_zero
label_desc: Use image node_id 10 for calling the DialogueSGDDataProcessor obj's underlying member method code with example usage: obj._create_examples_from_dialog(dialog, schemas, dataset_split, slot_carryover_candlist, subsample) and returns: examples