class_docstr: Parent class of gradio client
  To handle automatically refreshing client if detect gradio server changed
class_docstr_tok: ["Parent","class","of","gradio","client","To","handle","automatically","refreshing","client","if","detect","gradio","server","changed"]
docstr:         Query or Summarize or Extract using h2oGPT
          Args:
              instruction: Query for LLM chat.  Used for similarity search
  
              For query, prompt template is:
                "{pre_prompt_query}
                  """
                  {content}
                  """
                  {prompt_query}{instruction}"
               If added to summarization, prompt template is
                "{pre_prompt_summary}
                  """
                  {content}
                  """
                  Focusing on {instruction}, {prompt_summary}"
              text: textual content or list of such contents
              file: a local file to upload or files to upload
              url: a url to give or urls to use
              embed: whether to embed content uploaded
  
              langchain_mode: "LLM" to talk to LLM with no docs, "MyData" for personal docs, "UserData" for shared docs, etc.
              langchain_action: Action to take, "Query" or "Summarize" or "Extract"
              langchain_agents: Which agents to use, if any
              top_k_docs: number of document parts.
                          When doing query, number of chunks
                          When doing summarization, not related to vectorDB chunks that are not used
                          E.g. if PDF, then number of pages
              chunk: whether to chunk sources for document Q/A
              chunk_size: Size in characters of chunks
              document_choice: Which documents ("All" means all) -- need to use upload_api API call to get server's name if want to select
              document_subset: Type of query, see src/gen.py
              document_source_substrings: See gen.py
              document_source_substrings_op: See gen.py
              document_content_substrings: See gen.py
              document_content_substrings_op: See gen.py
  
              system_prompt: pass system prompt to models that support it.
                If 'auto' or None, then use automatic version
                If '', then use no system prompt (default)
              pre_prompt_query: Prompt that comes before document part
              prompt_query: Prompt that comes after document part
              pre_prompt_summary: Prompt that comes before document part
                 None makes h2oGPT internally use its defaults
                 E.g. "In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text"
              prompt_summary: Prompt that comes after document part
                None makes h2oGPT internally use its defaults
                E.g. "Using only the text above, write a condensed and concise summary of key results (preferably as bullet points):
  "
              i.e. for some internal document part fstring, the template looks like:
                  template = "%s
                  """
                  %s
                  """
                  %s" % (pre_prompt_summary, fstring, prompt_summary)
              hyde_llm_prompt: hyde prompt for first step when using LLM
              h2ogpt_key: Access Key to h2oGPT server (if not already set in client at init time)
              model: base_model name or integer index of model_lock on h2oGPT server
                              None results in use of first (0th index) model in server
                     to get list of models do client.list_models()
              pre_prompt_extraction: Same as pre_prompt_summary but for when doing extraction
              prompt_extraction: Same as prompt_summary but for when doing extraction
              do_sample: see src/gen.py
              temperature: see src/gen.py
              top_p: see src/gen.py
              top_k: see src/gen.py
              repetition_penalty: see src/gen.py
              penalty_alpha: see src/gen.py
              max_new_tokens: see src/gen.py
              min_max_new_tokens: see src/gen.py
              max_input_tokens: see src/gen.py
              max_total_input_tokens: see src/gen.py
  
              stream_output: Whether to stream output
              do_sample: whether to sample
              max_time: how long to take
  
              add_search_to_context: Whether to do web search and add results to context
              chat_conversation: List of tuples for (human, bot) conversation that will be pre-appended to an (instruction, None) case for a query
              text_context_list: List of strings to add to context for non-database version of document Q/A for faster handling via API etc.
                 Forces LangChain code path and uses as many entries in list as possible given max_seq_len, with first assumed to be most relevant and to go near prompt.
              docs_ordering_type: By default uses 'reverse_ucurve_sort' for optimal retrieval
              max_input_tokens: Max input tokens to place into model context for each LLM call
                                       -1 means auto, fully fill context for query, and fill by original document chunk for summarization
                                       >=0 means use that to limit context filling to that many tokens
              max_total_input_tokens: like max_input_tokens but instead of per LLM call, applies across all LLM calls for single summarization/extraction action
              max_new_tokens: Maximum new tokens
              min_max_new_tokens: minimum value for max_new_tokens when auto-adjusting for content of prompt, docs, etc.
  
              docs_token_handling: 'chunk' means fill context with top_k_docs (limited by max_input_tokens or model_max_len) chunks for query
                                                                               or top_k_docs original document chunks summarization
                                          None or 'split_or_merge' means same as 'chunk' for query, while for summarization merges documents to fill up to max_input_tokens or model_max_len tokens
              docs_joiner: string to join lists of text when doing split_or_merge.  None means '
  
  '
              hyde_level: 0-3 for HYDE.
                          0 uses just query to find similarity with docs
                          1 uses query + pure LLM response to find similarity with docs
                          2: uses query + LLM response using docs to find similarity with docs
                          3+: etc.
              hyde_template: see src/gen.py
              hyde_show_only_final: see src/gen.py
              doc_json_mode: see src/gen.py
  
              asserts: whether to do asserts to ensure handling is correct
  
          Returns: summary/answer: str or extraction List[str]
  
          
docstr_tok: ["Query","or","Summarize","or","Extract","using","h2oGPT","Args",":","instruction",":","Query","for","LLM","chat",".","Used","for","similarity","search","For","query",",","prompt","template","is",":","``","{","pre_prompt_query","}","``","''","''","{","content","}","``","''","''","{","prompt_query","}","{","instruction","}","''","If","added","to","summarization",",","prompt","template","is","``","{","pre_prompt_summary","}","``","''","''","{","content","}","``","''","''","Focusing","on","{","instruction","}",",","{","prompt_summary","}","''","text",":","textual","content","or","list","of","such","contents","file",":","a","local","file","to","upload","or","files","to","upload","url",":","a","url","to","give","or","urls","to","use","embed",":","whether","to","embed","content","uploaded","langchain_mode",":","``","LLM","''","to","talk","to","LLM","with","no","docs",",","``","MyData","''","for","personal","docs",",","``","UserData","''","for","shared","docs",",","etc",".","langchain_action",":","Action","to","take",",","``","Query","''","or","``","Summarize","''","or","``","Extract","''","langchain_agents",":","Which","agents","to","use",",","if","any","top_k_docs",":","number","of","document","parts",".","When","doing","query",",","number","of","chunks","When","doing","summarization",",","not","related","to","vectorDB","chunks","that","are","not","used","E.g",".","if","PDF",",","then","number","of","pages","chunk",":","whether","to","chunk","sources","for","document","Q\/A","chunk_size",":","Size","in","characters","of","chunks","document_choice",":","Which","documents","(","``","All","''","means","all",")","--","need","to","use","upload_api","API","call","to","get","server","'s","name","if","want","to","select","document_subset",":","Type","of","query",",","see","src\/gen.py","document_source_substrings",":","See","gen.py","document_source_substrings_op",":","See","gen.py","document_content_substrings",":","See","gen.py","document_content_substrings_op",":","See","gen.py","system_prompt",":","pass","system","prompt","to","models","that","support","it",".","If","'auto","'","or","None",",","then","use","automatic","version","If","``",",","then","use","no","system","prompt","(","default",")","pre_prompt_query",":","Prompt","that","comes","before","document","part","prompt_query",":","Prompt","that","comes","after","document","part","pre_prompt_summary",":","Prompt","that","comes","before","document","part","None","makes","h2oGPT","internally","use","its","defaults","E.g",".","``","In","order","to","write","a","concise","single-paragraph","or","bulleted","list","summary",",","pay","attention","to","the","following","text","''","prompt_summary",":","Prompt","that","comes","after","document","part","None","makes","h2oGPT","internally","use","its","defaults","E.g",".","``","Using","only","the","text","above",",","write","a","condensed","and","concise","summary","of","key","results","(","preferably","as","bullet","points",")",":","''","i.e",".","for","some","internal","document","part","fstring",",","the","template","looks","like",":","template","=","``","%","s","``","''","''","%","s","``","''","''","%","s","''","%","(","pre_prompt_summary",",","fstring",",","prompt_summary",")","hyde_llm_prompt",":","hyde","prompt","for","first","step","when","using","LLM","h2ogpt_key",":","Access","Key","to","h2oGPT","server","(","if","not","already","set","in","client","at","init","time",")","model",":","base_model","name","or","integer","index","of","model_lock","on","h2oGPT","server","None","results","in","use","of","first","(","0th","index",")","model","in","server","to","get","list","of","models","do","client.list_models","(",")","pre_prompt_extraction",":","Same","as","pre_prompt_summary","but","for","when","doing","extraction","prompt_extraction",":","Same","as","prompt_summary","but","for","when","doing","extraction","do_sample",":","see","src\/gen.py","temperature",":","see","src\/gen.py","top_p",":","see","src\/gen.py","top_k",":","see","src\/gen.py","repetition_penalty",":","see","src\/gen.py","penalty_alpha",":","see","src\/gen.py","max_new_tokens",":","see","src\/gen.py","min_max_new_tokens",":","see","src\/gen.py","max_input_tokens",":","see","src\/gen.py","max_total_input_tokens",":","see","src\/gen.py","stream_output",":","Whether","to","stream","output","do_sample",":","whether","to","sample","max_time",":","how","long","to","take","add_search_to_context",":","Whether","to","do","web","search","and","add","results","to","context","chat_conversation",":","List","of","tuples","for","(","human",",","bot",")","conversation","that","will","be","pre-appended","to","an","(","instruction",",","None",")","case","for","a","query","text_context_list",":","List","of","strings","to","add","to","context","for","non-database","version","of","document","Q\/A","for","faster","handling","via","API","etc",".","Forces","LangChain","code","path","and","uses","as","many","entries","in","list","as","possible","given","max_seq_len",",","with","first","assumed","to","be","most","relevant","and","to","go","near","prompt",".","docs_ordering_type",":","By","default","uses","'reverse_ucurve_sort","'","for","optimal","retrieval","max_input_tokens",":","Max","input","tokens","to","place","into","model","context","for","each","LLM","call","-1","means","auto",",","fully","fill","context","for","query",",","and","fill","by","original","document","chunk","for","summarization",">","=0","means","use","that","to","limit","context","filling","to","that","many","tokens","max_total_input_tokens",":","like","max_input_tokens","but","instead","of","per","LLM","call",",","applies","across","all","LLM","calls","for","single","summarization\/extraction","action","max_new_tokens",":","Maximum","new","tokens","min_max_new_tokens",":","minimum","value","for","max_new_tokens","when","auto-adjusting","for","content","of","prompt",",","docs",",","etc",".","docs_token_handling",":","'chunk","'","means","fill","context","with","top_k_docs","(","limited","by","max_input_tokens","or","model_max_len",")","chunks","for","query","or","top_k_docs","original","document","chunks","summarization","None","or","'split_or_merge","'","means","same","as","'chunk","'","for","query",",","while","for","summarization","merges","documents","to","fill","up","to","max_input_tokens","or","model_max_len","tokens","docs_joiner",":","string","to","join","lists","of","text","when","doing","split_or_merge",".","None","means","'","'","hyde_level",":","0-3","for","HYDE",".","0","uses","just","query","to","find","similarity","with","docs","1","uses","query","+","pure","LLM","response","to","find","similarity","with","docs","2",":","uses","query","+","LLM","response","using","docs","to","find","similarity","with","docs","3+",":","etc",".","hyde_template",":","see","src\/gen.py","hyde_show_only_final",":","see","src\/gen.py","doc_json_mode",":","see","src\/gen.py","asserts",":","whether","to","do","asserts","to","ensure","handling","is","correct","Returns",":","summary\/answer",":","str","or","extraction","List","[","str","]"]
code: def query_or_summarize_or_extract(
      self,
      h2ogpt_key: str = None,
      instruction: str = "",
      text: list[str] | str | None = None,
      file: list[str] | str | None = None,
      url: list[str] | str | None = None,
      embed: bool = True,
      chunk: bool = True,
      chunk_size: int = 512,
      langchain_mode: str = None,
      langchain_action: str | None = None,
      langchain_agents: List[str] = [],
      top_k_docs: int = 10,
      document_choice: Union[str, List[str]] = "All",
      document_subset: str = "Relevant",
      document_source_substrings: Union[str, List[str]] = [],
      document_source_substrings_op: str = "and",
      document_content_substrings: Union[str, List[str]] = [],
      document_content_substrings_op: str = "and",
      system_prompt: str | None = "",
      pre_prompt_query: str | None = pre_prompt_query0,
      prompt_query: str | None = prompt_query0,
      pre_prompt_summary: str | None = pre_prompt_summary0,
      prompt_summary: str | None = prompt_summary0,
      pre_prompt_extraction: str | None = pre_prompt_extraction0,
      prompt_extraction: str | None = prompt_extraction0,
      hyde_llm_prompt: str | None = hyde_llm_prompt0,
      model: str | int | None = None,
      stream_output: bool = False,
      do_sample: bool = False,
      temperature: float = 0.0,
      top_p: float = 0.75,
      top_k: int = 40,
      repetition_penalty: float = 1.07,
      penalty_alpha: float = 0.0,
      max_time: int = 360,
      max_new_tokens: int = 1024,
      add_search_to_context: bool = False,
      chat_conversation: list[tuple[str, str]] | None = None,
      text_context_list: list[str] | None = None,
      docs_ordering_type: str | None = None,
      min_max_new_tokens: int = 512,
      max_input_tokens: int = -1,
      max_total_input_tokens: int = -1,
      docs_token_handling: str = "split_or_merge",
      docs_joiner: str = "\n\n",
      hyde_level: int = 0,
      hyde_template: str = None,
      hyde_show_only_final: bool = True,
      doc_json_mode: bool = False,
      asserts: bool = False,
  ) -> Generator[tuple[str | list[str], list[str]], None, None]:
      """
      Query or Summarize or Extract using h2oGPT
      Args:
          instruction: Query for LLM chat.  Used for similarity search
  
          For query, prompt template is:
            "{pre_prompt_query}
              \"\"\"
              {content}
              \"\"\"
              {prompt_query}{instruction}"
           If added to summarization, prompt template is
            "{pre_prompt_summary}
              \"\"\"
              {content}
              \"\"\"
              Focusing on {instruction}, {prompt_summary}"
          text: textual content or list of such contents
          file: a local file to upload or files to upload
          url: a url to give or urls to use
          embed: whether to embed content uploaded
  
          langchain_mode: "LLM" to talk to LLM with no docs, "MyData" for personal docs, "UserData" for shared docs, etc.
          langchain_action: Action to take, "Query" or "Summarize" or "Extract"
          langchain_agents: Which agents to use, if any
          top_k_docs: number of document parts.
                      When doing query, number of chunks
                      When doing summarization, not related to vectorDB chunks that are not used
                      E.g. if PDF, then number of pages
          chunk: whether to chunk sources for document Q/A
          chunk_size: Size in characters of chunks
          document_choice: Which documents ("All" means all) -- need to use upload_api API call to get server's name if want to select
          document_subset: Type of query, see src/gen.py
          document_source_substrings: See gen.py
          document_source_substrings_op: See gen.py
          document_content_substrings: See gen.py
          document_content_substrings_op: See gen.py
  
          system_prompt: pass system prompt to models that support it.
            If 'auto' or None, then use automatic version
            If '', then use no system prompt (default)
          pre_prompt_query: Prompt that comes before document part
          prompt_query: Prompt that comes after document part
          pre_prompt_summary: Prompt that comes before document part
             None makes h2oGPT internally use its defaults
             E.g. "In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text"
          prompt_summary: Prompt that comes after document part
            None makes h2oGPT internally use its defaults
            E.g. "Using only the text above, write a condensed and concise summary of key results (preferably as bullet points):\n"
          i.e. for some internal document part fstring, the template looks like:
              template = "%s
              \"\"\"
              %s
              \"\"\"
              %s" % (pre_prompt_summary, fstring, prompt_summary)
          hyde_llm_prompt: hyde prompt for first step when using LLM
          h2ogpt_key: Access Key to h2oGPT server (if not already set in client at init time)
          model: base_model name or integer index of model_lock on h2oGPT server
                          None results in use of first (0th index) model in server
                 to get list of models do client.list_models()
          pre_prompt_extraction: Same as pre_prompt_summary but for when doing extraction
          prompt_extraction: Same as prompt_summary but for when doing extraction
          do_sample: see src/gen.py
          temperature: see src/gen.py
          top_p: see src/gen.py
          top_k: see src/gen.py
          repetition_penalty: see src/gen.py
          penalty_alpha: see src/gen.py
          max_new_tokens: see src/gen.py
          min_max_new_tokens: see src/gen.py
          max_input_tokens: see src/gen.py
          max_total_input_tokens: see src/gen.py
  
          stream_output: Whether to stream output
          do_sample: whether to sample
          max_time: how long to take
  
          add_search_to_context: Whether to do web search and add results to context
          chat_conversation: List of tuples for (human, bot) conversation that will be pre-appended to an (instruction, None) case for a query
          text_context_list: List of strings to add to context for non-database version of document Q/A for faster handling via API etc.
             Forces LangChain code path and uses as many entries in list as possible given max_seq_len, with first assumed to be most relevant and to go near prompt.
          docs_ordering_type: By default uses 'reverse_ucurve_sort' for optimal retrieval
          max_input_tokens: Max input tokens to place into model context for each LLM call
                                   -1 means auto, fully fill context for query, and fill by original document chunk for summarization
                                   >=0 means use that to limit context filling to that many tokens
          max_total_input_tokens: like max_input_tokens but instead of per LLM call, applies across all LLM calls for single summarization/extraction action
          max_new_tokens: Maximum new tokens
          min_max_new_tokens: minimum value for max_new_tokens when auto-adjusting for content of prompt, docs, etc.
  
          docs_token_handling: 'chunk' means fill context with top_k_docs (limited by max_input_tokens or model_max_len) chunks for query
                                                                           or top_k_docs original document chunks summarization
                                      None or 'split_or_merge' means same as 'chunk' for query, while for summarization merges documents to fill up to max_input_tokens or model_max_len tokens
          docs_joiner: string to join lists of text when doing split_or_merge.  None means '\n\n'
          hyde_level: 0-3 for HYDE.
                      0 uses just query to find similarity with docs
                      1 uses query + pure LLM response to find similarity with docs
                      2: uses query + LLM response using docs to find similarity with docs
                      3+: etc.
          hyde_template: see src/gen.py
          hyde_show_only_final: see src/gen.py
          doc_json_mode: see src/gen.py
  
          asserts: whether to do asserts to ensure handling is correct
  
      Returns: summary/answer: str or extraction List[str]
  
      """
      if self.config is None:
          self.setup()
      if self.persist:
          client = self
      else:
          client = self.clone()
      h2ogpt_key = h2ogpt_key or self.h2ogpt_key
      client.h2ogpt_key = h2ogpt_key
  
      self.check_model(model)
  
      # chunking not used here
      # MyData specifies scratch space, only persisted for this individual client call
      langchain_mode = langchain_mode or "MyData"
      loaders = tuple([None, None, None, None, None, None])
      doc_options = tuple([langchain_mode, chunk, chunk_size, embed])
      asserts |= bool(os.getenv("HARD_ASSERTS", False))
      if (
          text
          and isinstance(text, list)
          and not file
          and not url
          and not text_context_list
      ):
          # then can do optimized text-only path
          text_context_list = text
          text = None
  
      res = []
      if text:
          t0 = time.time()
          res = client.predict(
              text,
              *doc_options,
              *loaders,
              h2ogpt_key,
              api_name="/add_text",
          )
          t1 = time.time()
          print(
              "upload text: %s" % str(timedelta(seconds=t1 - t0)),
              flush=True,
          )
          if asserts:
              assert res[0] is None
              assert res[1] == langchain_mode
              assert "user_paste" in res[2]
              assert res[3] == ""
      if file:
          # upload file(s).  Can be list or single file
          # after below call, "file" replaced with remote location of file
          _, file = client.predict(file, api_name="/upload_api")
  
          res = client.predict(
              file,
              *doc_options,
              *loaders,
              h2ogpt_key,
              api_name="/add_file_api",
          )
          if asserts:
              assert res[0] is None
              assert res[1] == langchain_mode
              assert os.path.basename(file) in res[2]
              assert res[3] == ""
      if url:
          res = client.predict(
              url,
              *doc_options,
              *loaders,
              h2ogpt_key,
              api_name="/add_url",
          )
          if asserts:
              assert res[0] is None
              assert res[1] == langchain_mode
              assert url in res[2]
              assert res[3] == ""
              assert res[
                  4
              ]  # should have file name or something similar
      if res and not res[4] and "Exception" in res[2]:
          print("Exception: %s" % res[2], flush=True)
  
      # ask for summary, need to use same client if using MyData
      api_name = "/submit_nochat_api"  # NOTE: like submit_nochat but stable API for string dict passing
  
      pre_prompt_summary = (
          pre_prompt_summary
          if langchain_action == LangChainAction.SUMMARIZE_MAP.value
          else pre_prompt_extraction
      )
      prompt_summary = (
          prompt_summary
          if langchain_action == LangChainAction.SUMMARIZE_MAP.value
          else prompt_extraction
      )
  
      kwargs = dict(
          h2ogpt_key=h2ogpt_key,
          instruction=instruction,
          langchain_mode=langchain_mode,
          langchain_action=langchain_action,  # uses full document, not vectorDB chunks
          langchain_agents=langchain_agents,
          top_k_docs=top_k_docs,
          document_choice=document_choice,
          document_subset=document_subset,
          document_source_substrings=document_source_substrings,
          document_source_substrings_op=document_source_substrings_op,
          document_content_substrings=document_content_substrings,
          document_content_substrings_op=document_content_substrings_op,
          system_prompt=system_prompt,
          pre_prompt_query=pre_prompt_query,
          prompt_query=prompt_query,
          pre_prompt_summary=pre_prompt_summary,
          prompt_summary=prompt_summary,
          hyde_llm_prompt=hyde_llm_prompt,
          visible_models=model,
          stream_output=stream_output,
          do_sample=do_sample,
          temperature=temperature,
          top_p=top_p,
          top_k=top_k,
          repetition_penalty=repetition_penalty,
          penalty_alpha=penalty_alpha,
          max_time=max_time,
          max_new_tokens=max_new_tokens,
          add_search_to_context=add_search_to_context,
          chat_conversation=chat_conversation
          if chat_conversation
          else self.chat_conversation,
          text_context_list=text_context_list,
          docs_ordering_type=docs_ordering_type,
          min_max_new_tokens=min_max_new_tokens,
          max_input_tokens=max_input_tokens,
          max_total_input_tokens=max_total_input_tokens,
          docs_token_handling=docs_token_handling,
          docs_joiner=docs_joiner,
          hyde_level=hyde_level,
          hyde_template=hyde_template,
          hyde_show_only_final=hyde_show_only_final,
          doc_json_mode=doc_json_mode,
      )
  
      # in case server changed, update in case clone()
      self.server_hash = client.server_hash
  
      # ensure can fill conversation
      self.chat_conversation.append((instruction, None))
  
      # get result
      trials = 3
      for trial in range(trials):
          try:
              if not stream_output:
                  res = client.predict(
                      str(dict(kwargs)),
                      api_name=api_name,
                  )
                  # in case server changed, update in case clone()
                  self.server_hash = client.server_hash
                  res = ast.literal_eval(res)
                  response = res["response"]
                  if langchain_action != LangChainAction.EXTRACT.value:
                      response = response.strip()
                  else:
                      response = [
                          r.strip() for r in ast.literal_eval(response)
                      ]
                  sources = res["sources"]
                  scores_out = [x["score"] for x in sources]
                  texts_out = [x["content"] for x in sources]
                  if asserts:
                      if text and not file and not url:
                          assert any(
                              text[:cutoff] == texts_out
                              for cutoff in range(len(text))
                          )
                      assert len(texts_out) == len(scores_out)
  
                  yield response, texts_out
                  self.chat_conversation[-1] = (instruction, response)
              else:
                  job = client.submit(
                      str(dict(kwargs)), api_name=api_name
                  )
                  text0 = ""
                  response = ""
                  texts_out = []
                  while not job.done():
                      if (
                          job.communicator.job.latest_status.code.name
                          == "FINISHED"
                      ):
                          break
                      e = check_job(
                          job, timeout=0, raise_exception=False
                      )
                      if e is not None:
                          break
                      outputs_list = job.communicator.job.outputs
                      if outputs_list:
                          res = job.communicator.job.outputs[-1]
                          res_dict = ast.literal_eval(res)
                          response = res_dict[
                              "response"
                          ]  # keeps growing
                          text_chunk = response[
                              len(text0) :
                          ]  # only keep new stuff
                          if not text_chunk:
                              time.sleep(0.001)
                              continue
                          text0 = response
                          assert (
                              text_chunk
                          ), "must yield non-empty string"
                          yield text_chunk, texts_out
                      time.sleep(
                          0.1
                      )  # let LLM deliver larger chunks, don't need to get every token output immediately
  
                  # Get final response (if anything left), but also get the actual references (texts_out), above is empty.
                  res_all = job.outputs()
                  if len(res_all) > 0:
                      # 0.1 slightly longer than 0.02 in open source
                      check_job(job, timeout=0.1, raise_exception=True)
  
                      res = res_all[-1]
                      res_dict = ast.literal_eval(res)
                      response = res_dict["response"]
                      sources = res_dict["sources"]
                      texts_out = [x["content"] for x in sources]
                      yield response[len(text0) :], texts_out
                      self.chat_conversation[-1] = (
                          instruction,
                          response[len(text0) :],
                      )
                  else:
                      # 1.0 slightly longer than 0.3 in open source
                      check_job(job, timeout=1.0, raise_exception=True)
                      yield response[len(text0) :], texts_out
                      self.chat_conversation[-1] = (
                          instruction,
                          response[len(text0) :],
                      )
              break
          except Exception as e:
              print(
                  "h2oGPT predict failed: %s %s"
                  % (
                      str(e),
                      "".join(traceback.format_tb(e.__traceback__)),
                  ),
                  flush=True,
              )
              if trial == trials - 1:
                  raise
              else:
                  print("trying again: %s" % trial, flush=True)
                  time.sleep(1 * trial)
          finally:
              # in case server changed, update in case clone()
              self.server_hash = client.server_hash
  
imports: from __future__ import annotations
  import difflib
  import traceback
  import concurrent.futures
  import os
  import concurrent.futures
  import time
  import urllib.parse
  import uuid
  from concurrent.futures import Future
  from datetime import timedelta
  from enum import Enum
  from pathlib import Path
  from typing import Callable, Generator, Any, Union, List
  import ast
  from packaging import version
  from huggingface_hub import SpaceStage
  from huggingface_hub.utils import build_hf_headers
  from gradio_client import utils
  from importlib.metadata import distribution, PackageNotFoundError
  from gradio_client.client import Job, DEFAULT_TEMP_DIR, Endpoint
  from gradio_client import Client
label_desc: Use image node_id 18 for calling the GradioClient obj's underlying member method code with example usage: obj.query_or_summarize_or_extract(h2ogpt_key, instruction, text, file, url, embed, chunk, chunk_size, langchain_mode, langchain_action, langchain_agents, top_k_docs, document_choice, document_subset, document_source_substrings, document_source_substrings_op, document_content_substrings, document_content_substrings_op, system_prompt, pre_prompt_query, prompt_query, pre_prompt_summary, prompt_summary, pre_prompt_extraction, prompt_extraction, hyde_llm_prompt, model, stream_output, do_sample, temperature, top_p, top_k, repetition_penalty, penalty_alpha, max_time, max_new_tokens, add_search_to_context, chat_conversation, text_context_list, docs_ordering_type, min_max_new_tokens, max_input_tokens, max_total_input_tokens, docs_token_handling, docs_joiner, hyde_level, hyde_template, hyde_show_only_final, doc_json_mode, asserts) without return types