class_docstr: Enables text generation using HuggingFace Hub hosted chat-based LLMs. This component is designed to seamlessly
  inference chat-based models deployed on the Text Generation Inference (TGI) backend.
  
  You can use this component for chat LLMs hosted on Hugging Face inference endpoints, the rate-limited
  Inference API tier:
  
  ```python
  from haystack.components.generators.chat import HuggingFaceTGIChatGenerator
  from haystack.dataclasses import ChatMessage
  
  messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant"),
              ChatMessage.from_user("What's Natural Language Processing?")]
  
  
  client = HuggingFaceTGIChatGenerator(model="meta-llama/Llama-2-70b-chat-hf", token="<your-token>")
  client.warm_up()
  response = client.run(messages, generation_kwargs={"max_new_tokens": 120})
  print(response)
  ```
  
  For chat LLMs hosted on paid https://huggingface.co/inference-endpoints endpoint and/or your own custom TGI
  endpoint, you'll need to provide the URL of the endpoint as well as a valid token:
  
  ```python
  from haystack.components.generators.chat import HuggingFaceTGIChatGenerator
  from haystack.dataclasses import ChatMessage
  
  messages = [ChatMessage.from_system("\nYou are a helpful, respectful and honest assistant"),
              ChatMessage.from_user("What's Natural Language Processing?")]
  
  client = HuggingFaceTGIChatGenerator(model="meta-llama/Llama-2-70b-chat-hf",
                                       url="<your-tgi-endpoint-url>",
                                       token="<your-token>")
  client.warm_up()
  response = client.run(messages, generation_kwargs={"max_new_tokens": 120})
  print(response)
  ```
  
   Key Features and Compatibility:
       - **Primary Compatibility**: Designed to work seamlessly with any chat-based model deployed using the TGI
         framework. For more information on TGI, visit https://github.com/huggingface/text-generation-inference.
       - **Hugging Face Inference Endpoints**: Supports inference of TGI chat LLMs deployed on Hugging Face
         inference endpoints. For more details, refer to https://huggingface.co/inference-endpoints.
       - **Inference API Support**: Supports inference of TGI chat LLMs hosted on the rate-limited Inference
         API tier. Learn more about the Inference API at https://huggingface.co/inference-api.
         Discover available chat models using the following command:
         ```
         wget -qO- https://api-inference.huggingface.co/framework/text-generation-inference | grep chat
         ```
         and simply use the model ID as the model parameter for this component. You'll also need to provide a valid
         Hugging Face API token as the token parameter.
       - **Custom TGI Endpoints**: Supports inference of TGI chat LLMs deployed on custom TGI endpoints. Anyone can
         deploy their own TGI endpoint using the TGI framework. For more details, refer
         to https://huggingface.co/inference-endpoints.
  
   Input and Output Format:
       - **ChatMessage Format**: This component uses the ChatMessage format to structure both input and output,
         ensuring coherent and contextually relevant responses in chat-based text generation scenarios. Details on the
         ChatMessage format can be found at https://github.com/openai/openai-python/blob/main/chatml.md.
class_docstr_tok: ["Enables","text","generation","using","HuggingFace","Hub","hosted","chat-based","LLMs",".","This","component","is","designed","to","seamlessly","inference","chat-based","models","deployed","on","the","Text","Generation","Inference","(","TGI",")","backend",".","You","can","use","this","component","for","chat","LLMs","hosted","on","Hugging","Face","inference","endpoints",",","the","rate-limited","Inference","API","tier",":","``","`","python","from","haystack.components.generators.chat","import","HuggingFaceTGIChatGenerator","from","haystack.dataclasses","import","ChatMessage","messages","=","[","ChatMessage.from_system","(","``","\\nYou","are","a","helpful",",","respectful","and","honest","assistant","''",")",",","ChatMessage.from_user","(","``","What","'s","Natural","Language","Processing","?","''",")","]","client","=","HuggingFaceTGIChatGenerator","(","model=","''","meta-llama\/Llama-2-70b-chat-hf","''",",","token=","''","<","your-token",">","''",")","client.warm_up","(",")","response","=","client.run","(","messages",",","generation_kwargs=","{","``","max_new_tokens","''",":","120","}",")","print","(","response",")","``","`","For","chat","LLMs","hosted","on","paid","https",":","\/\/huggingface.co\/inference-endpoints","endpoint","and\/or","your","own","custom","TGI","endpoint",",","you","'ll","need","to","provide","the","URL","of","the","endpoint","as","well","as","a","valid","token",":","``","`","python","from","haystack.components.generators.chat","import","HuggingFaceTGIChatGenerator","from","haystack.dataclasses","import","ChatMessage","messages","=","[","ChatMessage.from_system","(","``","\\nYou","are","a","helpful",",","respectful","and","honest","assistant","''",")",",","ChatMessage.from_user","(","``","What","'s","Natural","Language","Processing","?","''",")","]","client","=","HuggingFaceTGIChatGenerator","(","model=","''","meta-llama\/Llama-2-70b-chat-hf","''",",","url=","''","<","your-tgi-endpoint-url",">","''",",","token=","''","<","your-token",">","''",")","client.warm_up","(",")","response","=","client.run","(","messages",",","generation_kwargs=","{","``","max_new_tokens","''",":","120","}",")","print","(","response",")","``","`","Key","Features","and","Compatibility",":","-","*","*","Primary","Compatibility","*","*",":","Designed","to","work","seamlessly","with","any","chat-based","model","deployed","using","the","TGI","framework",".","For","more","information","on","TGI",",","visit","https",":","\/\/github.com\/huggingface\/text-generation-inference",".","-","*","*","Hugging","Face","Inference","Endpoints","*","*",":","Supports","inference","of","TGI","chat","LLMs","deployed","on","Hugging","Face","inference","endpoints",".","For","more","details",",","refer","to","https",":","\/\/huggingface.co\/inference-endpoints",".","-","*","*","Inference","API","Support","*","*",":","Supports","inference","of","TGI","chat","LLMs","hosted","on","the","rate-limited","Inference","API","tier",".","Learn","more","about","the","Inference","API","at","https",":","\/\/huggingface.co\/inference-api",".","Discover","available","chat","models","using","the","following","command",":","``","`","wget","-qO-","https",":","\/\/api-inference.huggingface.co\/framework\/text-generation-inference","|","grep","chat","``","`","and","simply","use","the","model","ID","as","the","model","parameter","for","this","component",".","You","'ll","also","need","to","provide","a","valid","Hugging","Face","API","token","as","the","token","parameter",".","-","*","*","Custom","TGI","Endpoints","*","*",":","Supports","inference","of","TGI","chat","LLMs","deployed","on","custom","TGI","endpoints",".","Anyone","can","deploy","their","own","TGI","endpoint","using","the","TGI","framework",".","For","more","details",",","refer","to","https",":","\/\/huggingface.co\/inference-endpoints",".","Input","and","Output","Format",":","-","*","*","ChatMessage","Format","*","*",":","This","component","uses","the","ChatMessage","format","to","structure","both","input","and","output",",","ensuring","coherent","and","contextually","relevant","responses","in","chat-based","text","generation","scenarios",".","Details","on","the","ChatMessage","format","can","be","found","at","https",":","\/\/github.com\/openai\/openai-python\/blob\/main\/chatml.md","."]
docstr: Initialize the HuggingFaceTGIChatGenerator instance.
  
  :param model: A string representing the model path or URL. Default is "meta-llama/Llama-2-13b-chat-hf".
  :param url: An optional string representing the URL of the TGI endpoint.
  :param chat_template: This optional parameter allows you to specify a Jinja template for formatting chat
      messages. While high-quality and well-supported chat models typically include their own chat templates
      accessible through their tokenizer, there are models that do not offer this feature. For such scenarios,
      or if you wish to use a custom template instead of the model's default, you can use this parameter to
      set your preferred chat template.
  :param token: The Hugging Face token for HTTP bearer authorization.
      You can find your HF token at https://huggingface.co/settings/tokens.
  :param generation_kwargs: A dictionary containing keyword arguments to customize text generation.
      Some examples: `max_new_tokens`, `temperature`, `top_k`, `top_p`,...
      See Hugging Face's [documentation](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationParameters)
      for more information.
  :param stop_words: An optional list of strings representing the stop words.
  :param streaming_callback: An optional callable for handling streaming responses.
docstr_tok: ["Initialize","the","HuggingFaceTGIChatGenerator","instance",".",":","param","model",":","A","string","representing","the","model","path","or","URL",".","Default","is","``","meta-llama\/Llama-2-13b-chat-hf","''",".",":","param","url",":","An","optional","string","representing","the","URL","of","the","TGI","endpoint",".",":","param","chat_template",":","This","optional","parameter","allows","you","to","specify","a","Jinja","template","for","formatting","chat","messages",".","While","high-quality","and","well-supported","chat","models","typically","include","their","own","chat","templates","accessible","through","their","tokenizer",",","there","are","models","that","do","not","offer","this","feature",".","For","such","scenarios",",","or","if","you","wish","to","use","a","custom","template","instead","of","the","model","'s","default",",","you","can","use","this","parameter","to","set","your","preferred","chat","template",".",":","param","token",":","The","Hugging","Face","token","for","HTTP","bearer","authorization",".","You","can","find","your","HF","token","at","https",":","\/\/huggingface.co\/settings\/tokens",".",":","param","generation_kwargs",":","A","dictionary","containing","keyword","arguments","to","customize","text","generation",".","Some","examples",":","`","max_new_tokens","`",",","`","temperature","`",",","`","top_k","`",",","`","top_p","`",",","...","See","Hugging","Face","'s","[","documentation","]","(","https",":","\/\/huggingface.co\/docs\/huggingface_hub\/v0.18.0.rc0\/en\/package_reference\/inference_client","#","huggingface_hub.inference._text_generation.TextGenerationParameters",")","for","more","information",".",":","param","stop_words",":","An","optional","list","of","strings","representing","the","stop","words",".",":","param","streaming_callback",":","An","optional","callable","for","handling","streaming","responses","."]
code: def __init__(
      self,
      model: str = "meta-llama/Llama-2-13b-chat-hf",
      url: Optional[str] = None,
      token: Optional[str] = None,
      chat_template: Optional[str] = None,
      generation_kwargs: Optional[Dict[str, Any]] = None,
      stop_words: Optional[List[str]] = None,
      streaming_callback: Optional[
          Callable[[StreamingChunk], None]
      ] = None,
  ):
      """
      Initialize the HuggingFaceTGIChatGenerator instance.
  
      :param model: A string representing the model path or URL. Default is "meta-llama/Llama-2-13b-chat-hf".
      :param url: An optional string representing the URL of the TGI endpoint.
      :param chat_template: This optional parameter allows you to specify a Jinja template for formatting chat
          messages. While high-quality and well-supported chat models typically include their own chat templates
          accessible through their tokenizer, there are models that do not offer this feature. For such scenarios,
          or if you wish to use a custom template instead of the model's default, you can use this parameter to
          set your preferred chat template.
      :param token: The Hugging Face token for HTTP bearer authorization.
          You can find your HF token at https://huggingface.co/settings/tokens.
      :param generation_kwargs: A dictionary containing keyword arguments to customize text generation.
          Some examples: `max_new_tokens`, `temperature`, `top_k`, `top_p`,...
          See Hugging Face's [documentation](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationParameters)
          for more information.
      :param stop_words: An optional list of strings representing the stop words.
      :param streaming_callback: An optional callable for handling streaming responses.
      """
      transformers_import.check()
  
      if url:
          r = urlparse(url)
          is_valid_url = all([r.scheme in ["http", "https"], r.netloc])
          if not is_valid_url:
              raise ValueError(
                  f"Invalid TGI endpoint URL provided: {url}"
              )
  
      check_valid_model(model, token)
  
      # handle generation kwargs setup
      generation_kwargs = (
          generation_kwargs.copy() if generation_kwargs else {}
      )
      check_generation_params(generation_kwargs, ["n"])
      generation_kwargs["stop_sequences"] = generation_kwargs.get(
          "stop_sequences", []
      )
      generation_kwargs["stop_sequences"].extend(stop_words or [])
  
      self.model = model
      self.url = url
      self.chat_template = chat_template
      self.token = token
      self.generation_kwargs = generation_kwargs
      self.client = InferenceClient(url or model, token=token)
      self.streaming_callback = streaming_callback
      self.tokenizer = None
  
file_path: haystack/haystack/components/generators/chat/hugging_face_tgi.py
imports: import logging
  from dataclasses import asdict
  from typing import Any, Dict, List, Optional, Iterable, Callable
  from urllib.parse import urlparse
  from haystack import component, default_to_dict, default_from_dict
  from haystack.components.generators.utils import serialize_callback_handler, deserialize_callback_handler
  from haystack.dataclasses import ChatMessage, StreamingChunk
  from haystack.components.generators.hf_utils import check_valid_model, check_generation_params
  from haystack.lazy_imports import LazyImport
label_desc: Use image node_id 1 to create a new HuggingFaceTGIChatGenerator object  with example: obj = HuggingFaceTGIChatGenerator(model, url, token, chat_template, generation_kwargs, stop_words, streaming_callback)