ComponentsLLMsHugging Face Local PipelinesOn this pageHugging Face Local PipelinesHugging Face models can be run locally through the HuggingFacePipeline class.The Hugging Face Model Hub hosts over 120k models, 20k datasets, and 50k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together.These can be called from LangChain either through this local pipeline wrapper or by calling their hosted inference endpoints through the HuggingFaceHub class. For more information on the hosted pipelines, see the HuggingFaceHub notebook.To use, you should have the transformers python package installed, as well as pytorch. You can also install xformer for a more memory-efficient attention implementation.%pip install transformers --quietLoad the model​from langchain.llms import HuggingFacePipelinellm = HuggingFacePipeline.from_model_id(    model_id="bigscience/bloom-1b7",    task="text-generation",    model_kwargs={"temperature": 0, "max_length": 64},)Create Chain​With the model loaded into memory, you can compose it with a prompt to
form a chain.from langchain.prompts import PromptTemplatetemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)chain = prompt | llmquestion = "What is electroencephalography?"print(chain.invoke({"question": question}))Batch GPU Inference​If running on a device with GPU, you can also run inference on the GPU in batch mode.gpu_llm = HuggingFacePipeline.from_model_id(    model_id="bigscience/bloom-1b7",    task="text-generation",    device=0,  # -1 for CPU    batch_size=2,  # adjust as needed based on GPU map and model size.    model_kwargs={"temperature": 0, "max_length": 64},)gpu_chain = prompt | gpu_llm.bind(stop=["\n\n"])questions = []for i in range(4):    questions.append({"question": f"What is the number {i} in french?"})answers = gpu_chain.batch(questions)for answer in answers:    print(answer)PreviousHugging Face HubNextHuggingface TextGen InferenceLoad the modelCreate ChainBatch GPU Inference