ProvidersMoreDeepSparseOn this pageDeepSparseThis page covers how to use the DeepSparse inference runtime within LangChain.
It is broken into two parts: installation and setup, and then examples of DeepSparse usage.Installation and Setup​Install the Python package with pip install deepsparseChoose a SparseZoo model or export a support model to ONNX using OptimumWrappers​LLM​There exists a DeepSparse LLM wrapper, which you can access with:from langchain.llms import DeepSparseIt provides a unified interface for all models:llm = DeepSparse(model='zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none')print(llm('def fib():'))Additional parameters can be passed using the config parameter:config = {'max_generated_tokens': 256}llm = DeepSparse(model='zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none', config=config)PreviousDeepInfraNextDiffbotInstallation and SetupWrappersLLM