Benjamin Consolvo commited on
Commit
f6bce69
·
1 Parent(s): 83f55e5

readme updates

Browse files
Files changed (1) hide show
  1. README.md +15 -59
README.md CHANGED
@@ -4,34 +4,23 @@ emoji: 📚
4
  colorFrom: yellow
5
  colorTo: purple
6
  sdk: streamlit
7
- sdk_version: 1.42.2
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
  short_description: 'LLM Chatbot on Denvr Dataworks and Intel Gaudi'
12
  ---
13
 
14
- # LLM Chat App
 
15
 
16
- This application provides a user-friendly interface to interact with various LLM models hosted on Denvr Dataworks, powered by Intel Gaudi accelerators. The chatbot supports streaming responses and offers a selection of different language models, including Llama models and DeepSeek models.
17
 
18
- ## Features
19
 
20
- - **Model Selection**: Choose from multiple LLM models hosted on Intel Gaudi hardware
21
- - **Chat Interface**: Clean and intuitive Streamlit chat UI
22
- - **Streaming Responses**: Real-time streaming of AI-generated responses, including formatted code blocks if requested
23
- - **Conversation History**: Maintain context throughout your conversation
24
- - **New Chat**: Option to start a fresh conversation at any time
25
 
26
- ## Installation
27
-
28
- ### Prerequisites
29
-
30
- - Python 3.7+
31
- - Streamlit
32
- - OpenAI-compatible API key and endpoint
33
-
34
- ### Setup
35
 
36
  1. Clone the repository:
37
  ```bash
@@ -44,61 +33,28 @@ cd intel-ai-enterprise-inference
44
  pip install -r requirements.txt
45
  ```
46
 
47
- ## Configuration
48
-
49
  ### Secrets Management
50
 
51
- This application requires API credentials to be set up in Streamlit's secrets management:
52
 
53
  1. On Hugging Face Spaces:
54
- - Add your OpenAI-compatible API key under "Secrets" in the HF settings
55
- - Add the base URL for your model endpoint under "Variables" as `base_url`
56
 
57
- 2. For local development:
58
- - Create a `.streamlit/secrets.toml` file with:
59
  ```toml
60
  openai_apikey = "your-api-key-here"
61
  ```
62
- - Set the `base_url` environment variable to point to your model endpoint with hosted models.
63
-
64
- ## Running the Application
65
-
66
- ### On Hugging Face Spaces
67
-
68
- You can create a new Hugging Face Space [here](https://huggingface.co/new-space), and then use git operations to clone, commit, and push your code changes directly to your Space. Here is the live link to the Space that you can replicate:
69
- https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference.
70
-
71
- ### Local Development
72
-
73
  Run the Streamlit application locally:
74
 
75
  ```bash
76
  streamlit run app.py
77
  ```
78
 
79
-
80
- ## Using the Chatbot
81
-
82
- 1. Select your desired LLM model from the dropdown menu
83
- 2. Type your message in the chat input field
84
- 3. View the AI's response as it streams in real-time
85
- 4. Continue the conversation or start a new chat using the "Start New Chat" button
86
-
87
- ## Getting API Access
88
-
89
- To use this application, you need an OpenAI-compatible API key from Denvr Dataworks:
90
-
91
- 1. Visit [Denvr Dataworks](https://www.denvrdata.com/intel) to check the latest available models
92
- 2. Sign up for API access to receive your API key
93
- 3. Configure the key in your Streamlit secrets
94
-
95
- ## Troubleshooting
96
-
97
- - **API Key Issues**: Ensure your API key is correctly set in the Streamlit secrets
98
- - **Model Unavailability**: If a model is not responding, try selecting a different model
99
- - **Error Messages**: Check the error output for specific API or connection issues
100
- - **Rate Limiting**: You might encounter rate limits depending on your API plan
101
-
102
  ## Community and Support
103
 
104
  Join other AI developers on [Intel's DevHub Discord server](https://discord.gg/kfJ3NKEw5t) for discussions, support, and updates.
 
4
  colorFrom: yellow
5
  colorTo: purple
6
  sdk: streamlit
7
+ sdk_version: 1.45.1
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
  short_description: 'LLM Chatbot on Denvr Dataworks and Intel Gaudi'
12
  ---
13
 
14
+ # LLM Chatbot
15
+ Similar to ChatGPT, this application provides a user-friendly Streamlit interface to interact with various LLM models hosted on Denvr Dataworks, powered by Intel Gaudi accelerators. The chatbot supports streaming responses and offers a selection of different language models, including Llama, DeepSeek, and Qwen models. Try it yourself with the models available in the left drop-down menu.
16
 
17
+ [![llmchatbot](images/llmchatbot.png)](https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference)
18
 
19
+ ## Setup
20
 
21
+ If you want to hose the application locally with Streamlit, you can follow the steps below. If you want to host the application on Hugging Face Spaces, the easiest way is to duplicate the space as per the screenshot, and set up your own API secrets as detailed below. Just like any GitHub repository, you can use the same Git actions with the Hugging Face Space to clone, add, push, and commit your changes.
 
 
 
 
22
 
23
+ [![hf_dup](images/hf_dup.png)](https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference)
 
 
 
 
 
 
 
 
24
 
25
  1. Clone the repository:
26
  ```bash
 
33
  pip install -r requirements.txt
34
  ```
35
 
 
 
36
  ### Secrets Management
37
 
38
+ This application requires API credentials to be set up in Streamlit's secrets management. You need an OpenAI-compatible API key. In the case of this application, it is using an API key from [Denvr Dataworks](https://www.denvrdata.com/intel).
39
 
40
  1. On Hugging Face Spaces:
41
+ - Add your OpenAI-compatible API key under "Secrets" in the HF settings as `openai_apikey`
42
+ - Add the base URL for your model endpoint under "Variables" as `base_url`
43
 
44
+ 2. For local development, create a `.streamlit/secrets.toml` file with:
 
45
  ```toml
46
  openai_apikey = "your-api-key-here"
47
  ```
48
+ Set the `base_url` environment variable to point to your OpenAI-compliant model endpoint with hosted models.
49
+ ```bash
50
+ export base_url="https://api.inference.denvrdata.com/v1/"
51
+ ```
 
 
 
 
 
 
 
52
  Run the Streamlit application locally:
53
 
54
  ```bash
55
  streamlit run app.py
56
  ```
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  ## Community and Support
59
 
60
  Join other AI developers on [Intel's DevHub Discord server](https://discord.gg/kfJ3NKEw5t) for discussions, support, and updates.