Benjamin Consolvo
commited on
Commit
·
f6bce69
1
Parent(s):
83f55e5
readme updates
Browse files
README.md
CHANGED
@@ -4,34 +4,23 @@ emoji: 📚
|
|
4 |
colorFrom: yellow
|
5 |
colorTo: purple
|
6 |
sdk: streamlit
|
7 |
-
sdk_version: 1.
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
license: apache-2.0
|
11 |
short_description: 'LLM Chatbot on Denvr Dataworks and Intel Gaudi'
|
12 |
---
|
13 |
|
14 |
-
# LLM
|
|
|
15 |
|
16 |
-
|
17 |
|
18 |
-
##
|
19 |
|
20 |
-
|
21 |
-
- **Chat Interface**: Clean and intuitive Streamlit chat UI
|
22 |
-
- **Streaming Responses**: Real-time streaming of AI-generated responses, including formatted code blocks if requested
|
23 |
-
- **Conversation History**: Maintain context throughout your conversation
|
24 |
-
- **New Chat**: Option to start a fresh conversation at any time
|
25 |
|
26 |
-
|
27 |
-
|
28 |
-
### Prerequisites
|
29 |
-
|
30 |
-
- Python 3.7+
|
31 |
-
- Streamlit
|
32 |
-
- OpenAI-compatible API key and endpoint
|
33 |
-
|
34 |
-
### Setup
|
35 |
|
36 |
1. Clone the repository:
|
37 |
```bash
|
@@ -44,61 +33,28 @@ cd intel-ai-enterprise-inference
|
|
44 |
pip install -r requirements.txt
|
45 |
```
|
46 |
|
47 |
-
## Configuration
|
48 |
-
|
49 |
### Secrets Management
|
50 |
|
51 |
-
This application requires API credentials to be set up in Streamlit's secrets management
|
52 |
|
53 |
1. On Hugging Face Spaces:
|
54 |
-
|
55 |
-
|
56 |
|
57 |
-
2. For local development:
|
58 |
-
- Create a `.streamlit/secrets.toml` file with:
|
59 |
```toml
|
60 |
openai_apikey = "your-api-key-here"
|
61 |
```
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
-
|
66 |
-
### On Hugging Face Spaces
|
67 |
-
|
68 |
-
You can create a new Hugging Face Space [here](https://huggingface.co/new-space), and then use git operations to clone, commit, and push your code changes directly to your Space. Here is the live link to the Space that you can replicate:
|
69 |
-
https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference.
|
70 |
-
|
71 |
-
### Local Development
|
72 |
-
|
73 |
Run the Streamlit application locally:
|
74 |
|
75 |
```bash
|
76 |
streamlit run app.py
|
77 |
```
|
78 |
|
79 |
-
|
80 |
-
## Using the Chatbot
|
81 |
-
|
82 |
-
1. Select your desired LLM model from the dropdown menu
|
83 |
-
2. Type your message in the chat input field
|
84 |
-
3. View the AI's response as it streams in real-time
|
85 |
-
4. Continue the conversation or start a new chat using the "Start New Chat" button
|
86 |
-
|
87 |
-
## Getting API Access
|
88 |
-
|
89 |
-
To use this application, you need an OpenAI-compatible API key from Denvr Dataworks:
|
90 |
-
|
91 |
-
1. Visit [Denvr Dataworks](https://www.denvrdata.com/intel) to check the latest available models
|
92 |
-
2. Sign up for API access to receive your API key
|
93 |
-
3. Configure the key in your Streamlit secrets
|
94 |
-
|
95 |
-
## Troubleshooting
|
96 |
-
|
97 |
-
- **API Key Issues**: Ensure your API key is correctly set in the Streamlit secrets
|
98 |
-
- **Model Unavailability**: If a model is not responding, try selecting a different model
|
99 |
-
- **Error Messages**: Check the error output for specific API or connection issues
|
100 |
-
- **Rate Limiting**: You might encounter rate limits depending on your API plan
|
101 |
-
|
102 |
## Community and Support
|
103 |
|
104 |
Join other AI developers on [Intel's DevHub Discord server](https://discord.gg/kfJ3NKEw5t) for discussions, support, and updates.
|
|
|
4 |
colorFrom: yellow
|
5 |
colorTo: purple
|
6 |
sdk: streamlit
|
7 |
+
sdk_version: 1.45.1
|
8 |
app_file: app.py
|
9 |
pinned: false
|
10 |
license: apache-2.0
|
11 |
short_description: 'LLM Chatbot on Denvr Dataworks and Intel Gaudi'
|
12 |
---
|
13 |
|
14 |
+
# LLM Chatbot
|
15 |
+
Similar to ChatGPT, this application provides a user-friendly Streamlit interface to interact with various LLM models hosted on Denvr Dataworks, powered by Intel Gaudi accelerators. The chatbot supports streaming responses and offers a selection of different language models, including Llama, DeepSeek, and Qwen models. Try it yourself with the models available in the left drop-down menu.
|
16 |
|
17 |
+
[](https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference)
|
18 |
|
19 |
+
## Setup
|
20 |
|
21 |
+
If you want to hose the application locally with Streamlit, you can follow the steps below. If you want to host the application on Hugging Face Spaces, the easiest way is to duplicate the space as per the screenshot, and set up your own API secrets as detailed below. Just like any GitHub repository, you can use the same Git actions with the Hugging Face Space to clone, add, push, and commit your changes.
|
|
|
|
|
|
|
|
|
22 |
|
23 |
+
[](https://huggingface.co/spaces/Intel/intel-ai-enterprise-inference)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
|
25 |
1. Clone the repository:
|
26 |
```bash
|
|
|
33 |
pip install -r requirements.txt
|
34 |
```
|
35 |
|
|
|
|
|
36 |
### Secrets Management
|
37 |
|
38 |
+
This application requires API credentials to be set up in Streamlit's secrets management. You need an OpenAI-compatible API key. In the case of this application, it is using an API key from [Denvr Dataworks](https://www.denvrdata.com/intel).
|
39 |
|
40 |
1. On Hugging Face Spaces:
|
41 |
+
- Add your OpenAI-compatible API key under "Secrets" in the HF settings as `openai_apikey`
|
42 |
+
- Add the base URL for your model endpoint under "Variables" as `base_url`
|
43 |
|
44 |
+
2. For local development, create a `.streamlit/secrets.toml` file with:
|
|
|
45 |
```toml
|
46 |
openai_apikey = "your-api-key-here"
|
47 |
```
|
48 |
+
Set the `base_url` environment variable to point to your OpenAI-compliant model endpoint with hosted models.
|
49 |
+
```bash
|
50 |
+
export base_url="https://api.inference.denvrdata.com/v1/"
|
51 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
52 |
Run the Streamlit application locally:
|
53 |
|
54 |
```bash
|
55 |
streamlit run app.py
|
56 |
```
|
57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
58 |
## Community and Support
|
59 |
|
60 |
Join other AI developers on [Intel's DevHub Discord server](https://discord.gg/kfJ3NKEw5t) for discussions, support, and updates.
|