fix docs regarding TGI endpoint url (#408)
Browse filesThe README specify to use a `/generate_stream` url, when using a custom
TGI endpoint. This is incorrect, as it breaks APIs, such as the
conversation summary that depends on the `/` TGI interface.
This turns out to be a documentation-only bug, as `/` will point to
`/generate_stream` when `stream: true` is set in the request JSON.
Simply using the `/` url as the TGI endpoint therefore fixes the
summarization bug.
Fixes: #278
README.md
CHANGED
|
@@ -164,7 +164,7 @@ To do this, you can add your own endpoints to the `MODELS` variable in `.env.loc
|
|
| 164 |
|
| 165 |
{
|
| 166 |
// rest of the model config here
|
| 167 |
-
"endpoints": [{"url": "https://HOST:PORT
|
| 168 |
}
|
| 169 |
|
| 170 |
```
|
|
@@ -191,7 +191,7 @@ You can then add the generated information and the `authorization` parameter to
|
|
| 191 |
|
| 192 |
"endpoints": [
|
| 193 |
{
|
| 194 |
-
"url": "https://HOST:PORT
|
| 195 |
"authorization": "Basic VVNFUjpQQVNT",
|
| 196 |
}
|
| 197 |
]
|
|
@@ -232,11 +232,11 @@ If the model being hosted will be available on multiple servers/instances add th
|
|
| 232 |
|
| 233 |
"endpoints": [
|
| 234 |
{
|
| 235 |
-
"url": "https://HOST:PORT
|
| 236 |
"weight": 1
|
| 237 |
}
|
| 238 |
{
|
| 239 |
-
"url": "https://HOST:PORT
|
| 240 |
"weight": 2
|
| 241 |
}
|
| 242 |
...
|
|
|
|
| 164 |
|
| 165 |
{
|
| 166 |
// rest of the model config here
|
| 167 |
+
"endpoints": [{"url": "https://HOST:PORT"}]
|
| 168 |
}
|
| 169 |
|
| 170 |
```
|
|
|
|
| 191 |
|
| 192 |
"endpoints": [
|
| 193 |
{
|
| 194 |
+
"url": "https://HOST:PORT",
|
| 195 |
"authorization": "Basic VVNFUjpQQVNT",
|
| 196 |
}
|
| 197 |
]
|
|
|
|
| 232 |
|
| 233 |
"endpoints": [
|
| 234 |
{
|
| 235 |
+
"url": "https://HOST:PORT",
|
| 236 |
"weight": 1
|
| 237 |
}
|
| 238 |
{
|
| 239 |
+
"url": "https://HOST:PORT",
|
| 240 |
"weight": 2
|
| 241 |
}
|
| 242 |
...
|