README.md · ttomy/proxy-lite-demo-for-setup at 63beeb4036b77305ff8fc13e6fc4da9b0e2276c5

A mini, open-weights version of our Proxy assistant.

Getting Started

Installation

Clone the repository:

git clone https://github.com/convergence-ai/proxy-lite.git

Set-up the environment with:

make proxy

Or do it manually:

uv venv --python 3.11 --python-preference managed
uv sync
uv pip install -e .
playwright install

Usage

proxy --help

You can directly run the proxy with:

proxy "Book a table for 2 at an Italian restaurant in Kings Cross tonight at 7pm."

Proxy-Lite Endpoint

By default, Proxy-Lite will point to an endpoint set up on HuggingFace spaces. This is a demo endpoint and is not suitable for production use; it may be very slow when under heavy load.

We recommend hosting your own endpoint with vLLM, you can use the following command:

vllm serve --model convergence-ai/proxy-lite-7b \
    --trust-remote-code \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --port 8008 \

You can set the api_base to point to your local endpoint when calling Proxy-Lite:

proxy --api-base http://localhost:8008/v1 "Book a table...

or by setting the environment variable:

export PROXY_LITE_API_BASE=http://localhost:8008/v1