autogen/website/blog/2023-07-14-Local-LLMs/index.md
Initiate server  First, launch the controller  ```bash python -m fastchat.serve.controller ```  Then, launch the model worker(s)  ```bash python -m fastchat.serve.model_worker --model-path chatglm2-6b ```  Finally, launch the RESTful API server  ```bash python -m fastchat.serve.openai_api_server --host localhost --port 8000 ```  Normally this will work. However, if you encounter error like [this](https://github.com/lm-sys/FastChat/issues/1641), commenting out all the lines containing `finish_reason` in `fastchat/protocol/api_protocol.py` and `fastchat/protocol/openai_api_protocol.py` will fix the problem. The modified code looks like:  ```python class CompletionResponseChoice(BaseModel):     index: int     text: str     logprobs: Optional[int] = None     # finish_reason: Optional[Literal["stop", "length"]]  class CompletionResponseStreamChoice(BaseModel):     index: int     text: str     logprobs: Optional[float] = None     # finish_reason: Optional[Literal["stop", "length"]] = None ```  