File size: 411 Bytes
7c071a8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
### python demo
```
sudo apt-get update
sudo apt-get install pybind11-dev
pip3 install transformers_stream_generator einops tiktoken accelerate transformers==4.32.0
```
```
cd /workspace/LLM-TPU/models/Qwen1_5/python_demo
mkdir build && cd build
cmake .. && make
cp *cpython* ..
cd ..
python3 pipeline.py --model_path your_bmodel_path --tokenizer_path ../token_config/ --devid 0 --generation_mode greedy
```
|