metadata

title: Basic Smol Talk Chatbot
emoji: 🐠
colorFrom: green
colorTo: blue
sdk: gradio
sdk_version: 5.15.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: A replica of the basic smoltalk chatbot I run locally

Basic SmolLM2 chatbot:

This is a very basic chatbot using HuggingFaceTB/SmolLM2-[x]-Instruct hosted on the same host as the app. It is basically a replica of the chatbot I have for running locally.

Hardware Scale Up

I recommend running this on a little better hardware than this is set up with.
I'm using the free tier space, but it needs a few more CPU to write fast enough to be useful.
It is rather slow on this setup, but when I run this on my laptop, it works very well on CPU without GPU.

To run locally

Download the files or clone the repo.
Make sure you have a supported version of transformers and torch installed (or run pip3 install -r requirements.txt from the root folder of this repo).
Run python3 app.py from the root folder of this repo.
Set your browser to http://0.0.0.0:7860

Configuration options

In app.py, there are settings of the screen.
- If you run this locally on a laptop with at least 5CPU cores, I would recommend saving all your local work then set MODEL to HuggingFaceTB/SmolLM2-360M-Instruct.
- If this works without signs of resource saturation, try setting MODEL it to HuggingFaceTB/SmolLM2-1.7B-Instruct. This will write well and works fine on my laptop that is about 2 years old.

MAX_NEW_TOKENS = 250
MODEL="HuggingFaceTB/SmolLM2-135M-Instruct"
# MODEL="HuggingFaceTB/SmolLM2-360M-Instruct"
# MODEL="HuggingFaceTB/SmolLM2-1.7B-Instruct"
TEMPERATURE = 0.6
TOP_P = 0.95
REPETITION_PENALTY = 1.2