File size: 2,093 Bytes
de35df9 aad5f00 de35df9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
# 🌊 TinyWave: Compact & Expressive Speech Language Models
**TinyWave** is a family of **efficient 2B-parameter speech language models** distilled from the 7B SPIRIT-LM teacher. These models support **speech-to-speech** and **interleaved speech–text generation**, optimized for real-time use on **commodity hardware**.
Built through **layer-aligned knowledge distillation**, TinyWave models retain **93–97%** of their teacher’s performance while using only **⅓ of the parameters** — ideal for use in voice agents, assistive technologies, and edge devices.
> 📖 Read the paper: [Efficient Interleaved Speech Modeling through Knowledge Distillation (arXiv:2506.23670)](https://arxiv.org/abs/2506.23670)
> 🌐 Demo & samples: [tinywave-landing](https://mohammadmahdinoori.github.io/tinywave-landing/)
> 💻 Code: [github.com/mohammadmahdinoori/TinyWave](https://github.com/mohammadmahdinoori/TinyWave)
---
## 🔧 Model Variants
| Model | Modality | Tokenizer | Description |
|--------------------------------------------------------------|---------------------------|------------------------|---------------------------------------------|
| [`tinywave/speech-base-2b`](https://huggingface.co/tinywave/speech-base-2b) | Speech → Speech | `spiritlm_base` | Base phonetic-only speech generation |
| [`tinywave/speech-expressive-2b`](https://huggingface.co/tinywave/speech-expressive-2b) | Speech → Expressive Speech | `spiritlm_expressive` | Includes pitch + style tokens |
| [`tinywave/interleaved-expressive-2b`](https://huggingface.co/tinywave/interleaved-expressive-2b) | Text ↔ Speech (interleaved) | `spiritlm_expressive` | Multimodal expressive generation |
| [`tinywave/expressive-spirit-lm-interleaved-librilight`](https://huggingface.co/tinywave/expressive-spirit-lm-interleaved-librilight) | Teacher (7B, interleaved) | `spiritlm_expressive` | LoRA-corrected SPIRIT-LM for distillation | |