TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language ModelsIn our #ICLR2025 paper, we try to push what can be accomplished with model distillation! We applied TAID to train a very capable SOTA ‘smol’ Japanese🇯🇵 LLM that works entirely on-device, and even runs inside a web browser using a Javascript inference engine!

If you want to chat with a Japanese LLM running in your browser locally, check out our interactive web demo. ‘TinySwallow-1.5B’ understands both Japanese and English.

Web demo:
#ICLR2025