Training recipe for reproduction
#6
by
CCRss
- opened
Hello, great model.
Want to try training similar small model but for other language.
What was the stages of training?
For example:
- Stage pre-training
- lr-rate ?
- epochs
- training data
- training components (siglip, MLP?)
- weight decay
- warmup steps or something related to that
- Stage fine-tuning
- lr-rate ?
- epochs
- training data
- training components ?
- ...
And overall why this LLM was chosed? You experimented with other LLMs and decided to take this one? Same question regarding siglip.
Sorry if it's wrong questions, just curious.
Thanks for the model.
Hi, thanks for the interest! We'll have a full technical report out with the final 3.2 model set that will go through this kind of detail, so stay tuned for the full release soon.