Finally, a Replacement for BERT: Introducing ModernBERT
•
664
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
Bringing BERT into modernity via both architecture changes and scaling
-
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 904k • • 887 -
answerdotai/ModernBERT-large
Fill-Mask • 0.4B • Updated • 141k • • 407 -
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper • 2412.13663 • Published • 151
Bringing BERT into modernity via both architecture changes and scaling
-
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 904k • • 887 -
answerdotai/ModernBERT-large
Fill-Mask • 0.4B • Updated • 141k • • 407 -
Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
Paper • 2412.13663 • Published • 151
Models trained during quantization aware fine-tuning experiments using PyTorch's FSDP.
A list of datasets, models and papers for making LLMs better at function calling and tool usage