How to Scale Your EMA
Paper
•
2307.13813
•
Published
•
9
•
4
Dan Busbridge
dbusbridge
AI & ML interests
Deep learning, optimization, self-supervised learning, representation learning, large language modeling, equivariance, geometric deep learning, attention mechanisms, transformers
Recent Activity
authored
a paper
7 days ago
Stabilizing Transformer Training by Preventing Attention Entropy
Collapse
authored
a paper
7 days ago
The Role of Entropy and Reconstruction in Multi-View Self-Supervised
Learning
authored
a paper
7 days ago
Poly-View Contrastive Learning
Organizations
dbusbridge's activity
commented
2
papers
over 1 year ago