Dan Busbridge

dbusbridge's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

authored 5 papers 7 days ago

Stabilizing Transformer Training by Preventing Attention Entropy Collapse

Paper • 2303.06296 • Published Mar 11, 2023

The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning

Paper • 2307.10907 • Published Jul 20, 2023 • 8

Poly-View Contrastive Learning

Paper • 2403.05490 • Published Mar 8, 2024

Position Prediction as an Effective Pretraining Strategy

Paper • 2207.07611 • Published Jul 15, 2022 • 1

Distillation Scaling Laws

Paper • 2502.08606 • Published 8 days ago • 41

upvoted a paper 8 days ago

Distillation Scaling Laws

Paper • 2502.08606 • Published 8 days ago • 41

authored a paper 24 days ago

Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models

Paper • 2501.12370 • Published about 1 month ago • 11

authored a paper 5 months ago

Theory, Analysis, and Best Practices for Sigmoid Self-Attention

Paper • 2409.04431 • Published Sep 6, 2024 • 1

commented 2 papers over 1 year ago

How to Scale Your EMA

Paper • 2307.13813 • Published Jul 25, 2023 • 9 •

4

How to Scale Your EMA

Paper • 2307.13813 • Published Jul 25, 2023 • 9 •

4

authored a paper over 1 year ago

How to Scale Your EMA

Paper • 2307.13813 • Published Jul 25, 2023 • 9