Active
Effective learning rate & pretraining dynamics
Further details will be added here in a future update.
Projects
Two research projects on pretraining dynamics and learning rate schedules.
Past
When should the learning rate decay?
Further details will be added here in a future update.