When should the learning rate decay?

Further details will be added here in a future update.

Read the paper on arXiv