Learning Rate is Too Large

Posted by : ()

Category :

What if I see a training accuracy scalar graphic like this:

Accuracy

The accuracy curve of training mini-batch is going down a little bit over time after reached a relative high point. That might tell me the learning rate is too large.

When the learning rate is too large, the optimizer function can not converge the loss by adding derivative to variables–every step is too large, and the loss will become biger and biger.

About Sida Liu

I am currently a M.S. graduate student in Morphology, Evolution & Cognition Laboratory at University of Vermont. I am interested in artificial intelligence, artificial life, and artificial environment.

Follow @liusida
Useful Links