learning rate finder

[wyli]

Is your feature request related to a problem? Please describe.
this feature request is about integrating a light-weight learning rate finder
described in https://arxiv.org/abs/1506.01186

"an “LR range test”: run your model for several epochs while letting the learning rate increase linearly between low and high LR values. This test is enormously valuable whenever you are facing a new architecture or dataset."

see also

• fastai lr_find: https://github.com/fastai/fastai/blob/master/nbs/14_callback.schedule.ipynb
• lr_finder: https://github.com/davidtvs/pytorch-lr-finder/blob/master/torch_lr_finder/lr_finder.py

cc @Can-Zhao

[Can-Zhao]

Yes, exactly. Thank you!

[Nic-Ma]

Looks interest an idea, will take a look soon.
Thanks.