Reduce On Plateau Learning Rate Scheduler
Source:R/CallbackSetLRScheduler.R
mlr_callback_set.lr_scheduler_reduce_on_plateau.RdReduces the learning rate when the first validation measure, i.e. the first entry of the
learner's measures_valid, stops improving for patience evaluation steps.
Wraps torch::lr_reduce_on_plateau().
Note that patience counts evaluation steps, not epochs: when the learner's eval_freq is
greater than 1, the scheduler only steps in the epochs in which the validation measures are
evaluated, so patience evaluation steps correspond to patience * eval_freq epochs.
Unless mode is set, it is derived from the measure: "min" if the measure is minimized and
"max" if it is maximized. Training fails when mode contradicts the measure.
When the learner has no validation measure, e.g. in the final fit of an
AutoTuner that tunes the epochs internally, a warning is given and
the learning rate is not reduced.
Resuming
As for CallbackSetLRScheduler. For this schedule the restored state includes the best score
seen so far and how long it has been stagnating, so patience keeps counting across runs instead
of starting over.
Super classes
CallbackSet -> CallbackSetLRScheduler -> CallbackSetLRSchedulerReduceOnPlateau
Methods
CallbackSetLRSchedulerReduceOnPlateau$on_begin()
Creates the scheduler using the optimizer from the context, with the mode that fits the
first validation measure.