Stepsize decay schedule

Usage

stepsize_decay(
  method = c("none", "grad_norm_plateau"),
  patience = 3,
  gamma = 0.5,
  min_delta = 0,
  warmup = 0,
  min_stepsize = 0
)

method: decay strategy. "none" disables decay; "grad_norm_plateau" decays when mean grad.norm() across chains fails to decrease for a number of epochs.
patience: number of consecutive epochs without grad.norm() improvement before decaying stepsize.
gamma: decay factor applied when triggered (0 < gamma < 1).
min_delta: minimum required decrease in grad.norm() to be counted as improvement.
warmup: number of initial epochs to skip decay checks.
min_stepsize: lower bound for stepsize after decay (absolute value).

a list of control variables for stepsize decay (used in control_opt).

Define a stepsize decay strategy for use in control_opt(). Currently supports the plateau-based schedule using grad.norm().