Preconditioner SGD optimization

Preconditioner SGD optimization

precond_sgd(
  stepsize = 0.5,
  preconditioner = "fast",
  numerical_eps = 1e-05,
  precond_by_diff_chain = FALSE
)

Arguments

stepsize

stepsize for SGD

preconditioner

preconditioner, can be c("none", "fast", "full") "none" means no preconditioner, i.e., vanilla SGD, "full" means compute numerical hessian as preconditioner "fast" means compute numerical hessian except for the parameter of K matrix (for speed reason)

numerical_eps

numerical, the gap used for estimate preconditioner, default is 1e-5

precond_by_diff_chain

logical, compute preconditioner only at stop points and switch to use the preconditioner of the other chains. If FALSE, preconditioner will be computed at each iteration.

Value

a list of control variables for optimization (used in control_opt function)