Preconditioner SGD optimization

Source: R/optimizer.R

Preconditioner SGD optimization

precond_sgd(
  stepsize = 0.5,
  preconditioner = "fast",
  numerical_eps = 1e-05,
  precond_by_diff_chain = FALSE,
  compute_precond_each_iter = FALSE
)

Arguments

stepsize

stepsize for SGD

preconditioner

preconditioner, can be c("none", "fast", "full") "none" means no preconditioner, i.e., vanilla SGD, "full" means compute numerical hessian as preconditioner "fast" means compute numerical hessian except for the parameter of K matrix (for speed reason)

numerical_eps

numerical, the gap used for estimate preconditioner, default is 1e-5

precond_by_diff_chain

logical, if TRUE, use different chains to estimate preconditioner (only computed at check points), if FALSE, use the same chain to estimate preconditioner (computed at each iteration)

compute_precond_each_iter

logical, if TRUE, compute preconditioner at each iteration, if FALSE, only compute preconditioner at check points (if has only 1 chain running, it will be set TRUE)

Value

a list of control variables for optimization (used in control_opt function)