Root Mean Square Propagation (RMSProp) SGD optimization

Root Mean Square Propagation (RMSProp) SGD optimization

rmsprop(stepsize = 0.05, beta1 = 0.9, epsilon = 1e-08)

Arguments

stepsize

stepsize for SGD

beta1

beta1 for momentum

epsilon

epsilon for numerical stability

Value

a list of control variables for optimization (used in control_opt function)

Details

The update rule for RMSProp is: $$v_t = \beta_1 v_{t-1} + (1 - \beta_1) g_t^2$$ $$x_{t+1} = x_t - \text{stepsize} * \frac{g_t}{\sqrt{v_t} + \epsilon}$$