Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
NormanTUD committed Jul 17, 2024
1 parent 2a4e538 commit 93c8d27
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions explain.js
Original file line number Diff line number Diff line change
Expand Up @@ -2149,20 +2149,20 @@ function model_to_latex () {
& \\rule{110mm}{0.4pt} & \\\\
& \\textbf{for} \\: t=1 \\: \\textbf{to} \\: \\text{epochs} \\: \\textbf{do} & \\text{Loop from t=1 to epochs} \\\\
& \\hspace{5mm}\\textbf{if} \\: \\text{maximize} & \\text{If maximize is true} \\\\
& \\hspace{5mm}\\textbf{if} \\: \\text{maximize} & \\\\
& \\hspace{10mm}g_t \\leftarrow -\\nabla_{\\theta} f_t (\\theta_{t-1}) & \\text{Compute negative gradient of the objective function} \\\\
& \\hspace{5mm}\\textbf{else} & \\text{If maximize is false} \\\\
& \\hspace{5mm}\\textbf{else} & \\\\
& \\hspace{10mm}g_t \\leftarrow \\nabla_{\\theta} f_t (\\theta_{t-1}) & \\text{Compute gradient of the objective function} \\\\
& \\hspace{5mm}\\textbf{if} \\: \\lambda \\neq 0 & \\text{If weight decay is not zero} \\\\
& \\hspace{10mm}g_t \\leftarrow g_t + \\lambda \\theta_{t-1} & \\text{Add weight decay term to the gradient} \\\\
& \\hspace{5mm}m_t \\leftarrow \\beta_1 m_{t-1} + (1 - \\beta_1) g_t & \\text{Update biased first moment estimate} \\\\
& \\hspace{5mm}v_t \\leftarrow \\beta_2 v_{t-1} + (1-\\beta_2) g^2_t & \\text{Update biased second moment estimate} \\\\
& \\hspace{5mm}\\widehat{m_t} \\leftarrow m_t/\\big(1-\\beta_1^t \\big) & \\text{Compute bias-corrected first moment estimate} \\\\
& \\hspace{5mm}\\widehat{v_t} \\leftarrow v_t/\\big(1-\\beta_2^t \\big) & \\text{Compute bias-corrected second moment estimate} \\\\
& \\hspace{5mm}\\textbf{if} \\: \\text{amsgrad} & \\text{If AMSGrad is true} \\\\
& \\hspace{5mm}\\textbf{if} \\: \\text{amsgrad} & \\\\
& \\hspace{10mm}\\widehat{v_t}^{max} \\leftarrow \\mathrm{max}(\\widehat{v_t}^{max}, \\widehat{v_t}) & \\text{Update the maximum of the second moment estimates} \\\\
& \\hspace{10mm}\\theta_t \\leftarrow \\theta_{t-1} - \\gamma \\widehat{m_t}/\\big(\\sqrt{\\widehat{v_t}^{max}} + \\epsilon \\big) & \\text{Update parameters with AMSGrad correction} \\\\
& \\hspace{5mm}\\textbf{else} & \\text{If AMSGrad is false} \\\\
& \\hspace{5mm}\\textbf{else} & \\\\
& \\hspace{10mm}\\theta_t \\leftarrow \\theta_{t-1} - \\gamma \\widehat{m_t}/\\big(\\sqrt{\\widehat{v_t}} + \\epsilon \\big) & \\text{Update parameters without AMSGrad correction} \\\\
& \\rule{110mm}{0.4pt} & \\\\[-1.ex]
& \\bf{return} \\: \\theta_t & \\text{Return the updated parameters} \\\\[-1.ex]
Expand Down

0 comments on commit 93c8d27

Please sign in to comment.