ML EPS REG: Difference between revisions

Latest revision as of 08:48, 16 May 2022

ML_EPS_REG = [real]
Default: ML_EPS_REG = 1E-15

Description: Initial value for the threshold of the eigenvalues of the covariance matrix in the evidence approximation.

This threshold is used to determine which eigenvalues $\lambda _{k}$ of the covariance matrix $\mathbf {\Phi } ^{\mathrm {T} }\mathbf {\Phi } /\sigma _{\mathrm {v} }^{2}$ are used in the optimization of the regularization parameters $\sigma _{\mathrm {w} }^{2}$ and $\sigma _{\mathrm {v} }^{2}$ determined by the following equations

$\sigma _{\mathrm {w} }^{2}={\frac {|\mathbf {\bar {w}} |^{2}}{\gamma }},$

$\sigma _{\mathrm {v} }^{2}={\frac {|\mathbf {T} -\mathbf {\phi } \mathbf {\bar {w}} |^{2}}{M-\gamma }},$

$\gamma =\sum \limits _{k=1}^{N_{\mathrm {B} }}{\frac {\lambda _{k}}{\lambda _{k}+1/\sigma _{\mathrm {w} }^{2}}}$ .

All eigenvalues satisfying $\lambda _{i}/\lambda _{\mathrm {max} }$ > ML_EPS_REG are included in the above equations, whereas smaller eigenvalues are disregarded (they are anyway potentially inaccurate because of loss of significance).

If at any point during iterating the above equations, the quadratic norm of errors (eight column of REGR/REGRF in ML_LOGFILE) becomes too large (more than 1.2 times larger than in previous iterations), the code assumes that numerical issues (loss of significance) have occurred, and then ML_EPS_REG is automatically doubled. Furthermore, if the regression does not converge within 10 steps, ML_EPS_REG is also increased by a factor of 4. The maximum allowed iteration depths is 50 (the iteration number is the second entry of REGR/REGRF in ML_LOGFILE). When 50 iterations are reached, no force field is created and there is most likely something seriously wrong in the calculation.

The seventh entry of REGR/REGRF in the ML_LOGFILE shows the ratio of the regularization ( $\sigma _{v}^{2}/\sigma _{w}^{2}$ ) and the largest eigenvalue. Usually this number is a number with many varying digits. If this number becomes a "well rounded" number (e.g. 1.00000000E-14), this is an indication that the cap for the current ML_EPS_REG is reached. That means that regularization becomes crucial.

@@ Line 1: / Line 1: @@
-{{TAGDEF|ML_EPS_REG|[real]|1E-14}}
+{{DISPLAYTITLE:ML_EPS_REG}}
+{{TAGDEF|ML_EPS_REG|[real]|1E-15}}
-== Related Tags and Sections ==
+Description: Initial value for the threshold of the eigenvalues of the covariance matrix in the evidence approximation.
-{{TAG|ML_LMLFF}}, {{TAG|ML_MB}}
+----
+This threshold is used to determine which eigenvalues <math>\lambda_{k}</math> of the covariance matrix <math>\mathbf{\Phi}^{\mathrm{T}}\mathbf{\Phi}/\sigma^{2}_{\mathrm{v}}</math> are used in the optimization of the regularization parameters <math>\sigma^{2}_{\mathrm{w}}</math> and <math>\sigma^{2}_{\mathrm{v}}</math> determined by the following equations
+<math>
+\sigma^{2}_{\mathrm{w}}=\frac{|\mathbf{\bar{w}}|^{2}}{\gamma},
+</math>
+<math>
+\sigma^{2}_{\mathrm{v}}=\frac{|\mathbf{T}-\mathbf{\phi}\mathbf{\bar{w}}|^{2}}{M-\gamma},
+</math>
+<math>
+\gamma=\sum\limits_{k=1}^{N_{\mathrm{B}}} \frac{\lambda_{k}}{\lambda_{k}+1/\sigma^{2}_{\mathrm{w}}}
+</math>.
+All eigenvalues satisfying <math>\lambda_{i} / \lambda_{\mathrm{max}} </math> > {{TAG|ML_EPS_REG}} are included in the above equations, whereas smaller eigenvalues are disregarded (they are anyway potentially inaccurate because of loss of significance).
+If at any point during iterating the above equations, the quadratic norm of errors (eight column of <code>REGR/REGRF</code> in [[ML_LOGFILE]]) becomes too large (more than 1.2 times larger than in previous iterations), the code assumes that numerical issues
+(loss of significance) have occurred, and  then {{TAG|ML_EPS_REG}} is automatically doubled. Furthermore, if the regression does not converge within 10 steps, {{TAG|ML_EPS_REG}} is also increased by a factor of 4. The maximum allowed iteration depths is 50 (the iteration number is the second entry of <code>REGR/REGRF</code> in [[ML_LOGFILE]]). When 50 iterations are reached, no force field is created and there is most likely something seriously wrong in the calculation.
+The seventh entry of <code>REGR/REGRF</code> in the [[ML_LOGFILE]] shows the ratio of the regularization (<math>\sigma_{v}^{2}/ \sigma_{w}^{2}</math>) and the largest eigenvalue. Usually this number is a number with many varying digits. If this number becomes a "well rounded" number (e.g. 1.00000000E-14), this is an indication that the cap for the current {{TAG|ML_EPS_REG}} is reached. That means that regularization becomes crucial.
+== Related tags and articles ==
+{{TAG|ML_LMLFF}}, {{TAG|ML_IALGO_LINREG}}, {{TAG|ML_IREG}}, {{TAG|ML_SIGV0}}, {{TAG|ML_SIGW0}}
 {{sc|ML_EPS_LOW|Examples|Examples that use this tag}}
@@ Line 8: / Line 32: @@
 ----
-[[Category:INCAR]][[Category:Machine Learning]][[Category:Machine Learned Force Fields]][[Category: Alpha]]
+[[Category:INCAR tag]][[Category:Machine-learned force fields]]