ML EPS REG

ML_EPS_REG = [real]
Default: ML_EPS_REG = 1E-14 

Description: Initial value for the threshold of the eigenvalues of the covariance matrix in the evidence approximation.

---

This threshold is used to determine which eigenvalues ${\lambda }_k$ of the covariance matrix ${\mathrm{\Phi }}^{\mathrm{T}}\mathrm{\Phi }/{\sigma }_{\mathrm{v}}^2$ are used in the optimization of the regularization parameters ${\sigma }_{\mathrm{w}}^2$ and ${\sigma }_{\mathrm{v}}^2$ determined by the following equations

${\sigma }_{\mathrm{w}}^2=\frac{|\overline{\mathbf{w}}{|}^2}{\gamma },$

${\sigma }_{\mathrm{v}}^2=\frac{|\mathbf{𝐓}-\mathit{\varphi }\overline{\mathbf{w}}{|}^2}{M-\gamma },$

$\gamma =\sum _{k=1}^{N_{\mathrm{B}}}\frac{{\lambda }_k}{{\lambda }_k+1/{\sigma }_{\mathrm{w}}^2}$.

All eigenvalues satisfying ${\lambda }_i/{\lambda }_{\mathrm{m}\mathrm{a}\mathrm{x}}$ > ML_EPS_REG are contributing by the above equations.

The smaller the value of ML_EPS_REG the the smaller the error of the fit. But at the same time the effects of overfitting increase. We determined empirically that the default value of 1E-14 is a safe value in most cases.

If at any point in the iterations of the Evidence Approximation of the square of the quadratic norm of errors (eigth column of REGR/REGRF in ML_LOGFILE) gets too big (more than 1.2 times larger than before) then ML_EPS_REG is doubled.

The seventh entry of REGR/REGRF in the ML_LOGFILE shows the ratio of the regularization (${\sigma }_v^2/{\sigma }_w^2$) and the largest eigenvalue. Usually this number is a number with many varying digits. If this number becomes a "well rounded" number (e.g. 1.00000000E-14), it is an indication that the cap for the current ML_EPS_REG is reached. That means that regularization becomes crucial.

Related Tags and Sections

ML_LMLFF, ML_IALGO_LINREG, ML_IREG, ML_SIGV0, ML_SIGW0

Examples that use this tag

---