orthogonality of eigenstates evaluated with IALGO=38 and IALGO=48

[John_L]

When I did a conventional LDA calculation on an FCC lattice composed of iron using IALGO=38, (SUBROUTINE EDDAV), I can verify that all the eigenstates are orthogonal to each other within an accuracy as high as 10^(-8). When I tried the same calculation using IALGO=48, (SUBROUTINE EDRMM), my test shows that these eigenstates are not as perfectly orthogonal to each other as coming out of IALGO=38. In some cases, they can be quite off.


More specifically, suppose E_NKS and |PSI_NKS> are the KS eigenvalue and eigenstates out of the LDA Hamiltonian, H, I did the following two checks,

(1) evaluate <PSI_N'K'S|H|PSI_NKS> and check whether it ~ delta_(N',N) delta_(K',K)
(2) compare E_NKS and <PSI_NKS|H|PSI_NKS>.

What I got are the following (TMP poses as a measure of how far off the expected agreements),

(A) If using IALGO=38, the calculation converges well to the expected EDIFF=1.e-6. Both checks pass the test within an error of 10^(-8) on a sequence of randomly selected K points

(B.1) If using IALGO=48, the calculation can converge well, but neither checks pass the test on a sequence of randomly selected K points, as shown below

WARNING: NK= 55 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.14162E-05
WARNING: NK= 126 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.19116E-04
WARNING: NK= 126 eigenvalue minus diagonal elements gives TMP= 0.10826E-04
WARNING: NK= 192 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.18584E-05
WARNING: NK= 264 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.15048E-05
WARNING: NK= 306 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.59638E-05
WARNING: NK= 306 eigenvalue minus diagonal elements gives TMP= 0.10375E-04
WARNING: NK= 354 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.24978E-05
WARNING: NK= 354 eigenvalue minus diagonal elements gives TMP= 0.94314E-06
WARNING: NK= 355 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.12503E-05
WARNING: NK= 394 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.21147E-05
WARNING: NK= 394 eigenvalue minus diagonal elements gives TMP= 0.37140E-04
WARNING: NK= 444 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.18498E-05
WARNING: NK= 444 eigenvalue minus diagonal elements gives TMP= 0.42648E-07
WARNING: NK= 458 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.21931E-05
WARNING: NK= 458 eigenvalue minus diagonal elements gives TMP= 0.98272E-07

(B.2) If using IALGO=48 but with RMM failed in some ZHEGV evaluations but still converging to the right energy, neither checks pass the test on a sequence of randomly selected K points with more severe violations of the expected relations, as shown below

WARNING: NK= 281 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.99259E-04
WARNING: NK= 281 eigenvalue minus diagonal elements gives TMP= 0.38873E-04
WARNING: NK= 419 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.23437E-05
WARNING: NK= 419 eigenvalue minus diagonal elements gives TMP= 0.18006E-05
WARNING: NK= 700 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.75541E-05
WARNING: NK= 700 eigenvalue minus diagonal elements gives TMP= 0.25216E-05
WARNING: NK= 721 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.56891E-03
WARNING: NK= 721 eigenvalue minus diagonal elements gives TMP= 0.55502E-03
WARNING: NK= 745 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.41127E-04
WARNING: NK= 745 eigenvalue minus diagonal elements gives TMP= 0.10832E-04
WARNING: NK= 872 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.11980E-04
WARNING: NK= 872 eigenvalue minus diagonal elements gives TMP= 0.38447E-04
WARNING: NK= 971 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.11135E-04
WARNING: NK= 971 eigenvalue minus diagonal elements gives TMP= 0.76651E-04
WARNING: NK= 1179 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.10313E-04
WARNING: NK= 1179 eigenvalue minus diagonal elements gives TMP= 0.11447E-03
WARNING: NK= 1229 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.14712E-04
WARNING: NK= 1229 eigenvalue minus diagonal elements gives TMP= 0.19513E-04
WARNING: NK= 1487 <PSI_N'K'S|H|PSI_NKS> not diagonal, TMP= 0.47975E-05
WARNING: NK= 1487 eigenvalue minus diagonal elements gives TMP= 0.81306E-06

Is my observation correct? Why would RMM converge worse than DAV? How should I trust the resulting eigenstates out of the RMM calculation given their poor orthogonalization, and deviation between the eigenvalues and the expectation value of H evaluated with the eigenstates?

A related question is, how might the error message of "WARNING in EDDRMM: call to ZHEGV failed, returncode = 6 3 12" affect the overall convergence of the iteration? In the case given here, it finally converges to the correct energy, but I met with cases which failed to converge to the correct energy. So, how should I trust the calculation if it brings up this error message?

The VASP version I am using is vasp.5.3.5. The POTCAR is from paw.5.2/lda/Fe

[merzuk.kaltak]

Please upload a full error report.

[John_L]

Please upload a full error report.

? Can you specify what is the error report? OUTCAR or screen output?

Towards the orthogonalization test, I coded myself and the error output is posted above. Since the test passed on DAV method, I thus think my code is fine, and ascribed the failure to fully fullfil orthogonalization under RMM is due to the RMM method itself.

[John_L]

Please upload a full error report.

Can you specify which one is the error report? OUTCAR or screen output?

Towards the orthogonalization test, I coded it myself making use of the KS eigenspectrum. The error report has been included in my original post. Since my code correctly verified DAV method to fullfil the orthogonalization condition, I thus think my code works fine. But the RMM method does not pass it.

[John_L]

Please let me know what is the full error report you mentioned. As to the fact I reported, you might just check by yourself the orthogonality of the eigenstates coming out of RMM method. I guess this could happen to any systems.

[merzuk.kaltak]

A full error report is described in the "forum posting guidelines" as described here

[John_L]

Here is the full error report. The running command is the usual mpirun on a node, no batch script is used.

Extra comments are included in the outtt.QO file, saving stdout information.

Thanks for your further help!

[John_L]

Could it be related to that I requested too many bands to be converged? Fe only has 8 valence electrons, and I requested 32 bands, which is far more than needed, and causing complications in reaching convergence in very high energy unoccupied states.

[henrique_miranda]

Thanks for the bug report.
This issue will require careful analysis from our side. This might take some time.

In the meantime here is some additional information that might explain the reason for the issue you are reporting:

In both IALGO=38 and 48 the diagonalization of the hamiltonian is done iteratively.
This means that we obtain the first eigenstate (eigenvalue and eigenpair), by applying the hamiltonian to a trial vector multiple times (wiki/index.php/NDAV). Then the code proceeds to find the second one.
To find this second one, we have to orthogonalize the trial vector with the first one.
This proceeds like this until all the requested eigenpairs are found. Each new vector has to be orthogonal to the previous ones.
In practive in VASP multiple eigenpairs might be found at the same time:
wiki/index.php/IALGO#The_blocked-Davidson_scheme

The orthogonalization step is the slowest part of the iterative diagonalization schemes.
That is the issue that the RMM-DIIS algorithm tries to address.
Read here about the implementation in the original paper:
https://iopscience.iop.org/article/10.1 ... /9/018/pdf
For a more VASP specific description:
https://journals.aps.org/prb/abstract/1 ... B.54.11169
there is also here a list of common issues found:
wiki/index.php/IALGO#RMM-DIIS

With respect to the convergence of the eigenstates: in VASP often you are often only interested in obtaining convergence for the ground-state.
As such the criteria for stoping the overall iterative procedure is the total energy:
wiki/index.php/EDIFF
This depends only on the occupied states.
Still, the orthogonalization is enforced during the iterative procedure so you should get orthogonal states independently of whether they are occupied or not.

[henrique_miranda]

Some further information regarding this question.

As I've mentioned in the previous post the RMM-DIIS algorithm tries to avoid the explicit orthogonalization which is the most expensive part of the iterative diagonalizaton.
The details of how this is done are explained in the papers I posted above.
This means that the states at the end of the RMM-DIIS might not be completely orthogonal to each other.
To ensure that at the end of the RMM-DIIS step you still get orthogonal states VASP performs an explicit orthogonalization of all the bands.
See in electron.F the call to ORTHCH if (.NOT.INFO%LORTHO).

Now I have not seen the code that you added to test the orthogonalization but perhaps you are calling it before this explicit orthogonalization step?