Page 1 of 1

HSE06 scf fails to converge

Posted: Mon Sep 26, 2022 5:49 pm
by guanzhi_li
Dear VASP Community,

I am trying to do HSE-level calculation for GaN with a C_N substitutional impurity. I want the defect to be occupied so use ISPIN = 2 and ISMEAR = -2 to set the occupations mannually. However, the scf jumped back and forth between two energies and didn't converge. I have asked help from a few of VASP users in my department but hasn't solved the problem yet. I'd appreciate any advice you can give me.

I first optimized my struture at PBE-level using Gamma point. It was successful. Here is my INCAR for structure optimization.
SYSTEM = GaN_CN

! How to initialize charge density and wave functions
ISTART = 1 ! 0 to start a new job, 1 for continuing and reading in wf
ICHARG = 2 ! 2 to start a new job, 1 for continuing and reading in chg
INIWAV = 1 ! chooses initially random wf, numerically best

! Electronic control flags
ENCUT = 400 ! cutoff energy
EDIFF = 1E-8 ! convergence threshold
PREC = Normal ! numerical precision
ALGO = Normal ! Davidson algorithm, usually best choice
NELMIN = 5 ! min electronic steps
NELM = 50 ! max electronic steps

ISPIN = 2 ! Spin polarized calculation: 1 = No, 2 = Yes
LMAXMIX = 4

! Hybrid functional flags. Uncomment to run a hybrid calculation;
!LHFCALC = .TRUE. ! Turns on doing the hybrid functional
!HFSCREEN = 0.2 ! Screening length, we won't adjust this
!AEXX = 0.33 ! This is the amount of exact exchanged mixed in, we'll need to check
!PRECFOCK = Fast

! Parallelization flags
NCORE = 16 ! Band parallelization, 16 is good for PBE, 4 is good for HSE
KPAR = 1 ! Set to number of k-point blocks (remember to multiply nodes/procs in jobscript)

! Ionic control flags
EDIFFG = -1E-2 ! Convergence for ionic relaxation
NSW = 50 ! Number of ionic steps per relaxation job
IBRION = 2 ! Relaxation algorithm, -1 is off, 2 is conjugate gradient
ISIF = 2 ! What to relax. 2 does just the internal DOF, 3 does the lattice vectors too

! Electronic broadening
!ISMEAR = 0 ! 0 is Gaussian broadening
!SIGMA = 0.1 ! amount of broadening

ISMEAR = -2
FERWE = 432*1 144*0
FERDO = 428*1 1 1 0 1 144*0
NBANDS = 576

LWAVE = .TRUE. ! write out wavefunctions
LCHARG = .TRUE. ! write out chargedensity
LVTOT = .FALSE. ! write out full potential
LVHAR = .FALSE. ! write out electrostatic potential

Then I turned on hybrid functional flags and restart from the converged WAVECAR. The INCAR for HSE calculation is

SYSTEM = GaN_CN

! How to initialize charge density and wave functions
ISTART = 1 ! 0 to start a new job, 1 for continuing and reading in wf
ICHARG = 2 ! 2 to start a new job, 1 for continuing and reading in chg
INIWAV = 1 ! chooses initially random wf, numerically best

! Electronic control flags
ENCUT = 400 ! cutoff energy
EDIFF = 1E-6 ! convergence threshold
PREC = Normal ! numerical precision
ALGO = Normal ! Davidson algorithm, usually best choice
NELMIN = 5 ! min electronic steps
NELM = 50 ! max electronic steps

ISPIN = 2 ! Spin polarized calculation: 1 = No, 2 = Yes
LMAXMIX = 4

! Hybrid functional flags. Uncomment to run a hybrid calculation;
LHFCALC = .TRUE. ! Turns on doing the hybrid functional
HFSCREEN = 0.2 ! Screening length, we won't adjust this
AEXX = 0.33 ! This is the amount of exact exchanged mixed in, we'll need to check
PRECFOCK = Fast

! Parallelization flags
NCORE = 16 ! Band parallelization, 16 is good for PBE, 4 is good for HSE
KPAR = 1 ! Set to number of k-point blocks (remember to multiply nodes/procs in jobscript)

! Ionic control flags
EDIFFG = -1E-2 ! Convergence for ionic relaxation
NSW = 0 ! Number of ionic steps per relaxation job
IBRION = 2 ! Relaxation algorithm, -1 is off, 2 is conjugate gradient
ISIF = 2 ! What to relax. 2 does just the internal DOF, 3 does the lattice vectors too

! Electronic broadening
!ISMEAR = 0 ! 0 is Gaussian broadening
!SIGMA = 0.1 ! amount of broadening

ISMEAR = -2
FERWE = 432*1 144*0
FERDO = 428*1 1 1 0 1 144*0
NBANDS = 576

LWAVE = .TRUE. ! write out wavefunctions
LCHARG = .TRUE. ! write out chargedensity
LVTOT = .FALSE. ! write out full potential
LVHAR = .FALSE. ! write out electrostatic potential

The log of HSE calculation is

running on 256 total cores
distrk: each k-point on 256 cores, 1 groups
distr: one band on 16 cores, 16 groups
using from now: INCAR
vasp.5.4.4.18Apr17-6-g9f103f2a35 (build May 20 2021 18:19:39) complex

POSCAR found type information on POSCAR Ga N C
POSCAR found : 3 types and 96 ions

-----------------------------------------------------------------------------
| |
| ADVICE TO THIS USER RUNNING 'VASP/VAMP' (HEAR YOUR MASTER'S VOICE ...): |
| |
| You have a (more or less) 'large supercell' and for larger cells |
| it might be more efficient to use real space projection opertators |
| So try LREAL= Auto in the INCAR file. |
| Mind: For very accurate calculation you might also keep the |
| reciprocal projection scheme (i.e. LREAL=.FALSE.) |
| |
-----------------------------------------------------------------------------

LDA part: xc-table for Pade appr. of Perdew
found WAVECAR, reading the header
POSCAR, INCAR and KPOINTS ok, starting setup
FFT: planning ...
reading WAVECAR
the WAVECAR file was read successfully
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 -0.814476546887E+03 -0.81448E+03 -0.15579E+02 3184 0.293E+01 0.718E+01
DAV: 2 -0.739202106646E+03 0.75274E+02 -0.64853E+01 3008 0.345E+01 0.332E+01
DAV: 3 -0.739384029139E+03 -0.18192E+00 -0.47840E+01 2736 0.157E+01 0.187E+01
DAV: 4 -0.744174409802E+03 -0.47904E+01 -0.10195E+01 2880 0.970E+00 0.131E+01
DAV: 5 -0.743875011005E+03 0.29940E+00 0.54942E+00 2752 0.621E+00 0.788E+00
DAV: 6 -0.744025338861E+03 -0.15033E+00 0.64615E+00 2640 0.412E+00 0.481E+00
DAV: 7 -0.744151630933E+03 -0.12629E+00 0.70386E+00 2688 0.353E+00 0.358E+00
DAV: 8 -0.744214299109E+03 -0.62668E-01 0.66152E+00 2672 0.302E+00 0.321E+00
DAV: 9 -0.744189732684E+03 0.24566E-01 0.73928E+00 2688 0.253E+00 0.310E+00
DAV: 10 -0.744216761456E+03 -0.27029E-01 0.69713E+00 2752 0.257E+00 0.306E+00
DAV: 11 -0.744197769572E+03 0.18992E-01 0.74475E+00 2672 0.246E+00 0.304E+00
DAV: 12 -0.744220038029E+03 -0.22268E-01 0.69969E+00 2736 0.244E+00 0.304E+00
DAV: 13 -0.744200450121E+03 0.19588E-01 0.74378E+00 2672 0.235E+00 0.304E+00
DAV: 14 -0.744222130275E+03 -0.21680E-01 0.69819E+00 2752 0.238E+00 0.303E+00
DAV: 15 -0.744201070632E+03 0.21060E-01 0.74404E+00 2704 0.231E+00 0.304E+00
DAV: 16 -0.744221531713E+03 -0.20461E-01 0.70012E+00 2800 0.235E+00 0.303E+00
DAV: 17 -0.744201706611E+03 0.19825E-01 0.74378E+00 2768 0.228E+00 0.303E+00
DAV: 18 -0.744222593607E+03 -0.20887E-01 0.69896E+00 2864 0.233E+00 0.303E+00
DAV: 19 -0.744202068377E+03 0.20525E-01 0.74332E+00 2800 0.227E+00 0.303E+00
DAV: 20 -0.744221696587E+03 -0.19628E-01 0.70064E+00 2848 0.232E+00 0.303E+00
DAV: 21 -0.744202235724E+03 0.19461E-01 0.74330E+00 2768 0.226E+00 0.303E+00
DAV: 22 -0.744222444738E+03 -0.20209E-01 0.69957E+00 2864 0.231E+00 0.303E+00
DAV: 23 -0.744202392150E+03 0.20053E-01 0.74291E+00 2768 0.226E+00 0.303E+00
DAV: 24 -0.744221553323E+03 -0.19161E-01 0.70105E+00 2864 0.231E+00 0.303E+00
DAV: 25 -0.744202469963E+03 0.19083E-01 0.74298E+00 2800 0.225E+00 0.303E+00
DAV: 26 -0.744222388576E+03 -0.19919E-01 0.69976E+00 2848 0.230E+00 0.303E+00
DAV: 27 -0.744202442580E+03 0.19946E-01 0.74286E+00 2768 0.225E+00 0.303E+00
DAV: 28 -0.744221215434E+03 -0.18773E-01 0.70148E+00 2832 0.230E+00 0.304E+00
DAV: 29 -0.744201884237E+03 0.19331E-01 0.74401E+00 2768 0.225E+00 0.304E+00
DAV: 30 -0.744222781874E+03 -0.20898E-01 0.69924E+00 2880 0.230E+00 0.303E+00
DAV: 31 -0.744203093322E+03 0.19689E-01 0.74177E+00 2768 0.225E+00 0.303E+00
DAV: 32 -0.744221043999E+03 -0.17951E-01 0.70186E+00 2832 0.230E+00 0.304E+00
DAV: 33 -0.744201883191E+03 0.19161E-01 0.74407E+00 2768 0.225E+00 0.303E+00
DAV: 34 -0.744222870137E+03 -0.20987E-01 0.69908E+00 2880 0.229E+00 0.303E+00
DAV: 35 -0.744203014646E+03 0.19855E-01 0.74192E+00 2800 0.225E+00 0.303E+00
DAV: 36 -0.744220977886E+03 -0.17963E-01 0.70196E+00 2816 0.230E+00 0.304E+00
DAV: 37 -0.744201858723E+03 0.19119E-01 0.74413E+00 2768 0.225E+00 0.303E+00
DAV: 38 -0.744222911522E+03 -0.21053E-01 0.69901E+00 2880 0.229E+00 0.303E+00
DAV: 39 -0.744202996644E+03 0.19915E-01 0.74197E+00 2768 0.225E+00 0.303E+00
DAV: 40 -0.744221025909E+03 -0.18029E-01 0.70188E+00 2816 0.230E+00 0.304E+00
DAV: 41 -0.744201828524E+03 0.19197E-01 0.74418E+00 2768 0.225E+00 0.303E+00
DAV: 42 -0.744222914189E+03 -0.21086E-01 0.69899E+00 2896 0.229E+00 0.303E+00
DAV: 43 -0.744202976423E+03 0.19938E-01 0.74199E+00 2768 0.225E+00 0.303E+00
DAV: 44 -0.744221013840E+03 -0.18037E-01 0.70189E+00 2816 0.230E+00 0.304E+00
DAV: 45 -0.744201799953E+03 0.19214E-01 0.74422E+00 2800 0.225E+00 0.303E+00
DAV: 46 -0.744222927422E+03 -0.21127E-01 0.69896E+00 2880 0.229E+00 0.303E+00
DAV: 47 -0.744202970137E+03 0.19957E-01 0.74200E+00 2768 0.225E+00 0.303E+00
DAV: 48 -0.744221011589E+03 -0.18041E-01 0.70189E+00 2816 0.230E+00 0.304E+00
DAV: 49 -0.744201841598E+03 0.19170E-01 0.74416E+00 2768 0.225E+00 0.303E+00
DAV: 50 -0.744222989865E+03 -0.21148E-01 0.69885E+00 2880 0.229E+00
1 F= -.74422299E+03 E0= -.74422299E+03 d E =0.000000E+00 mag= 1.0000
writing wavefunctions

As you can see, it kept jumping between two states and didn't converge.

Then I followed the advice in https://www.vasp.at/vasp-workshop/optelectron.pdf, setting ICHARG=12. I also changed ISTART to 0. Still failed to converge. Like before, the energy jump back and forth. The last few steps in the log is

DAV: 41 -0.843977333173E+03 -0.42146E-01 0.10880E+01 2272 0.553E-01
DAV: 42 -0.843934628017E+03 0.42705E-01 0.11748E+01 2288 0.553E-01
DAV: 43 -0.843977435191E+03 -0.42807E-01 0.10875E+01 2240 0.552E-01
DAV: 44 -0.843933728401E+03 0.43707E-01 0.11763E+01 2224 0.553E-01
DAV: 45 -0.843977562619E+03 -0.43834E-01 0.10873E+01 2304 0.553E-01
DAV: 46 -0.843933852749E+03 0.43710E-01 0.11761E+01 2288 0.551E-01
DAV: 47 -0.843977537350E+03 -0.43685E-01 0.10871E+01 2288 0.552E-01
DAV: 48 -0.843933034564E+03 0.44503E-01 0.11776E+01 2256 0.551E-01
DAV: 49 -0.843977596320E+03 -0.44562E-01 0.10869E+01 2256 0.553E-01
DAV: 50 -0.843932925450E+03 0.44671E-01 0.11776E+01 2256 0.552E-01
1 F= -.84393293E+03 E0= -.84393293E+03 d E =0.000000E+00 mag= 96.0000
writing wavefunctions

Then the advice from the slide is 'bug report after positions have been checked'. So I guess the optimized structure I got might be unreasonable due to local minimum or the underestimated bang gap with PBE. So I tried to find another structure which can get my scf converged. Luckily, I found such a structure. The scf converged at this structure but the force on atoms are too large. Thus I tried to optimize it at HSE-level, using IBRION = 2 and POTIM = 0.1. But starting at the 4th ionic step, the scf failed to converge.

N E dE d eps ncg rms rms(c)
DAV: 1 -0.738744016805E+03 -0.13907E-02 -0.11172E+00 2432 0.150E+00 0.161E-01
DAV: 2 -0.738757926020E+03 -0.13909E-01 -0.50497E-02 3792 0.442E-01 0.105E-01
DAV: 3 -0.738758078576E+03 -0.15256E-03 -0.22602E-03 2752 0.110E-01 0.700E-02
DAV: 4 -0.738757968614E+03 0.10996E-03 -0.44806E-04 2976 0.745E-02 0.391E-02
DAV: 5 -0.738757969686E+03 -0.10726E-05 -0.28746E-04 2848 0.446E-02 0.224E-02
DAV: 6 -0.738757968878E+03 0.80884E-06 -0.59987E-05 3056 0.519E-02 0.121E-02
DAV: 7 -0.738757971699E+03 -0.28211E-05 -0.46679E-05 2992 0.188E-02 0.759E-03
DAV: 8 -0.738757972133E+03 -0.43437E-06 -0.86674E-06 3056 0.160E-02 0.532E-03
DAV: 9 -0.738757972164E+03 -0.30734E-07 -0.38566E-06 3024 0.931E-03 0.347E-03
DAV: 10 -0.738757972234E+03 -0.70228E-07 -0.74416E-07 2880 0.593E-03 0.246E-03
DAV: 11 -0.738757972216E+03 0.17884E-07 0.14261E-06 3024 0.407E-03 0.177E-03
DAV: 12 -0.738757972154E+03 0.61875E-07 0.33377E-06 2976 0.301E-03 0.144E-03
DAV: 13 -0.738757972070E+03 0.83921E-07 0.51772E-06 2816 0.188E-03 0.145E-03
DAV: 14 -0.738757971920E+03 0.15070E-06 0.80303E-06 2736 0.201E-03 0.162E-03
DAV: 15 -0.738757971709E+03 0.21023E-06 0.11935E-05 2736 0.135E-03 0.192E-03
DAV: 16 -0.738757971391E+03 0.31838E-06 0.17732E-05 2672 0.157E-03 0.232E-03
DAV: 17 -0.738757970927E+03 0.46352E-06 0.26282E-05 2352 0.159E-03 0.281E-03
DAV: 18 -0.738757970229E+03 0.69891E-06 0.38940E-05 2288 0.173E-03 0.342E-03
DAV: 19 -0.738757969208E+03 0.10202E-05 0.57656E-05 2336 0.204E-03 0.415E-03
DAV: 20 -0.738757967682E+03 0.15263E-05 0.85412E-05 2400 0.237E-03 0.505E-03
DAV: 21 -0.738757965441E+03 0.22411E-05 0.12641E-04 2448 0.281E-03 0.614E-03
DAV: 22 -0.738757962097E+03 0.33440E-05 0.18726E-04 2544 0.341E-03 0.747E-03
DAV: 23 -0.738757957188E+03 0.49092E-05 0.27701E-04 2672 0.405E-03 0.909E-03
DAV: 24 -0.738757949867E+03 0.73210E-05 0.41031E-04 2752 0.500E-03 0.110E-02
DAV: 25 -0.738757939128E+03 0.10739E-04 0.60653E-04 2816 0.593E-03 0.134E-02
DAV: 26 -0.738757923120E+03 0.16007E-04 0.89820E-04 2944 0.740E-03 0.163E-02
DAV: 27 -0.738757899706E+03 0.23414E-04 0.13259E-03 3040 0.869E-03 0.199E-02
DAV: 28 -0.738757864811E+03 0.34896E-04 0.19621E-03 3088 0.110E-02 0.241E-02
DAV: 29 -0.738757814097E+03 0.50714E-04 0.28889E-03 3136 0.127E-02 0.293E-02
DAV: 30 -0.738757738592E+03 0.75505E-04 0.42666E-03 3120 0.165E-02 0.356E-02
DAV: 31 -0.738757630199E+03 0.10839E-03 0.62495E-03 3136 0.186E-02 0.432E-02
DAV: 32 -0.738757469529E+03 0.16067E-03 0.91856E-03 3168 0.247E-02 0.521E-02
DAV: 33 -0.738757244240E+03 0.22529E-03 0.13317E-02 3168 0.270E-02 0.630E-02
DAV: 34 -0.738756914440E+03 0.32980E-03 0.19362E-02 3200 0.370E-02 0.755E-02
DAV: 35 -0.738756472191E+03 0.44225E-03 0.27512E-02 3184 0.387E-02 0.905E-02
DAV: 36 -0.738755842926E+03 0.62927E-03 0.39113E-02 3248 0.549E-02 0.107E-01
DAV: 37 -0.738755065758E+03 0.77717E-03 0.53568E-02 3232 0.545E-02 0.126E-01
DAV: 38 -0.738754016707E+03 0.10491E-02 0.73094E-02 3248 0.787E-02 0.145E-01
DAV: 39 -0.738752890904E+03 0.11258E-02 0.94343E-02 3200 0.737E-02 0.166E-01
DAV: 40 -0.738751475011E+03 0.14159E-02 0.12103E-01 3248 0.105E-01 0.185E-01
DAV: 41 -0.738750248957E+03 0.12261E-02 0.14463E-01 3136 0.931E-02 0.205E-01
DAV: 42 -0.738748769353E+03 0.14796E-02 0.17283E-01 3216 0.125E-01 0.221E-01
DAV: 43 -0.738747822562E+03 0.94679E-03 0.19147E-01 3136 0.109E-01 0.236E-01
DAV: 44 -0.738746571715E+03 0.12508E-02 0.21544E-01 3152 0.137E-01 0.247E-01
DAV: 45 -0.738746075936E+03 0.49578E-03 0.22555E-01 3152 0.122E-01 0.257E-01
DAV: 46 -0.738745121616E+03 0.95432E-03 0.24380E-01 3056 0.141E-01 0.264E-01
DAV: 47 -0.738745011314E+03 0.11030E-03 0.24648E-01 3056 0.131E-01 0.270E-01
DAV: 48 -0.738744287120E+03 0.72419E-03 0.26025E-01 3056 0.142E-01 0.273E-01
DAV: 49 -0.738744431031E+03 -0.14391E-03 0.25796E-01 3040 0.136E-01 0.276E-01
DAV: 50 -0.738743849087E+03 0.58194E-03 0.26892E-01 2960 0.142E-01
4 F= -.73874385E+03 E0= -.73874385E+03 d E =-.122301E-02 mag= 1.0000
trial-energy change: -0.001223 1 .order -0.014759 -0.018532 -0.010985
step: 3.9288(harm= 3.9288) dis= 0.01255 next Energy= -738.765379 (dE=-0.228E-01)
bond charge predicted
writing wavefunctions

In loop 13 it almost converged but finally not.

Now I have no idea what to do next. I was wondering if you would give me some advice.

Thanks for your patience. :))

Re: HSE06 scf fails to converge

Posted: Tue Sep 27, 2022 1:04 pm
by martin.schlipf
A couple of comments:
1) Please always submit a complete set of input and output files. Otherwise, we cannot reproduce your results and only guess what is happening.
2) You are not doing HSE calculations. HSE06 is defined as AEXX = 0.25, HFSCREEN = 0.2. Any other setting is your own hybrid functional.
3) Please read the section of the ISMEAR = -2 tag carefully and check whether the comments regarding LDIAG apply. For this look at the eigenvalues.
4) ALGO = All or Damped might be more robust for hybrid functionals.
5) Other than that you can looking at the mixing tags (AMIX, BMIX, ...) or try to gradually introduce the vacancy, but that requires a lot of manual tweaking.

One more thing: Did you try to reproduce the behavior in a smaller cell. It can drastically improve your time to solution if you don't always need to wait until a long calculation on a big cell finishes. Therefore it is almost always a good idea first to look for the minimal system that reproduces your problem.

Re: HSE06 scf fails to converge

Posted: Tue Sep 27, 2022 11:36 pm
by guanzhi_li
martin.schlipf wrote: Tue Sep 27, 2022 1:04 pm A couple of comments:
1) Please always submit a complete set of input and output files. Otherwise, we cannot reproduce your results and only guess what is happening.
2) You are not doing HSE calculations. HSE06 is defined as AEXX = 0.25, HFSCREEN = 0.2. Any other setting is your own hybrid functional.
3) Please read the section of the ISMEAR = -2 tag carefully and check whether the comments regarding LDIAG apply. For this look at the eigenvalues.
4) ALGO = All or Damped might be more robust for hybrid functionals.
5) Other than that you can looking at the mixing tags (AMIX, BMIX, ...) or try to gradually introduce the vacancy, but that requires a lot of manual tweaking.

One more thing: Did you try to reproduce the behavior in a smaller cell. It can drastically improve your time to solution if you don't always need to wait until a long calculation on a big cell finishes. Therefore it is almost always a good idea first to look for the minimal system that reproduces your problem.
Hi Martin,

Thanks so much for your comments and advice. The problem is solved by setting LDIAG = .FALSE.. It turns out that the eigenenergy of the defect goes down when it is occupied so the order of the orbitals changes. I switched to ALGO = Damped, and the SCF converged very fast. This is really a good lesson to me. I read VASP wiki before but never noticed the comments regarding orbital order and recommended ALGO for hybrid functional calculations. I went back to the wiki pages and read them carefully. In the page of LHFCALC, I noticed that one can reduce TIME if convergence is not obtained. So when my first attempt of Damped algorithm failed, I reduced TIME to 0.3, which works well.

Thank you again for your great advice. I benefited a lot. :D