Not enough memory - Revision history

Huebsch at 09:10, 25 May 2022

2022-05-25T09:10:39Z

← Older revision		Revision as of 09:10, 25 May 2022
Line 24:		Line 24:

	----		----
	[[Category:Performance]][[Category:Howto]]		[[Category:Performance]][[Category:Howto]][[Category:Memory]]

Vaspmaster at 09:50, 3 November 2020

2020-11-03T09:50:37Z

← Older revision		Revision as of 09:50, 3 November 2020
Line 6:		Line 6:
	*For large and many-atom systems, it is advised to increase {{TAG\|NCORE}} to larger values (say to 4, 8 potentially to or even beyond the number of cores per node). This allows to decrease the memory requirements per core for the storage of the non-local projectors. Furthermore, real space projection, {{TAG\|LREAL}}= A, also decreases the required memory per core.		*For large and many-atom systems, it is advised to increase {{TAG\|NCORE}} to larger values (say to 4, 8 potentially to or even beyond the number of cores per node). This allows to decrease the memory requirements per core for the storage of the non-local projectors. Furthermore, real space projection, {{TAG\|LREAL}}= A, also decreases the required memory per core.
	*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).		*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From ~~vasp~~.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).		*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From VASP.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).
	* Make sure to use scaLAPACK if your system becomes large. If scaLAPACK is not available, VASP needs to store an {{TAG\|NBANDS}} x {{TAG\|NBANDS}} matrix on each core, in order to diagonalize the Hamiltonian in the subspace of the calculated orbitals. If scaLAPACK is compiled in and used, the matrix is distributed over all cores jointly handling one k-point. Note that decreasing {{TAG\|KPAR}} reduces the memory demand for this matrix (if scaLAPACK is used).		* Make sure to use scaLAPACK if your system becomes large. If scaLAPACK is not available, VASP needs to store an {{TAG\|NBANDS}} x {{TAG\|NBANDS}} matrix on each core, in order to diagonalize the Hamiltonian in the subspace of the calculated orbitals. If scaLAPACK is compiled in and used, the matrix is distributed over all cores jointly handling one k-point. Note that decreasing {{TAG\|KPAR}} reduces the memory demand for this matrix (if scaLAPACK is used).

Line 20:		Line 20:
	wavefun : 331264. kBytes		wavefun : 331264. kBytes

	and inspect how much memory VASP uses per core. "~~Base~~" is the estimated memory use for the executable and libraries, "nonlr-proj" the required memory for the non-local projection operators, "grid" that for 3D arrays representing the charge density, potentials, etc., and wavefun the requirements for the one-electron wavefunctions (orbitals). Storage of the orbitals is usually most memory demanding.		and inspect how much memory VASP uses per core. "base" is the estimated memory use for the executable and libraries, "nonlr-proj" the required memory for the non-local projection operators, "grid" that for 3D arrays representing the charge density, potentials, etc., and "wavefun" the requirements for the one-electron wavefunctions (orbitals). Storage of the orbitals is usually most memory demanding.


	----		----
	[[Category:Performance]][[Category:Howto]]		[[Category:Performance]][[Category:Howto]]

Kresse at 08:16, 3 November 2020

2020-11-03T08:16:53Z

← Older revision		Revision as of 08:16, 3 November 2020
Line 7:		Line 7:
	*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).		*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From vasp.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).		*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From vasp.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).
	* Make sure to use scaLAPACK if your ~~systems become~~ large. If scaLAPACK is not available, VASP needs to store a {{TAG\|NBANDS}} x {{TAG\|NBANDS}} matrix on each core, in order to diagonalize the Hamiltonian in the subspace of the calculated orbitals. If scaLAPACK is compiled in and used, the matrix is distributed over all cores jointly handling one k-point. Note that decreasing {{TAG\|KPAR}}		* Make sure to use scaLAPACK if your system becomes large. If scaLAPACK is not available, VASP needs to store an {{TAG\|NBANDS}} x {{TAG\|NBANDS}} matrix on each core, in order to diagonalize the Hamiltonian in the subspace of the calculated orbitals. If scaLAPACK is compiled in and used, the matrix is distributed over all cores jointly handling one k-point. Note that decreasing {{TAG\|KPAR}} reduces the memory demand for this matrix (if scaLAPACK is used).
	~~also~~ reduces the memory demand for this matrix (if scaLAPACK is used).

	A final hint is in place. At some key places the code ~~writes out~~ the required memory per core. Please search		A final hint is in place. At some key places, the VASP code reports the required memory per core in the {{TAG\|OUTCAR}} file. Please search the lines
	the lines

	total amount of memory used by VASP MPI-rank0 457796. kBytes		total amount of memory used by VASP MPI-rank0 457796. kBytes
Line 22:		Line 20:
	wavefun : 331264. kBytes		wavefun : 331264. kBytes

	and inspect how much memory VASP uses per core. "Base" is the estimated memory use for the executable and libraries, "nonlr-proj" the required memory for the non-local projection operators, "grid" that for 3D arrays representing the charge density, potentials, etc., and wavefun the requirements for the one-electron wavefunctions (orbitals). Storage of the orbitals is usually most memory		and inspect how much memory VASP uses per core. "Base" is the estimated memory use for the executable and libraries, "nonlr-proj" the required memory for the non-local projection operators, "grid" that for 3D arrays representing the charge density, potentials, etc., and wavefun the requirements for the one-electron wavefunctions (orbitals). Storage of the orbitals is usually most memory demanding.
	demanding.


	----		----
	[[Category:Performance]][[Category:Howto]]		[[Category:Performance]][[Category:Howto]]

Kresse at 08:13, 3 November 2020

2020-11-03T08:13:01Z

← Older revision		Revision as of 08:13, 3 November 2020
Line 7:		Line 7:
	*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).		*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From vasp.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).		*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From vasp.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).
			* Make sure to use scaLAPACK if your systems become large. If scaLAPACK is not available, VASP needs to store a {{TAG\|NBANDS}} x {{TAG\|NBANDS}} matrix on each core, in order to diagonalize the Hamiltonian in the subspace of the calculated orbitals. If scaLAPACK is compiled in and used, the matrix is distributed over all cores jointly handling one k-point. Note that decreasing {{TAG\|KPAR}}
			also reduces the memory demand for this matrix (if scaLAPACK is used).

	A final hint is in place. At some key places the code writes out the required memory per core. Please search		A final hint is in place. At some key places the code writes out the required memory per core. Please search

Kresse at 08:09, 3 November 2020

2020-11-03T08:09:09Z

← Older revision		Revision as of 08:09, 3 November 2020
Line 3:		Line 3:


	~~The~~ following steps can be taken in order to reduce the memory requirements per core.		If memory shortage is encountered, the following steps can be taken in order to reduce the memory requirements per core.
	*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of cores per node ~~or socket~~). This allows to decrease the memory requirements per core ~~in particular~~ for the storage of the non-local projectors. Furthermore, real space projection {{TAG\|LREAL}}= A decreases the required memory per core.		*For large and many-atom systems, it is advised to increase {{TAG\|NCORE}} to larger values (say to 4, 8 potentially to or even beyond the number of cores per node). This allows to decrease the memory requirements per core for the storage of the non-local projectors. Furthermore, real space projection, {{TAG\|LREAL}}= A, also decreases the required memory per core.
	*{{TAG\|~~KAR~~}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|~~KAR~~}}=1 results in the smallest memory ~~requirements~~ per core (but ~~the slowest~~ calculations, since VASP needs to rely on other less efficient parallelization strategies).		*{{TAG\|KPAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KPAR}}=1 results in the smallest memory footprint per core (but slower calculations, since VASP needs to rely on other less efficient parallelization strategies).
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). ~~Symmetrisation~~ is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) ~~have~~ a switch to run a more memory conserving symmetrization. ~~This can be selected by specifying~~ {{TAG\|ISYM}}=2. Results might ~~however~~ differ ~~somewhat~~ from {{TAG\|ISYM}}=1 (usually ~~only 1/100th of an meV). Also avoid writing or reading the {{TAG\|CHGCAR}} file ({{TAG\|LCHARG}}=''.FALSE.''~~).		*Switch of symmetrisation ({{TAG\|ISYM}}=0). Charge symmetrisation is done locally on each node requiring three fairly large arrays. VASP.4.4.2 (and newer versions) posses a switch to run a more memory conserving symmetrization. From vasp.5 onwards, the memory conserving version, {{TAG\|ISYM}}=2, is the default. Results might differ slightly from {{TAG\|ISYM}}=1 (usually by about 1E-5 eV).

	A final hint is in place. At some key places the code writes out the required memory per core. Please search		A final hint is in place. At some key places the code writes out the required memory per core. Please search
	the lines		the lines

	total amount of memory used by VASP MPI-rank0		total amount of memory used by VASP MPI-rank0 457796. kBytes
			=======================================================================
			base : 30000. kBytes
			nonlr-proj: 12085. kBytes
			fftplans : 29652. kBytes
			grid : 54584. kBytes
			one-center: 211. kBytes
			wavefun : 331264. kBytes

	and inspect how much memory VASP uses per core.		and inspect how much memory VASP uses per core. "Base" is the estimated memory use for the executable and libraries, "nonlr-proj" the required memory for the non-local projection operators, "grid" that for 3D arrays representing the charge density, potentials, etc., and wavefun the requirements for the one-electron wavefunctions (orbitals). Storage of the orbitals is usually most memory
			demanding.


	----		----
	[[Category:Performance]][[Category:Howto]]		[[Category:Performance]][[Category:Howto]]

Kresse at 17:46, 2 November 2020

2020-11-02T17:46:36Z

← Older revision		Revision as of 17:46, 2 November 2020
Line 5:		Line 5:
	The following steps can be taken in order to reduce the memory requirements per core.		The following steps can be taken in order to reduce the memory requirements per core.
	*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of cores per node or socket). This allows to decrease the memory requirements per core in particular for the storage of the non-local projectors. Furthermore, real space projection {{TAG\|LREAL}}= A decreases the required memory per core.		*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of cores per node or socket). This allows to decrease the memory requirements per core in particular for the storage of the non-local projectors. Furthermore, real space projection {{TAG\|LREAL}}= A decreases the required memory per core.
	*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are		*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are distributed, but the storage of the orbitals is not distributed over cores. This means that using {{TAG\|KAR}}=1 results in the smallest memory requirements per core (but the slowest calculations, since VASP needs to rely on other less efficient parallelization strategies).
	distributed, but the storage of the orbitals is not distributed over cores. This means that using
	{{TAG\|KAR}}=1 results in the smallest memory requirements per core (but the slowest calculations, since
	VASP needs to rely on other less efficient parallelization strategies).
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying {{TAG\|ISYM}}=2. Results might however differ somewhat from {{TAG\|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG\|CHGCAR}} file ({{TAG\|LCHARG}}=''.FALSE.'').		*Switch of symmetrisation ({{TAG\|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying {{TAG\|ISYM}}=2. Results might however differ somewhat from {{TAG\|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG\|CHGCAR}} file ({{TAG\|LCHARG}}=''.FALSE.'').

Kresse at 17:46, 2 November 2020

2020-11-02T17:46:14Z

← Older revision		Revision as of 17:46, 2 November 2020
Line 4:		Line 4:

	The following steps can be taken in order to reduce the memory requirements per core.		The following steps can be taken in order to reduce the memory requirements per core.
	*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of		*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of cores per node or socket). This allows to decrease the memory requirements per core in particular for the storage of the non-local projectors. Furthermore, real space projection {{TAG\|LREAL}}= A decreases the required memory per core.
	cores per node or socket). This allows to decrease the memory
	requirements per core in particular for the storage of the non-local projectors. Furthermore, real space projection
	{{TAG\|LREAL}}= A decreases the required memory per core.
	*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are		*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are
	distributed, but the storage of the orbitals is not distributed over cores. This means that using		distributed, but the storage of the orbitals is not distributed over cores. This means that using

Kresse at 17:45, 2 November 2020

2020-11-02T17:45:47Z

← Older revision		Revision as of 17:45, 2 November 2020
Line 3:		Line 3:


	The following ~~things~~ can be ~~tried~~ to reduce the memory requirements per core.		The following steps can be taken in order to reduce the memory requirements per core.
	*Switch of symmetrisation ({{TAG\|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying {{TAG\|ISYM}}=2. Results might however differ somewhat from {{TAG\|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG\|CHGCAR}} file ({{TAG\|LCHARG}}=''.FALSE.'').		*For large many atom systems, increase {{TAG\|NCORE}} to larger values (say to 4, 8 or even the number of
	*For large many ~~atoms~~ systems, increase {{TAG\|NCORE}} to larger values. This allows to decrease the memory		cores per node or socket). This allows to decrease the memory
	requirements per core in ~~order to store~~ the non-local projectors. Furthermore, real space projection		requirements per core in particular for the storage of the non-local projectors. Furthermore, real space projection
	{{TAG\|LREAL}}= A decreases the required memory per core.		{{TAG\|LREAL}}= A decreases the required memory per core.
	*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are		*{{TAG\|KAR}} allows to distribute the k-points over cores. Unfortunately, only the calculations are
Line 12:		Line 12:
	{{TAG\|KAR}}=1 results in the smallest memory requirements per core (but the slowest calculations, since		{{TAG\|KAR}}=1 results in the smallest memory requirements per core (but the slowest calculations, since
	VASP needs to rely on other less efficient parallelization strategies).		VASP needs to rely on other less efficient parallelization strategies).
			*Switch of symmetrisation ({{TAG\|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying {{TAG\|ISYM}}=2. Results might however differ somewhat from {{TAG\|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG\|CHGCAR}} file ({{TAG\|LCHARG}}=''.FALSE.'').

	A final hint is in place. At some key places the code writes out the required memory per core. Please search		A final hint is in place. At some key places the code writes out the required memory per core. Please search

Kresse at 17:43, 2 November 2020

2020-11-02T17:43:57Z

@@ Line 1: / Line 1: @@
-Nowadays, for standard DFT and hybrid functional calculations,  memory is not an issues.
+Nowadays, for standard DFT and hybrid functional calculations,  memory is usually not an issues.
 Furthermore, by increasing the number of cores, the memory requirements per core can be reduced significantly.
@@ Line 5: / Line 5: @@
 The following things can be tried to reduce the memory requirements per core.
 *Switch of symmetrisation ({{TAG|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying  {{TAG|ISYM}}=2. Results might however differ somewhat from  {{TAG|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG|CHGCAR}} file ({{TAG|LCHARG}}=''.FALSE.'').
-*Use {{TAG|NPAR}}=1.
+*For large many atoms systems, increase {{TAG|NCORE}} to larger values. This allows to decrease the memory
 ----
 [[Category:Performance]][[Category:Howto]]

Kresse at 17:32, 2 November 2020

2020-11-02T17:32:59Z

@@ Line 1: / Line 1: @@
-First of all, the memory requirements of the serial version
+Nowadays, for standard DFT and hybrid functional calculations,  memory is not an issues.
-can be estimated using the ''makeparam'' utility (see
+Furthermore, by increasing the number of cores, the memory requirements per core can be reduced significantly.
-The following things can be tried to reduce the memory
+The following things can be tried to reduce the memory requirements per core.
-requirements on each node.
+*Switch of symmetrisation ({{TAG|ISYM}}=0). Symmetrisation is done locally on each node requiring three fairly arrays. VASP.4.4.2 (and newer versions) have a switch to run a more memory conserving symmetrization. This can be selected by specifying  {{TAG|ISYM}}=2. Results might however differ somewhat from  {{TAG|ISYM}}=1 (usually only 1/100th of an meV). Also avoid writing or reading the {{TAG|CHGCAR}} file ({{TAG|LCHARG}}=''.FALSE.'').
 *Use {{TAG|NPAR}}=1.