NCSHMEM - Revision history

Kresse: /* Related tags and articles */

2026-02-05T09:40:20Z

Kresse at 09:35, 5 February 2026

2026-02-05T09:35:49Z

← Older revision		Revision as of 09:35, 5 February 2026
Line 9:		Line 9:
	Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very significantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.		Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very significantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.
	Specifically, most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket, with slower communication between the NUMA domains.		Specifically, most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket, with slower communication between the NUMA domains.
	Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, ~~shmem~~ will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.		Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, memory will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.

	== Related tags and articles ==		== Related tags and articles ==

Kresse at 09:34, 5 February 2026

2026-02-05T09:34:43Z

← Older revision		Revision as of 09:34, 5 February 2026
Line 6:		Line 6:
	By default, shared memory is not used in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).		By default, shared memory is not used in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).
	To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.		To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.
	If NCSHMEM is set to -1, VASP will automatically determine how many ~~nodes~~ physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.		If NCSHMEM is set to -1, VASP will automatically determine how many MPI tasks physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node (and one core is assigned to each MPI task), NCSHMEM will default to 48 or 128, respectively.
	Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very significantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.		Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very significantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.
	Specifically, most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket, with slower communication between the NUMA domains.		Specifically, most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket, with slower communication between the NUMA domains.

Kresse at 09:33, 5 February 2026

2026-02-05T09:33:33Z

← Older revision		Revision as of 09:33, 5 February 2026
Line 1:		Line 1:
	{{TAGDEF\|NCSHMEM\|[integer]\|1 }}		{{TAGDEF\|NCSHMEM\|[integer]\|1 }}

	Description: ~~{{TAG\|NCSHMEM}}~~ Specifies the number of MPI ranks that share a single shared memory segment in the non-cubic scaling GW routines.		Description: Specifies the number of MPI ranks that share a single shared memory segment in the non-cubic scaling GW routines.

	----		----

Kresse at 09:33, 5 February 2026

2026-02-05T09:33:00Z

← Older revision		Revision as of 09:33, 5 February 2026
Line 1:		Line 1:
	{{TAGDEF\|NCSHMEM\|[integer]\|1 }}		{{TAGDEF\|NCSHMEM\|[integer]\|1 }}

	Description: {{TAG\|NCSHMEM}} ~~determines~~ the number of ~~compute cores sharing the~~ memory ~~in MPI~~ in the non-cubic scaling GW routines.		Description: {{TAG\|NCSHMEM}} Specifies the number of MPI ranks that share a single shared memory segment in the non-cubic scaling GW routines.

	----		----

Kresse at 09:27, 5 February 2026

2026-02-05T09:27:23Z

← Older revision		Revision as of 09:27, 5 February 2026
Line 7:		Line 7:
	To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.		To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.
	If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.		If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.
	Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very ~~signficantly~~ (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.		Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very significantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.
	~~Note that~~ most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket.		Specifically, most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket, with slower communication between the NUMA domains.
	Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, shmem will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.		Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, shmem will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.

Kresse at 09:26, 5 February 2026

2026-02-05T09:26:14Z

← Older revision		Revision as of 09:26, 5 February 2026
Line 7:		Line 7:
	To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.		To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.
	If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.		If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.
	Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines. In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.		Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines very signficantly (up to a factor 4). In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.
	Note that most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket.		Note that most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket.
	Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, shmem will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.		Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, shmem will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.

Kresse at 09:25, 5 February 2026

2026-02-05T09:25:12Z

← Older revision		Revision as of 09:25, 5 February 2026
Line 4:		Line 4:

	----		----
	By default no shared memory ~~MPI~~ is in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).		By default, shared memory is not used in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).
	To use shared memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.		To use shared-memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.
	If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.		If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.
	Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines. In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.		Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines. In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.

Kresse: More detailed description and discussion of NUMA domains

2026-02-05T09:24:08Z

More detailed description and discussion of NUMA domains

← Older revision		Revision as of 09:24, 5 February 2026
Line 5:		Line 5:
	----		----
	By default no shared memory MPI is in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).		By default no shared memory MPI is in the non-cubic scaling GW routines ({{TAG\|ALGO}}=EVGW, EVGW0, QPGW and QPGW0).
	~~For using~~ shared memory MPI in the non-cubic scaling GW set NCSHMEM=-1.		To use shared memory MPI in the non-cubic scaling GW, set NCSHMEM to -1 or a positive integer.
	~~{{NB\|warning\|~~For the ~~openmp version NCHSMEM~~ is ~~not active~~ and ~~shared~~ memory ~~is not used for MPI~~. In that case ~~the~~ memory is shared between ~~threads~~.}}		If NCSHMEM is set to -1, VASP will automatically determine how many nodes physically share memory and set NCSHMEM to that number. For instance, if the machine has 48 or 128 cores per node, NCSHMEM will default to 48 or 128, respectively.
			Note that larger numbers can degrade performance, particularly on NUMA architectures and multi socket machines. In this case, it is often necessary to manually decrease NCSHMEM to a value between 8 and 32 in order to strike a balance between memory savings and performance.
			Note that most EPYC HPC clusters are configured with a mode that divides the memory into four NUMA domains per socket.
			Therefore, if you use a machine with 64 cores per socket, each NUMA domain will consist of 16 cores and NCSHMEM=16 will yield the best performance (in this case, shmem will not be shared between NUMA domains). The performance penalty may be particularly significant if memory is shared between sockets.

	== Related tags and articles ==		== Related tags and articles ==
	[[Shared memory]], {{TAG\|ALGO}}, [[Practical guide to GW calculations]]		[[Shared memory]], {{TAG\|ALGO}}, [[Practical guide to GW calculations]]

	[[Category:Performance]][[Category:Many-body perturbation theory]][[Category:GW]][[Category:Memory]]		[[Category:Performance]][[Category:Many-body perturbation theory]][[Category:GW]][[Category:Memory]]

Karsai at 09:38, 21 July 2022

2022-07-21T09:38:06Z

← Older revision		Revision as of 09:38, 21 July 2022
Line 10:		Line 10:
	[[Shared memory]], {{TAG\|ALGO}}, [[Practical guide to GW calculations]]		[[Shared memory]], {{TAG\|ALGO}}, [[Practical guide to GW calculations]]

	[[Category:Performance]][[Category:~~VASP6~~]][[Category:~~Machine-learned force fields~~]][[Category:Memory]]		[[Category:Performance]][[Category:Many-body perturbation theory]][[Category:GW]][[Category:Memory]]

← Older revision		Revision as of 09:40, 5 February 2026
Line 12:		Line 12:

	== Related tags and articles ==		== Related tags and articles ==
	[[Shared memory]], {{TAG\|ALGO}}, [[Practical guide to GW calculations]]		[[Shared memory]], {{TAG\|ML_NCSHMEM}}, {{TAG\|ALGO}}, [[Practical guide to GW calculations]]

	[[Category:Performance]][[Category:Many-body perturbation theory]][[Category:GW]][[Category:Memory]]		[[Category:Performance]][[Category:Many-body perturbation theory]][[Category:GW]][[Category:Memory]]