VASP5.4.1 SLOW SPEED with intel compiler 14~ and OpenMPI

[hiromasa]

Hi,

I've encountered clearly SLOW speed of VASP 5.4.1(with patch #3),
when using intel compiler 14 and later and OpenMPI 1.6.5 and later.
No problem with VASP 5.3.5 or IntelMPI.
Does anyone have same experience ?
Would you share any advice and comments to resolve this?

Below are my performance data:

Input: 216 Si atoms, with PAW GGA
CPU: Intel Xeon E5-2698 v3
Number of MPI processes: 4

(a) VASP 5.4.1p3 / ic13.1.3 / OpenMPI 1.6.5 = 155 secs (good)
(b) VASP 5.4.1p3 / ic13.1.3 / OpenMPI 1.8.8 = 135 secs (good)
(c) VASP 5.4.1p3 / ic14.0.3 / OpenMPI 1.6.5 = 484 secs (SLOW!)
(d) VASP 5.4.1p3 / ic15.0.5 / OpenMPI 1.6.5 = 518 secs (SLOW!)
(e) VASP 5.4.1p3 / ic15.0.5 / OpenMPI 1.8.8 = 416 secs (SLOW!)
(f) VASP 5.4.1p3 / ic16.0.1 / OpenMPI 1.8.8 = 466 secs (SLOW!)
(g) VASP 5.4.1p3 / ic15.0.5 / IntelMPI 5.1.2 = 130 secs (good)
(h) VASP 5.3.5 / ic13.1.3 / OpenMPI 1.6.5 = 158 secs (good)
(i) VASP 5.3.5 / ic15.0.5 / OpenMPI 1.6.5 = 155 secs (good)
(j) VASP 5.3.5 / ic15.0.5 / OpenMPI 1.8.8 = 137 secs (good)
(k) VASP 5.3.5 / ic16.0.1 / IntelMPI 5.1.2 = 130 secs (good)

* When I try with other numbers of MPI processes(1,2,8,16,32), same SLOW speed is observed.

Thanks in advance,
HIROMASA

[hiromasa]

Hi,

Happily I've solved this problem myself. Here's the workaround.

In my VASP build cases with both OpenMPI and IntelMPI, I used ScaLAPACK of MKL.
When I tried NOT to use ScaLAPACK with OpenMPI, the speed became same as IntelMPI.
Since libmkl_scalapack_lp64 is used in common with both OpenMPI and IntelMPI,
the cause is BLACS in MKL, and only BLACS for IntelMPI works fine.

Thanks,
HIROMASA

[admin]

Thank you for sharing your opinion.