Requests for technical support from the VASP group should be posted in the VASP-forum.

A CentOS based installation of VASP

From Vaspwiki
Jump to navigationJump to search

The following guide was written for CentOS 7 installed in virtualbox 5. No steps regarding performance has been taken. Linking to optimized libraries (especially the FFT libraries) can result in a speed-up by a factor 2 to 3.

Install following packages in CentOS:

# yum install gcc-gfortran gcc-g++ lapack-devel fftw-devel openmpi3-devel wget rsync

After this, make sure that the Fortran MPI wrapper "mpif90" is added to your PATH, by executing

$ export PATH=$PATH:/usr/lib64/openmpi3/bin

Test if it's working:

$ mpif90 --version
GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)
Copyright (C) 2015 Free Software Foundation, Inc.

If you don't see similar lines, then you have to check your installation or maybe correct the PATH variable.

Make the changes to the PATH variable permanently for the user:

$ echo "export PATH=$PATH:/usr/lib64/openmpi3/bin" > ~/.bashrc 

Unfortunately, CentOS 7 does not provide packages for ScaLAPACK, so you either have an alternative ScaLAPACK library present, or you compile this library from scratch.

The following steps describe how to compile ScaLAPACK from scratch. First pull the source code from the netlib website

[~/src]$ wget https://www.netlib.org/scalapack/scalapack-2.0.2.tgz .

Extract the package and use the makefile template to build the library

[~/src]$ tar -xvf scalapack-2.0.2.tgz ; cd scalapack-2.0.2
[~/src/scalapack-2.0.2]$ cp SLmake.inc.example SLmake.inc

The file SLmake.inc has to be adapted to point to the correct LAPACK library. In the following it is assumed that the LAPACK libraries provided in the CentOS repositories are used. For this purpose change the last lines in SLmake.inc containing the variables BLASLIB and LPACKLIB into

BLASLIB = -L/usr/lib64 -lblas
LAPACKLIB = -L/usr/lib64 -llapack

and compile the library:

[~/src/scalapack-2.0.2]$ make lib

A successful compilation should write a file "libscalapack.a" into the same directory where SLmake.inc is located. In our case the path of the file is "~/src/scalapack-2.0.2/libscalapack.a"

Next, the actual compilation of vasp-5.4.4 is discussed. We assume that the tarball "vasp-5.4.4.tar.gz" has been downloaded from the online portal and is located in "~/src/"

[~/src]$ tar -xvf  vasp-5.4.4.tar.gz ; cd vasp.5.4.4

For a successful compilation, the correct library paths have to be set in a file "~/src/vasp.5.4.4/makefile.include". In our case the "makefile.include" looks as follows:

# Precompiler options
CPP_OPTIONS= -DHOST=\"LinuxGNU\" \
             -DMPI -DMPI_BLOCK=8000 \
             -Duse_collective \
             -DscaLAPACK \
             -DCACHE_SIZE=4000 \
             -Davoidalloc \
             -Duse_bse_te \
             -Dtbdyn

CPP        = gcc -E -P -C -w $*$(FUFFIX) >$*$(SUFFIX) $(CPP_OPTIONS)

FC         = mpif90
FCL        = mpif90

FREE       = -ffree-form -ffree-line-length-none

FFLAGS     = -w
OFLAG      = -O2
OFLAG_IN   = $(OFLAG)
DEBUG      = -O0 

LIBDIR     = /usr/lib64
BLAS       = -L$(LIBDIR) -lblas
LAPACK     = -L$(LIBDIR) -llapack
SCALAPACK  = -L/home/user/src/scalapack-2.0.2 -lscalapack

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)

LLIBS      += -L$(LIBDIR) -lfftw3
INCS       = -I/usr/include

OBJECTS    = fftmpiw.o fftmpi_map.o  fftw3d.o  fft3dlib.o

OBJECTS_O1 += fftw3d.o fftmpi.o fftmpiw.o
OBJECTS_O2 += fft3dlib.o 

# For what used to be vasp.5.lib
CPP_LIB    = $(CPP)
FC_LIB     = $(FC)
CC_LIB     = gcc
CFLAGS_LIB = -O
FFLAGS_LIB = -O1
FREE_LIB   = $(FREE)

OBJECTS_LIB= linpack_double.o getshmem.o 

# For the parser library
CXX_PARS   = g++ 

LIBS       += parser
LLIBS      += -Lparser -lparser -lstdc++

# Normally no need to change this
SRCDIR     = ../../src
BINDIR     = ../../bin

Note that the line

LLIBS      = $(SCALAPACK) $(LAPACK) $(BLAS)

This variable LLIBS is used by the fortran linker and depends on the variables SCALAPACK, LAPACK and BLAS which are defined as

LIBDIR     = /usr/lib64
BLAS       = -L$(LIBDIR) -lblas
LAPACK     = -L$(LIBDIR) -llapack
SCALAPACK  = -L/home/user/src/scalapack-2.0.2 -lscalapack

The definition of the BLAS and LAPACK variables here mean that the library files "libblas.so" (shorthand is -lblas) and "liblapack.so" (shorthand -llapack) are located in the directory LIBDIR=/usr/lib64:

[~/src/vasp.5.4.4]$ ls -l /usr/lib64/libblas.so /usr/lib64/liblapack.so
lrwxrwxrwx. 1 root root Aug 26 06:41 /usr/lib64/libblas.so ->  libblas.so.3.4.2
lrwxrwxrwx. 1 root root Aug 26 06:41 /usr/lib64/liblapack.so -> liblapack.so.3.4.2

If you have compiled ScaLAPACK by hand as described above, you have used the same library files and created the scalapack library in "~/src/scalapack-2.0.2/libscalapack.a". The path to the scalapack library must be correct, otherwise the compilation of VASP will fail.

To compile vasp execute following command in "~/src/vasp.5.4.4/"

[~/src/vasp.5.4.4/]$ make std gam ncl

After successful compilation, three binaries of vasp can be found in "~/src/vasp.5.4.4/bin", namely the standard full k-point version "vasp_std", the gamma-only version "vasp_gam" and the non-collinear version "vasp_ncl".

Running vasp in parallel with, say 4 MPI ranks, is done using

$ mpirun -np 4 /path/to/vasp_std

Here mpirun should be in the PATH directory (as mpif90 is), except you use an alternative MPI wrapper.