Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

option, parallelism,...

Moderators: jbeuken, Jordan, pouillon

Forum rules
Please have a look at ~abinit/doc/config/build-config.ac in the source package for detailed and up-to-date information about the configuration of Abinit 8 builds.
For a video explanation on how to build Abinit 7.x for Linux, please go to: http://www.youtube.com/watch?v=DppLQ-KQA68.
IMPORTANT: when an answer solves your problem, please check the little green V-like button on its upper-right corner to accept it.
Post Reply
mhoffmann84
Posts: 3
Joined: Tue Jul 07, 2020 9:31 am
Location: Linz, Austria
Contact:

Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by mhoffmann84 » Tue Jul 07, 2020 9:44 am

Hi there,
I was playing around with the libraries and configure on my workstation for abinit9.0.4. and finally was happy that configure states that HDF5, netcdf and netcdf-fortran have a "yes" in the last table with MPI support.

Now I started a quick calculation and the code itself states that those libraries are not parallel.

Any idea what I could have missed?
config command and part of output ist below.

Thanks a lot.
BR
Martin

install libxc from gitlab.com to ~/libxc
with version 4.2.3
git checkout 4.2.3

./configure --config-cache --with-mpi --enable-openmp --enable-mpi-io --with-fallbacks \
--with-libxc=/home/hoffmann/libxc \
--with-hdf5 \
--with-netcdf \
--with-netcdf-fortran \
--with-fftw3 FFTW3_LIBS='-L/usr/lib -lsfftw -lfftw3 -lfftw3f -lfftw3_mpi -lfftw3_threads' \
FFTW3_CPPFLAGS="-I/usr/include" FFTW3_FCFLAGS="-I/usr/include" \
LINALG_LIBS='-L/usr/lib -lblas -llapack' \
FC=mpif90 CC=mpicc CXX=mpicxx \
--prefix=/home/hoffmann/code/

in ./configure run
Core build parameters
---------------------

* C compiler : gnu version 7.5
* Fortran compiler : gnu version 7.5
* architecture : unknown unknown (64 bits)
* debugging : basic
* optimizations : standard

* OpenMP enabled : yes (collapse: yes)
* MPI enabled : yes (flavor: auto)
* MPI in-place : no
* MPI-IO enabled : yes
* GPU enabled : no (flavor: none)

* LibXML2 enabled : no
* HDF5 enabled : yes (MPI support: yes)
* NetCDF enabled : yes (MPI support: yes)
* NetCDF-F enabled : yes (MPI support: yes)

* FFT flavor : fftw3-threads (libs: auto-detected)
* LINALG flavor : netlib (libs: auto-detected)

* Build workflow : monolith

User avatar
jbeuken
Posts: 346
Joined: Tue Aug 18, 2009 9:24 pm
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by jbeuken » Wed Jul 08, 2020 12:38 pm

Hi Martin,

which OS ?
if ubuntu, output of :

Code: Select all

dpkg -l | egrep -E "openmpi|hdf5|netcdf"
------
Jean-Michel Beuken
Computer Scientist

mhoffmann84
Posts: 3
Joined: Tue Jul 07, 2020 9:31 am
Location: Linz, Austria
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by mhoffmann84 » Wed Jul 08, 2020 1:35 pm

Dear jbeuken,
thanks for the quick reply. Yes it is ubuntu.

Here is the output:
ii hdf5-helpers 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - Helper tools
ii libhdf5-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - serial version
ii libhdf5-cpp-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - C++ libraries
ii libhdf5-dev 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - serial version
ii libhdf5-openmpi-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - OpenMPI version
ii libhdf5-openmpi-dev 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - OpenMPI version
ii libnetcdf-c++4-1 4.3.0+ds-5 amd64 C++ interface for scientific data access to large binary data
ii libnetcdf-c++4-dev 4.3.0+ds-5 amd64 creation, access, and sharing of scientific data in C++
ii libnetcdf-dev 1:4.6.0-2build1 amd64 creation, access, and sharing of scientific data
ii libnetcdf13:amd64 1:4.6.0-2build1 amd64 Interface for scientific data access to large binary data
ii libnetcdff-dev 4.4.4+ds-3 amd64 creation, access, and sharing of scientific data in Fortran
ii libnetcdff6 4.4.4+ds-3 amd64 Fortran interface for scientific data access to large binary data
ii libopenmpi-dev 2.1.1-8 amd64 high performance message passing library -- header files
ii libopenmpi2:amd64 2.1.1-8 amd64 high performance message passing library -- shared library
ii libpnetcdf-dev:amd64 1.9.0-2 amd64 Development files for the parallel netCDF library
ii libpnetcdf0d:amd64 1.9.0-2 amd64 Libraryfor reading and writing parallel NetCDF files
ii openmpi-bin 2.1.1-8 amd64 high performance message passing library -- binaries
ii openmpi-common 2.1.1-8 all high performance message passing library -- common files

Thanks again.

Best Regards
Martin

User avatar
jbeuken
Posts: 346
Joined: Tue Aug 18, 2009 9:24 pm
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by jbeuken » Wed Jul 08, 2020 4:09 pm

With all packages installed, ABINIT's build system (BS) detects serial and parallel installed versions of HDF5 and netCDF .

Unfortunately, the design of the packages is badly configured... :cry:

In addition, there are toxic dependencies... :cry:
e.g., installation of parallel netCDF implies installation of HDF5 serial !?! :o
You can see it with:

Code: Select all

ldd src/98_main/abinit
I've never been able to get a functional configuration using only ubuntu packages ( under 18.04, 19.10 and 20.04 ) with HDF5 parallel ( ok with HDF5 serial )

Even Ubuntu's openmpi stack is not correcte : mpi-io support is not complete...

If you really need parallel HDF5, you have to compile from source including OpenMPI 4.
I have functional recipes for building OpenMPI 4, HDF5 // , netCDF(-F) and libXC with GNU or Intel compilers.

For the mandored packages (HDF5, netCDF(-F) and libXC), you can use the tarball in the
abinit-9.1.4/fallbacks/
folder and run the script:

Code: Select all

./build-abinit-fallbacks.sh
I need to find time to write a doc
------
Jean-Michel Beuken
Computer Scientist

Post Reply