Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

option, parallelism,...

Moderators: fgoudreault, mcote

Forum rules
Please have a look at ~abinit/doc/config/build-config.ac in the source package for detailed and up-to-date information about the configuration of Abinit 8 builds.
For a video explanation on how to build Abinit 7.x for Linux, please go to: http://www.youtube.com/watch?v=DppLQ-KQA68.
IMPORTANT: when an answer solves your problem, please check the little green V-like button on its upper-right corner to accept it.
Locked
mhoffmann84
Posts: 3
Joined: Tue Jul 07, 2020 9:31 am
Location: Linz, Austria
Contact:

Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by mhoffmann84 » Tue Jul 07, 2020 9:44 am

Hi there,
I was playing around with the libraries and configure on my workstation for abinit9.0.4. and finally was happy that configure states that HDF5, netcdf and netcdf-fortran have a "yes" in the last table with MPI support.

Now I started a quick calculation and the code itself states that those libraries are not parallel.

Any idea what I could have missed?
config command and part of output ist below.

Thanks a lot.
BR
Martin

install libxc from gitlab.com to ~/libxc
with version 4.2.3
git checkout 4.2.3

./configure --config-cache --with-mpi --enable-openmp --enable-mpi-io --with-fallbacks \
--with-libxc=/home/hoffmann/libxc \
--with-hdf5 \
--with-netcdf \
--with-netcdf-fortran \
--with-fftw3 FFTW3_LIBS='-L/usr/lib -lsfftw -lfftw3 -lfftw3f -lfftw3_mpi -lfftw3_threads' \
FFTW3_CPPFLAGS="-I/usr/include" FFTW3_FCFLAGS="-I/usr/include" \
LINALG_LIBS='-L/usr/lib -lblas -llapack' \
FC=mpif90 CC=mpicc CXX=mpicxx \
--prefix=/home/hoffmann/code/

in ./configure run
Core build parameters
---------------------

* C compiler : gnu version 7.5
* Fortran compiler : gnu version 7.5
* architecture : unknown unknown (64 bits)
* debugging : basic
* optimizations : standard

* OpenMP enabled : yes (collapse: yes)
* MPI enabled : yes (flavor: auto)
* MPI in-place : no
* MPI-IO enabled : yes
* GPU enabled : no (flavor: none)

* LibXML2 enabled : no
* HDF5 enabled : yes (MPI support: yes)
* NetCDF enabled : yes (MPI support: yes)
* NetCDF-F enabled : yes (MPI support: yes)

* FFT flavor : fftw3-threads (libs: auto-detected)
* LINALG flavor : netlib (libs: auto-detected)

* Build workflow : monolith

User avatar
jbeuken
Posts: 365
Joined: Tue Aug 18, 2009 9:24 pm
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by jbeuken » Wed Jul 08, 2020 12:38 pm

Hi Martin,

which OS ?
if ubuntu, output of :

Code: Select all

dpkg -l | egrep -E "openmpi|hdf5|netcdf"
------
Jean-Michel Beuken
Computer Scientist

mhoffmann84
Posts: 3
Joined: Tue Jul 07, 2020 9:31 am
Location: Linz, Austria
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by mhoffmann84 » Wed Jul 08, 2020 1:35 pm

Dear jbeuken,
thanks for the quick reply. Yes it is ubuntu.

Here is the output:
ii hdf5-helpers 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - Helper tools
ii libhdf5-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - serial version
ii libhdf5-cpp-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - C++ libraries
ii libhdf5-dev 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - serial version
ii libhdf5-openmpi-100:amd64 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - OpenMPI version
ii libhdf5-openmpi-dev 1.10.0-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - OpenMPI version
ii libnetcdf-c++4-1 4.3.0+ds-5 amd64 C++ interface for scientific data access to large binary data
ii libnetcdf-c++4-dev 4.3.0+ds-5 amd64 creation, access, and sharing of scientific data in C++
ii libnetcdf-dev 1:4.6.0-2build1 amd64 creation, access, and sharing of scientific data
ii libnetcdf13:amd64 1:4.6.0-2build1 amd64 Interface for scientific data access to large binary data
ii libnetcdff-dev 4.4.4+ds-3 amd64 creation, access, and sharing of scientific data in Fortran
ii libnetcdff6 4.4.4+ds-3 amd64 Fortran interface for scientific data access to large binary data
ii libopenmpi-dev 2.1.1-8 amd64 high performance message passing library -- header files
ii libopenmpi2:amd64 2.1.1-8 amd64 high performance message passing library -- shared library
ii libpnetcdf-dev:amd64 1.9.0-2 amd64 Development files for the parallel netCDF library
ii libpnetcdf0d:amd64 1.9.0-2 amd64 Libraryfor reading and writing parallel NetCDF files
ii openmpi-bin 2.1.1-8 amd64 high performance message passing library -- binaries
ii openmpi-common 2.1.1-8 all high performance message passing library -- common files

Thanks again.

Best Regards
Martin

User avatar
jbeuken
Posts: 365
Joined: Tue Aug 18, 2009 9:24 pm
Contact:

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by jbeuken » Wed Jul 08, 2020 4:09 pm

With all packages installed, ABINIT's build system (BS) detects serial and parallel installed versions of HDF5 and netCDF .

Unfortunately, the design of the packages is badly configured... :cry:

In addition, there are toxic dependencies... :cry:
e.g., installation of parallel netCDF implies installation of HDF5 serial !?! :o
You can see it with:

Code: Select all

ldd src/98_main/abinit
I've never been able to get a functional configuration using only ubuntu packages ( under 18.04, 19.10 and 20.04 ) with HDF5 parallel ( ok with HDF5 serial )

Even Ubuntu's openmpi stack is not correcte : mpi-io support is not complete...

If you really need parallel HDF5, you have to compile from source including OpenMPI 4.
I have functional recipes for building OpenMPI 4, HDF5 // , netCDF(-F) and libXC with GNU or Intel compilers.

For the mandored packages (HDF5, netCDF(-F) and libXC), you can use the tarball in the
abinit-9.1.4/fallbacks/
folder and run the script:

Code: Select all

./build-abinit-fallbacks.sh
I need to find time to write a doc
------
Jean-Michel Beuken
Computer Scientist

ttlbyte
Posts: 1
Joined: Mon Nov 02, 2020 5:53 pm

Re: Configure says HDF5 and netCDF have MPI support but the code doesn't (v9.0.4)

Post by ttlbyte » Mon Nov 02, 2020 6:52 pm

Hi Jean-Michel ,

I think I have a smiliar problem. I compiled hdf5, netcdf, netcdf-f with mpich and intel compiler. The abinit configure runs all correct. But the output always fails with HDF5 mpi io. Can you share your receipts for building Openmpi 4, HDF5 and Netcdf? I would really appreciate it.

Thanks in advance
Shiqi

Locked