2

I had try to install openmpi 1.8.1 from the source code at Ubuntu 14.04 server. I would like to use mpi for large scale calculation on single machine with multi cores.

The simple "Hello world" test can be run without any problem.

However, when I trying to compile the program which I need to use. I got the following error:

/usr/bin/ld: cannot find -lmpi_usempi 
/usr/bin/ld: cannot find -lmpi_mpifh 
/usr/bin/ld: cannot find -lmpi
collect2: error: ld returned 1 exit status
make: *** [mcp2_mpi] Error 1

I had try to set the LD_LIBRAY_PATH to the directory where the openmpi lib can be find. It seems doesn't work. I had try to check the ld command in verbose mode.

$ ld -lmpi_usempi --verbose
attempt to open /usr/x86_64-linux-gnu/lib64/libmpi_usempi.so failed
attempt to open /usr/x86_64-linux-gnu/lib64/libmpi_usempi.a failed
attempt to open //usr/local/lib/x86_64-linux-gnu/libmpi_usempi.so failed
attempt to open //usr/local/lib/x86_64-linux-gnu/libmpi_usempi.a failed
attempt to open //usr/local/lib64/libmpi_usempi.so failed
attempt to open //usr/local/lib64/libmpi_usempi.a failed
attempt to open //lib/x86_64-linux-gnu/libmpi_usempi.so failed
attempt to open //lib/x86_64-linux-gnu/libmpi_usempi.a failed
attempt to open //lib64/libmpi_usempi.so failed
attempt to open //lib64/libmpi_usempi.a failed
attempt to open //usr/lib/x86_64-linux-gnu/libmpi_usempi.so failed
attempt to open //usr/lib/x86_64-linux-gnu/libmpi_usempi.a failed
attempt to open //usr/lib64/libmpi_usempi.so failed
attempt to open //usr/lib64/libmpi_usempi.a failed
attempt to open //usr/local/lib/libmpi_usempi.so failed
attempt to open //usr/local/lib/libmpi_usempi.a failed
attempt to open //lib/libmpi_usempi.so failed
attempt to open //lib/libmpi_usempi.a failed
attempt to open //usr/lib/libmpi_usempi.so failed
attempt to open //usr/lib/libmpi_usempi.a failed
ld: cannot find -lmpi_usempi

It seems the ld doesn't look for the libraries in the LD_LIBRARY_PATH. I had try to create a file in /etc/ld.so.conf.d/ to include the path of openmpi lib. Then the command sudo ldconfig -v. I find the following output:

    /usr/local/openmpi/lib:
    libopen-rte.so.7 -> libopen-rte.so.7.0.3
    libmpi_mpifh.so.2 -> libmpi_mpifh.so.2.3.0
    libopen-trace-format.so.1 -> libopen-trace-format.so.1.0.0
    libmpi_cxx.so.1 -> libmpi_cxx.so.1.1.3
    liboshmem.so.1 -> liboshmem.so.1.0.0
    libvt-mpi.so.0 -> libvt-mpi.so.0.0.0
    libvt-hyb.so.0 -> libvt-hyb.so.0.0.0
    libvt-mt.so.0 -> libvt-mt.so.0.0.0
    libotfaux.so.0 -> libotfaux.so.0.0.0
    libvt-mpi-unify.so.0 -> libvt-mpi-unify.so.0.0.0
    libopen-pal.so.6 -> libopen-pal.so.6.1.1
    libmpi_usempi.so.1 -> libmpi_usempi.so.1.3.0
    libmpi.so.1 -> libmpi.so.1.5.0
    libvt.so.0 -> libvt.so.0.0.0
    libmca_common_sm.so.4 -> libmca_common_sm.so.4.0.3
    libompitrace.so.0 -> libompitrace.so.0.0.0

It seems the Libs are included in the ld search path.

However, I still got the same problem like before.

/usr/bin/ld: cannot find -lmpi_usempi
/usr/bin/ld: cannot find -lmpi_mpifh
/usr/bin/ld: cannot find -lmpi
collect2: error: ld returned 1 exit status
make: *** [mcp2_mpi] Error 1

Is there some suggestions to this issue? Thank you very much.

Kristopher Ives
  • 5,419
  • 2
  • 27
  • 36
Thomas Ding
  • 21
  • 1
  • 1
  • 3

4 Answers4

1

Working Ubuntu 20.04 setup

In general, you have to pass pkg-config flags to the compilation command to be able to compile, e.g. either:

sudo apt install libopenmpi-dev
gcc $(pkg-config --cflags mpi) main.c $(pkg-config --libs mpi)

sudo apt install libmpich-dev
gcc $(pkg-config --cflags mpich) main.c $(pkg-config --libs mpich)

depending if you want openmpi or MPICH: https://stackoverflow.com/questions/2427399/mpich-vs-openmpi

In theory both of those compile a simple test like this fine:

#include <mpi.h>

int main() {
  MPI_Finalize();
  return 0;
}

On Ubuntu 20.04 however only the MPICH works because the libopenmpi-dev package has a bug and forgot to symlink:

/usr/lib/x86_64-linux-gnu/libmpi.so -> /usr/lib/x86_64-linux-gnu/libmpi.so.40

so you can just do it manually as a workaround:

sudo ln -s /usr/lib/x86_64-linux-gnu/libmpi.so.40 \
           /usr/lib/x86_64-linux-gnu/libmpi.so

Bug report at: https://bugs.launchpad.net/ubuntu/+source/openmpi/+bug/1869612

I found this by looking for libmpi.so with:

locate libmpi.so
dpkg -L libopenmpi-dev

But it works if I just pass that directly to the compilation command:

gcc $(pkg-config --cflags mpi) main.c /usr/lib/x86_64-linux-gnu/libmpi.so.40

I also now learnt about the existence of /usr/bin/mpicc package from the openmpi-bin package https://stackoverflow.com/questions/20739909/what-is-mpic-file-used-for but the libopenmpi-dev version appears broken on Ubuntu also due to the missing libmpi.so symlink.

Ciro Santilli OurBigBook.com
  • 26,663
  • 14
  • 108
  • 107
1

The steps you've tried affect the run time library search path. Assuming that you are using gcc, to set the compile time search path you need to use the compiler's -L flag

   -Ldir
       Add directory dir to the list of directories to be searched for -l.

So for example if you have installed the libraries into /usr/local/openmpi/lib, modify your gcc command line to

-L /usr/local/openmpi/lib -lmpi_usempi -lmpi_mpifh -lmpi
steeldriver
  • 131,985
  • 21
  • 239
  • 326
  • Thank you very much for your reply. I had try the way you have mentioned. The error information is the same. I am sure I had the library files in the path. What I am surprise is that LD_LIBRARY_PATH and ld.so.conf.d doesn't work in Ubuntu 14.04 64 bit. – Thomas Ding Oct 23 '14 at 01:22
1

Solved: "cannot find -lmpi*" when compiling openmpi code

Solutions: I try to check the other options in my compiling command line. I add an extra -static in the compile command line. By removing this, it finally works for me.

Wilf
  • 29,694
  • 16
  • 106
  • 164
  • You can keep the -static link flag if you have installed the static version of the mpi. – fchen Mar 16 '19 at 22:30
0

According to OpenMPI documentation, it is suggested not to build MPI statically. The answer by Wilf confused me because one of my library (CUDD, if anyone is curious) usually required the -static flag to build. If I just simply remove the flag, the build will fail.

By studying the documentation of MPICH and gcc, I found that mpicxx -static and gcc -static are not the same idea. Luckily, CUDD can actually be built as a shared library. I configured and built the shared library, and then the -static flag can be removed without a problem.

Chien Kai Ma
  • 21
  • 1