I have a system A (aka Host) with compilers and a system B (aka Target) where I would like to run a MPI application. On system B there is no compiler or mpirun. I would like to do the following: compile mpirun (from openmpi) on system A and then using it on system B (with executable compiled on system A). I have never compiled for a different platform. Can this thing work? What do I do with shared libraries? I see that mpirun links to :
linux-vdso.so.1 => (0x00002aaaaaaab000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003a89e00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003a89600000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003a89a00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003a89200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003a88e00000)
do I have to statically link mpirun to libraries on system A? How do I do that?
Also, I would like to know if I can compile mpif90 on system A, so I can use it directly to compile programs on system B. Keep in mind that there is no compiler on system B, and I am not willing to install them
output of uname -a on Host
Linux host 2.6.32-220.el6.x86_64 #1 SMP Tue Dec 6 19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux
output of uname -a on Target
Linux target 2.6.35-32-server #67-Ubuntu SMP Mon Mar 5 21:13:25 UTC 2012 x86_64 GNU/Linux
so far I have tried to compile with the following configuration
CC=icc
FC=ifort
CXX=icpc
LDFLAGS=-static-intel
./configure --prefix=/gpfs/data/garzilli/data/local/openmpi-1.8.4-cc/ --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --enable-static --disable-shared
I know that in this case I am not technically cross-compiling, because host and build coincide, in principle they could be different. Maybe, should I also set --target, because I am compiling a compiler?
It is possible to compile on one platform and execute on the other. I do this often with Fortran binaries compile with ifort and mpif90. I just ran a test where I compiled an executable with the ifort compiler and mpif90 on OpenSUSE 2.6.34 kernel x86_64 and ran it on a Amazon Linux kernel version 3.14 x86_64 that had no mpi libs installed.
The problem is that without mpirun (or mpiexec), you will only be able to run a single instance of the program. So, if you have to launch multiple instances, you will need something like mpirun on the other machine (system B) to launch the multiple instances.
Related
Thanks in advance.
my development environment：
$ cat /proc/version
Linux version 5.4.0-66-generic (buildd#lgw01-amd64-016) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #74~18.04.2-Ubuntu SMP Fri Feb 5 11:17:31 UTC 2021
$ ld --version
GNU ld (GNU Binutils for Ubuntu) 2.30
Copyright (C) 2018 Free Software Foundation, Inc.
$ getconf GNU_LIBC_VERSION
glibc 2.27
$ #my glibc source version is 2.32.9000-development
$ cat ./version.h
/* This file just defines the current version number of libc. */
#define RELEASE "development"
#define VERSION "2.32.9000"
For some reasons, I need to modify and test glibc. I follow the steps of this website(https://sourceware.org/glibc/wiki/Testing/Builds#Compile_against_glibc_in_an_installed_location) to modify glibc and write test programs.
compile glibc.(confgure and make)
install glibc.(make install to a directory)
...other steps in the website above.
I successfully modified some pthread functions and passed the test (the test program I wrote can compiled against the install glibc and ran successfully). ldd the program.
$ ldd ./exec/1-1.out
linux-vdso.so.1 (0x00007ffcbf367000)
libpthread.so.0 => /home/cjl-target/gnu/install/lib64/libpthread.so.0 (0x00007fcadcea9000)
libc.so.6 => /home/cjl-target/gnu/install/lib64/libc.so.6 (0x00007fcadcaed000)
/home/cjl-target/gnu/install/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fcadd2ca000)
As shown above, the shared libraries that the program depends on all point to the glibc installation path.
But when I compiled message-queue's test program(test mq_unlink) and ran it, failed as bellow:
./exec/1-1.out: symbol lookup error: /lib/x86_64-linux-gnu/libpthread.so.0: undefined symbol: __libc_vfork, version GLIBC_PRIVATE
check the library that is depended by the program:
$ ldd ./exec/1-1.out
linux-vdso.so.1 (0x00007ffce3f72000)
librt.so.1 => /home/cjl-target/gnu/install/lib64/librt.so.1 (0x00007f0a389a2000)
libc.so.6 => /home/cjl-target/gnu/install/lib64/libc.so.6 (0x00007f0a385e6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0a383c7000)
/home/cjl-target/gnu/install/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f0a38dac000)
As shown above, the shared libraries libpthread.so.0 points to the system library. Why?
my compile script is(from the website above):
# dobuild.sh
SYSROOT=/home/xxx/xxx/xxx #the glibc's installation path
(set -x; \
gcc \
-L${SYSROOT}/usr/lib64 \
-I${SYSROOT}/usr/include \
--sysroot=${SYSROOT} \
-Wl,-rpath=${SYSROOT}/lib64 \
-Wl,--dynamic-linker=${SYSROOT}/lib64/ld-linux-x86-64.so.2 \
-Wall $*
)
when I compile the pthread's test program:./dobuild 1-1.c -pthread -Wall
when I compile the mq's test program:./dobuild 1-1.c -lrt -Wall
In addition, it is confusing that when invoke the pthread_create in the mq_unlink's test program, compiling it ./dobuild 1-1.c -lrt -pthread, the ldd result shows that all dependent libraries point to the installed glibc.
I've tried multiple variations of this, but none of them seem to work. Any ideas?
First, you should stop using ldd -- in the presence of multiple GLIBCs on a host, ldd is more likely to mislead than to illuminate.
If you want to see which libraries are really loaded, do this instead:
LD_TRACE_LOADED_OBJECTS=1 ./exec/1-1.out
Second, you should almost never use $* in shell scripts. Use "$#" instead (note: quotes are important). See this answer.
Third, the behavior you are observing is easily explained. To understand it, you need to know the difference between DT_RPATH and DT_RUNPATH, described here.
You can verify that your binaries are currently using RUNPATH, like so:
readelf -d 1-1.out | grep 'R.*PATH'
And you can verify that everything starts working as you expect by adding -Wl,--disable-new-dtags to the link command (which would cause the binary to use RPATH instead).
To summarize:
RUNPATH affects the search for the binary itself, but not for any libraries the binary depends on.
RPATH affects the search path for the binary and all libraries it depends on.
with RUNPATH, expected libpthread.so.0 is found only when the binary depends on it directly, but not when the dependency on libpthread is indirect (via librt).
with RPATH, expected libpthread.so.0 is found regardless of whether the dependency is direct or indirect.
Update:
If I want to use DT_RUNPATH, how to set the library runpath for librt?
You would need to link librt.so with -rpath=${SYSROOT}/lib64.
You could edit the rt/Makefile, or build with:
make LDFLAGS-rt.so='-Wl,--enable-new-dtags,-z,nodelete,-rpath=${SYSROOT}/lib64'
You would need to do the same for any other library that may bring transitive dependency on other parts of GLIBC. I don't know of a general way to do this, but setitng LDFLAGS-lib.so='-Wl,-rpath=${SYSROOT}/lib64' and rebuilding everything might do the trick.
I have pre-compiled binaries of the arm-linux-gnueabihf GCC toolchain. Host is x86_64 while the intended target is arm7 architecture. I am able to cross compile a sample program using this cross-compiler but it is not able to link with dynamic libraries. On the host, I use:
arm-linux-gnueabihf -c -o test test.c
The compiled executable with ldd command shows following output on host as well as target:
ldd ./test
not a dynamic executable
However, I use the same source code and compile natively on the target, I observe:
ldd ./test
libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6e29000)
/lib/ld-linux-armhf.so.3 (0xb6f18000)
The whole concept of linker seems tricky in case of cross-compilation. Is there a standard way to link shared dynamic libraries while cross compiling?
I have a router Asus RT-AC68R with 32GB memory pen mounted to /opt. On it is installed entware including all the tools to build programs natively on the platform ARMv7. Router use TomatoUSB firmware.
I wrote a program to support the module based on the chip FT245 connected via USB to the router. The program compiled on Linux x86_64 and it works correctly on a standard Linux PC. It was - of course - necessary installing library supplied by the manufacturer (FTDI). The manufacturer provides these libraries for various platforms, including the ARMv7 versions soft and hard float. On router I installed library hard float.
These libraries are HERE
So I'm trying to compile the program, prior to the installation of the pen FTDI libraries, according to the documentation. Unfortunately, after starting the program an library error occurs.
Below the way I compile this program (named arco) and the result of running this program on my ARMv7 based router.
root#asus:/opt# gcc -Wl,-rpath=/opt/usr/local/lib -Wl,--dynamic-linker=/opt/lib/ld-linux.so.3 -L/opt/usr/local/lib -O2 -pipe -march=armv7-a -mtune=cortex-a9 -fno-caller-saves -mfloat-abi=soft -l ftd2xx arco.c -o arco
root#asus:/opt# ./arco
./arco: error while loading shared libraries: /opt/usr/local/lib/libftd2xx.so: internal error
Additional information about my dev environment:
root#asus:/opt/FTDI# echo $PATH
/opt/bin:/opt/sbin:/opt/bin:/opt/sbin:/bin:/usr/bin:/sbin:/usr/sbin:/home/root:/mmc/sbin:/mmc/bin:/mmc/usr/sbin:/mmc/usr/bin:/opt/sbin:/opt/bin:/opt/usr/sbin:/opt/usr/bin:
root#asus:/opt/FTDI# echo $LD_LIBRARY_PATH
/lib:/usr/lib:/usr/local/lib:/opt/lib:/opt/usr/lib:/opt/include
root#asus:/opt/FTDI# opkg list-installed|grep gcc
gcc - 5.4.0-1
libgcc - 5.4.0-5
root#asus:/opt# uname -a
Linux asus 2.6.36.4brcmarm #5 SMP PREEMPT Tue Aug 16 22:49:47 CEST 2016 armv7l GNU/Linux
I am fairly new when it comes to platform ARMv7 but maybe there is something in compilation options I use:
-mfloat-abi=soft
a library installed is hard-float. I do not know if this could be the reason so expect council colleagues.
Thank you in advance for your answers!
I have an ARM machine that runs Linux (BusyBox).
# uname -a
# Linux XXXXXXXX 2.6.28 #1 PREEMPT Fri Sep 26 22:47:38 UTC 2014 armv5tel GNU/Linux
I've cross-compiled a simple program on my Ubuntu 32-bit desktop:
./configure --host=arm-linux-gnueabi LDFLAGS="-static"
make
But when I try to run it on the ARM machine, it gives me Segmentation Fault error.
Program is super simple:
#include <stdio.h>
int main()
{
printf("Hello, World!");
return 0;
}
Here are a few things I've already tried/checked:
I've checked md5 hashes on both machines to eliminate the possibility that something went wrong at the time of copying an executable over the network
Stripped the executable with arm-linux-gnueabi-strip. I was comparing my executable with another executable that was already in the target machine with file:
# file my_program
# my_program: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, for GNU/Linux 2.6.31, BuildID[sha1]=0x4b1f2773e54b141d5157b86f0f10438a372625c9, stripped
# file their_program
# their_program: ELF 32-bit LSB executable, ARM, version 1 (GNU/Linux), statically linked, stripped
What am I doing wrong?
Apparently my cross-compile toolchain was not the right one.
I ended up using crosstool-ng. Btw it's very simple to use and a great tool, all you have to do is to choose the right toolchain for your device.
I have built an arm-unknown-linux-uclibcgnueabi toolchain with crosstool-ng, which solved my problem.
I installed Matlab in my Linux Mint 14 Nadia (a uname -a shows: Linux Ideapad-Z570 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux) and when calling it from the command line I would get a: "/lib64/libc.so not found".
I followed the help on mathworks by making a link in /lib64 as:
ln -s /lib/x86_64-linux-gnu/libc.so.6 .
That solved the issue.
Now, if I do a locate of this library I get:
locate "libc.so"
/lib/i386-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc.so.6
/usr/lib/x86_64-linux-gnu/libc.so
I will be compiling with gcc in this computer and I would like to have full 64bit compilations. What does exactly mean to have all these different libc.so libraries? which one will the gnu compiler be using? do I need to do anything different with gcc to compile for 64 bits?
I would also love to optimize as much as I can for my new i7 core!!!
/lib/i386-linux-gnu/libc.so.6
This is is 32-bit version of the library.
/lib/x86_64-linux-gnu/libc.so.6
This is the 64-bit version of the library.
Both are usually symbolic links to the actual library file, which will usually be named according to the glibc release number, for example libc-2.15.so
/usr/lib/x86_64-linux-gnu/libc.so
This is not a library, but a linker script file, which refers to the above symlinks.
Why do we need all these:
First, regardless of libc version installed, the linker will always search for libc.so, because the compiler driver will always pass to the linker the -lc options. The name libc stays the same and denotes to most recent version of the library.
The symlinks libc.so.6 are named after the soname of the library, which, more or less corresponds to the ABI version of the library. The executables, linked against libc.so in fact contain runtime dependencies on libc.so.6.
If we imagine the someday a grossly ABI incompatible libc is released, it's soname could be named libc.so.7, for example and this version coukld coexists with the older libc.so.6 version, thus executables linked against one or the other can coexist in the same system,
And finally, the name libc-2.15.so refers to the libc release, when you install a new libc package, the name will change to libc-2.16.so. Provided that it is binary compatible with the previous release, the libc.so.6 link will stay named that way and the existing executables will continue to work.
To find which one to use, you have to first find the order that ld (the linker) uses to find libraries, like so:
ld --verbose | grep SEARCH
For me it gave me this output:
SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib64"); SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib"); SEARCH_DIR("/usr/lib"); SEARCH_DIR("/usr/local/lib");
This means that on my computer, ld looks in these directories, in order:
/usr/x86_64-unknown-linux-gnu/lib64
/usr/x86_64-unknown-linux-gnu/lib
/usr/lib
/usr/local/lib
So if libc was in /usr/x86_64-unknown-linux-gnu/lib64, and libc was also in /usr/lib, it would use the /usr/x86_64-unknown-linux-gnu/lib64 version, because it was listed first.
The symlink you created will have no effect whatsoever on GCC. The 32-bit version is only used when you compile using the -m32 GCC flag. GCC will not attempt to generate 32-bit binaries unless you specifically tell it to (by using that flag.)