Does anyone know if it is possible to compile MPI with gcc?. I need to use gcc, no mpicc.
mpicc is just a wrapper around certain set of compilers. Most implementations have their mpicc wrappers understand a special option like -showme (Open MPI) or -show (Open MPI, MPICH and derivates) that gives the full list of options that the wrapper passes on to the backend compiler.
For example, in Open MPI, wrappers are C++ programs that read plain text configuration files and build command line options that are further passed on to the compiler. mpicc -showme shows the full list of such options:
$ mpicc -showme
icc
-I/opt/MPI/openmpi-1.5.3/linux/intel/include
-I/opt/MPI/openmpi-1.5.3/linux/intel/include/openmpi
-fexceptions
-pthread
-I/opt/MPI/openmpi-1.5.3/linux/intel/lib
-Wl,-rpath,/opt/MPI/openmpi-1.5.3/linux/intel/lib
-I/opt/MPI/openmpi-1.5.3/linux/intel/lib
-L/opt/MPI/openmpi-1.5.3/linux/intel/lib
-lmpi
-ldl
-Wl,--export-dynamic
-lnsl
-lutil
(it's really a single line that I have split here to improve readability)
It that particular case Intel C Compiler icc is used as the backend compiler but we also have variants that use GCC. You can also get the list of options needed for the comple phase (usually known as CFLAGS) with mpicc -showme:compile:
$ mpicc -showme:compile
-I/opt/MPI/openmpi-1.5.3/linux/intel/include
-I/opt/MPI/openmpi-1.5.3/linux/intel/include/openmpi
-fexceptions
-pthread
-I/opt/MPI/openmpi-1.5.3/linux/intel/lib
as well as the list of options that you need to pass to the linker (known as LDFLAGS) with mpicc -showme:link:
$ mpicc -showme:link
-fexceptions
-pthread
-I/opt/MPI/openmpi-1.5.3/linux/intel/lib
-Wl,-rpath,/opt/MPI/openmpi-1.5.3/linux/intel/lib
-I/opt/MPI/openmpi-1.5.3/linux/intel/lib
-L/opt/MPI/openmpi-1.5.3/linux/intel/lib
-lmpi
-ldl
-Wl,--export-dynamic
-lnsl
-lutil
These could be used, e.g. in a Makefile, like this:
...
CFLAGS += $(shell mpicc -showme:compile)
LDFLAGS += $(shell mpicc -showme:link)
...
As far as I know -showme:compile and -showme:link are specific to Open MPI and other implementations only give the full list of options when called with -show.
I still think it's better to use mpicc directly because if it happens that something in the MPI setup is changed, it will be immediately reflected in the wrapper while you would have to change your build script / Makefile manually (unless you use -showme:compile and -showme:link to obtain the options automatically).
mpicc -compile_info for MPICH.
Yes, you can use gcc actually. But in my case (on Ubuntu) mpicc is just a wrapper of gcc, here is the output of command mpicc -showme:
gcc -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -L/usr/lib/openmpi/lib -lmpi
While in Open MPI docs:
The Open MPI team strongly recommends that you simply use Open MPI's "wrapper" compilers to compile your MPI applications. That is, instead of using (for example) gcc to compile your program, use mpicc.
We repeat the above statement: the Open MPI Team strongly recommends that the use the wrapper compilers to compile and link MPI applications.
If you find yourself saying, "But I don't want to use wrapper compilers!", please humor us and try them. See if they work for you. Be sure to let us know if they do not work for you.
Many people base their "wrapper compilers suck!" mentality on bad behavior from poorly-implemented wrapper compilers in the mid-1990's. Things are much better these days; wrapper compilers can handle almost any situation, and are far more reliable than you attempting to hard-code the Open MPI-specific compiler and linker flags manually.
That being said, there are some -- very, very few -- situations where using wrapper compilers can be problematic -- such as nesting multiple wrapper compilers of multiple projects. Hence, Open MPI provides a workaround to find out what command line flags you need to compile MPI applications.
Here this answer is useful for you.
For MPICH, according to the mpicc man pages, mpicc -compile_info shows the flags for compiling a program, and mpicc -link_info shows the flags for linking a program.
Yes, you can certainly compile an MPI program without the convenience of the mpicc wrapper. On most implementations mpicc is a shell script (or similar) which sets environment variables, finds and links various libraries, all the sort of stuff that you might otherwise put into a Makefile.
I suggest that you find an instance of the mpicc script and deconstruct it.
mpicc is already using gcc as a backend
Related
Like C compile flags -fno-pic -O -nostdinc -c -static -fno-builtin -fno-strict-aliasing -m32 -fno-stack-protector can compile codes into pure ELF without library rely on unique system. Is there a possible way GO can do that?
Simple answer is no. Go relies on the operating system (Windows, linux, macOS, BSD). Go has a runtime designed to work on specific environments https://github.com/golang/go/wiki/MinimumRequirements.
There are some open source projects that will help you achieve it, but I wouldn't put them in production as they support a limited number of hardware chips and they are not supported in the same way as the standard library. Some examples are:
https://github.com/ziutek/emgo
https://github.com/tinygo-org/tinygo
These frameworks/libraries will help you run Go code on bare metal.
I having a hell of a time tracking down documentation for all of the gcc options. I'm on an ubuntu machine so not even sure if this is relevant to me.
Comes from this make rule:
my-server: my-server.c
$(CC) -Wall -pthread my-server.c -o my-server
I've found this option summary:
https://gcc.gnu.org/onlinedocs/gcc-4.8.0/gcc/Option-Summary.html
From man gcc
-pthread
Add support for multithreading using the POSIX threads library. This option sets flags for both the preprocessor and linker. It does not affect the thread safety of object code produced by the compiler or that of libraries supplied with it. These are HP-UX specific flags.
Note that it differs from -lpthread. The -lpthread option doesn't set the preprocessor flags. For instance, the macros _REENTRANT and __USE_REENTRANT, etc.
In the documentation for GCC (6.2.0), it is treated as a platform-specific option. For RS6000 and PowerPC, it says:
-pthread.
Adds support for multithreading with the pthreads library. This option sets flags for both the preprocessor and linker.
For Solaris 2, it says:
-pthreads.
Add support for multithreading using the POSIX threads library. This option sets flags for both the preprocessor and linker. This option does not affect the thread safety of object code produced by the compiler or that of libraries supplied with it.
-pthread.
This is a synonym for -pthreads.
The common 'thread' (sorry — couldn't resist) is that it ensures that the code is compiled and linked with options that ensure that the POSIX threads library is used. The same will be true on other platforms where the option works (Linux, macOS Sierra, Mac OS X before it, …) it will link with the right library and enable any relevant preprocessor options.
You could compare the output of:
gcc -v -pthread -o x1 pthread-prog.c
gcc -v. -o x2 pthread-prog.c
to see what differences the -pthread option makes. On macOS Sierra, it adds -D_REENTRANT to the cc1 phase, and -pthread to the 'COLLECT_GCC_OPTIONS' setting.
I'm interested in answers, approaches, and ideas out of the box. At a high level, the main page is pretty sparse and they mainly list -g, with one level, suggesting that -O0 is also either very helpful or essential.
But I'm wondering what other clang flags can be given to give maximum debugging. Is there an equivalent to gcc's -ggdb3 which includes some of the source or annotations directly in the object output? Or could there be? Is it possible and helpful to recompile the OS and its original libraries to have debugging (and if so, if I'm using Debian, can I have it write the debugging into the main .deb package instead of putting a separate debugged .deb package which stores debugging data in /usr/lib/debug?)? Will a static build of a binary affect the ability to see a good stacktrace? And is there anything that needs to be done to ensure that addr2line works well? Is it needed to compile all libraries (even glibc) with clang to get the maximum debugging benefit? I note that there is a project to recompile Debian with clang, and otherwise am open to a distribution that does so or otherwise places emphasis on debugging.
On Linux there are also options like an LD_PRELOAD set to /lib/libSegFault.so, or a set of LD_LIBRARY_PATH reassignments to /usr/lib/debug instead of the usual /usr/lib location (including redirecting libc itself to the debugged version). Is there a central place or external sources for answers to this question of how to enhance debuggability of a binary? The bigger mystery is clang, since I see in the long gcc man page that there are various options which can increase debugging (or reduce optimisations), but on the other hand the documentation for clang only shows a smaller set. It's possible that clang will accept more options than given, including gcc flags (which may either translate to a no-op or to more debugging - hard to tell without a canonical source of information).
Also from a package build perspective, since an external package may not respect CFLAGS, I've redirected /usr/bin/strip to be a no-op command that always succeeds, but other ideas on ensuring compliance are suggested (I believe that pkgsrc does a good job of wrapping gcc and the linker in shell scripts - useful to insert mandatory flags). Also there may be various ld options that can be passed to increase debugging of the outputted target. Also, it's quite possible that BSD (including FreeBSD 10, based upon clang) may have a different linking architecture which could make it easier to request and find debugged symbols in the generated libraries and executables.
To take debugging more broadly defined, I've set LD_WARN=yes, LD_DEBUG=unused, SEGFAULT_SIGNALS="all", LD_PRELOAD=.../libSegFault.so (as mentioned above), and LD_BIND_NOW=yes. Also I believe I can prefer that gcc search for libraries in /usr/lib/debug - above the standard search paths using strategic -Bs. Also, using --whole-archive for a static build might ensure that more objects are included in the linked output. There's also ulimit -c unlimited, and on Linux a nice way to differentiate core files like:
sysctl -w kernel.core_pattern="core.%t.SIG-%s.PID-%p.ID-%g-%u.%h.%E"
For gcc I've used and seen flags like: -O0 -fno-omit-frame-pointer -fverbose-asm -ggdb3 -mno-omit-leaf-frame-pointer -mtune=generic -fvar-tracking -D_GLIBCXX_DEBUG=1 -frecord-gcc-switches -femit-class-debug-always -fmath-errno -fno-eliminate-unused-debug-symbols -fno-eliminate-unused-debug-types -fno-merge-debug-strings -mieee-fp -mtune=generic -static-libgcc -fexceptions -fvar-tracking -fbounds-check -rdynamic -UNDEBUG -DDEBUG=1 (-ffreestanding -static-libgcc -pass-exit-codes) -fno-stack-check (since I believe I've read that the latter can interfere with debugging)
Other flags are there for other reasons but the emphasis is to be on maximum debugging. With all or most of the above, it's unclear to what extent clang would support or use there, or whether there are other options.
Clang does not support the -ggdb3 flag, only -g, as you have noticed. If you try to use it, you'll get the message:
clang: warning: argument unused during compilation: '-ggdb3'
so you can run your entire command line through Clang and it will tell you which of those GCC flags it supports and which it does not, some will print warnings, others may error out, but Clang will not silently ignore them. Here are the ones that Clang rejected when I tried your long command: -static-libgcc and -pass-exit-codes.
As pointed out in another SO answer, clang -cc1 --help can be used to list supported compilation flags, where we see the following which may be of interest to you:
-disable-llvm-optzns: Don't run LLVM optimization passes
-fno-elide-constructors: Disable C++ copy constructor elision
-mdisable-fp-elim: Disable frame pointer elimination optimization
I'm working through a text on linking, and wanted to work along with some examples in said text.
To better understand whats going on when I invoke the gcc driver, I was looking into doing all the compilation old-school by hand;
preprocessing using cpp
compiling with cc1
assembling with as
linking using ld
Unfortunately, on my Mac I don't seem to be able to reference cc1directly (no listing of cc1 in man). What are my options?
Read some material about GCC internals. First the wikipage on GCC. Then, you could play with the MELT probe (you may want a Linux system to use it).
Then read the GCC manual. And the GCC resource center
Use gcc -v -Wall to compile, it will show what cc1 is running (and -Wall is always useful). So try compiling with gcc -v -Wall hello.c -o helloworld.bin
the cc1 program don't sit in your PATH or in /usr/bin/ but -on my system- in /usr/lib/gcc/x86_64-linux-gnu/4.8/cc1
The command gcc -print-prog-name=cc1 will tell you which cc1 is used by your gcc. See GCC debugging options for more.
The preprocessing is now inside cc1.
The gcc program is just a driver, starting cc1 (which does most of the compiling work), then as, ld etc...
The MELT site contains some documentation, and some slides explaining GCC, which you could find interesting. MELT is a domain specific language to extend GCC.
See also the picture on http://starynkevitch.net/Basile/gcc-melt/cc1-internals.svg and the below picture
picture from http://starynkevitch.net/Basile/gcc-melt/gcc-MELT-inside.png, done by me, CC BY SA
The cc1 is producing a .s assembly file; the as (started by gcc) is transforming it into .o. The ld linker (started by gcc) will produce a library or an executable.
PS. I have a Linux system, but things are very similar on MacOSX.
reference on linking
A good book about linking is Levine's Linkers & loaders book.
PS. MELT is obsolete in 2021, but I am working on the Bismon static source code analyzer and on RefPerSys (which generates C++ code).
For reference, I installed gcc-10 with brew on my macOS (Catalina).
While cc1 is not directly accessible (via PATH), it can be found in
/usr/local/Cellar/gcc/10.2.0/libexec/gcc/x86_64-apple-darwin19/10.2.0
If you are lost, try gcc -v and from the verbose information you may find where your cc1 is.
It is almost common knowledge that one should always compile with -Wall.
What other useful options are you using when compiling with gcc/g++?
You may want -Wextra in addition of -Wall.
When debugging your program, -g is needed by gdb.
and GCC accepts both -g and -O if you really want
At last, recent versions of GCC (i.e. 4.6 or the 4.7 snapshot) gives better warnings than older ones.
You could use -pedantic and restrict yourself to some standard, avoiding GNU extensions.
I love GNU extensions so I don't want to avoid them.
So I use -std=gnu99 for C code, and -std=gnu0x or -std=gnu11 for C++11 code because I like extensions.
And you might consider using or even developing a GCC plugin or a MELT extension for your own specific needs.