I am trying to better understand link-time optimization in GCC and -fuse-linker-plugin seems to matter in that context. However, I do not exactly understand how.
Suppose my compilation with link-time optimization looks like this:
gcc a.c -c -o a.o -flto -fno-fat-lto-objects
gcc b.c -c -o b.o -flto -fno-fat-lto-objects
gcc main.c -o main.out a.o b.o
What would be a good use case for enabling -fuse-linker-plugin?
I found the official GCC documentation somewhat confusing on that point. They suggest that -fuse-linker-plugin matters only for archive files (.a) but most conversation I have seen mentions that option without discussing archive files.
I'm trying to learn how to write makefiles. I have started reading the manual of gnu make: https://www.gnu.org/software/make/manual/html_node/Simple-Makefile.html#Simple-Makefile
I have 3 files in the same directory:
main.cpp: which creates a rectangle and prints some information. Therefor it includes Rectangle.h
Rectangle.h: header file for rectangle class
Rectangle.cpp: implementation of rectangle class
I am having troubles with the include of Rectangle.h in main.cpp. My makefile is:
main: main.o rectangle.o
g++ -o main.exe main.o rectangle.o
main.o: main.cpp
g++ main.cpp
rectangle.o: Rectangle.cpp
g++ Rectangle.cpp
clean:
rm main.exe main.o rectangle.o
I know something is missing to create main.o but I can't find out what it is. I tried adding various variations of Rectangle.h/.o/.cpp and finding something on the internet but I was unable to find something.
Help will be much appreciated :)
PS: The code is fine, I can compile it with the command:
g++ -o main.exe main.cpp Rectangle.cpp
man g++
When you invoke GCC, it normally does preprocessing, compilation, assembly and linking. The "overall options" allow you to stop this process at an intermediate stage. For example, the -c option says not to run the linker. Then the output consists of object files output by the assembler.
I have written a makefile like the following:
COMPILER = g++
COMPILER_FLAGS = -c -g -O0 -Wall -w
EXEC=mhu9_mp3
LD = g++
LINKER = g++
mhu9_mp3 : mhu9_mp3.o teapot.o EasyBMP.o
$(LD) teapot.o mhu9_mp3.o EasyBMP.o -L./SOIL/lib -lSOIL -o mhu9_mp3 -L./usr/lib -lGL -lGLU -lglut -lGLEW
teapot.o : teapot.cpp teapot.h
$(COMPILER) $(COMPILER_FLAGS) teapot.cpp
mhu9_mp3.o : mhu9_mp3.cpp teapot.h
$(COMPILER) $(COMPILER_FLAGS) mhu9_mp3.cpp
EasyBMP.o : EasyBMP_1.06/EasyBMP.cpp
$(COMPILER) $(COMPILER_FLAGS) EasyBMP_1.06/EasyBMP.cpp
clean:
-rm -f *.o mhu9_mp3
where libSOIL.a is a static library (I linked it by using "-L./SOIL/lib -lSOIL") and -lGL -lGLU -lglut -lGLEW are dynamic libraries. I have linked them together at the same line. In my cpp file I have a function call that calls a function defined in SOIL.h . However, when I compiled I got this error:
undefined reference to `SOIL_load_image'
I'm pretty sure I have included the header file SOIL.h, and also I have linked in the static library libSOIL.a. So what is the reason for this error? I try adding a -static flag before the -lSOIL but the linker wrongly assumes other dynamics libraries are static. Is there any better ways of linking static and dynamic libraries at the same time?
Put the list of dynamic libraries before the list of static libraries.
Example:
$(LD) -L<path> -l<dynamiclib1> -l<dynamiclib2> -static -l<staticlib1> -l<staticlib2>
Counterpart of -static is -Bdynamic. So if you really have to, it could be something like
ld -o foo foo.o -static -lstaticlib -Bdynamic -ldynamiclib0 -ldynamiclib1
However, this makes sense only when both dynamic and static libraries are present. If only one exists, linker will chose one itself.
In your case, problem may be caused by wrong link order or by the fact that SOIL_load_image is indeed missing (verify it with nm or objdump).
I was trying to do something like this in a makefile:
program.exe: ui.o main.o
gcc ......etc
ui.o: window1.o window2.o
gcc -c window1.o window2.o -o ui.o #this doesn't want to work
window1.o: window1.c window1.h window1_events.c window1_controls.c ...
gcc -c window1.c window1_events.c window1_controls.c... -o window1.o
window2.o: ...
gcc ...
main.o: ...
gcc ...
but when I compile like this, it gives the error "input file unused because linking not done," and then I get a bunch of unresolved externs, etc--problems which are resolved by changing
program.exe: ui.o main.o
gcc ...
to
program.exe: window1.o window2.o main.o
gcc ...
so is it possible to just link object files together, to avoid having mile-long lines in a makefile and break down the build process a little more?
Yes: to merge several object files into one, use ld -r or ld -Ur:
From "man ld" on Linux:
-r
--relocatable
Generate relocatable output---i.e., generate an output file that can
in turn serve as input to ld. This is often called partial linking.
As a side effect, in environments that support standard Unix magic
numbers, this option also sets the output file’s magic number to
"OMAGIC".
If this option is not specified, an absolute file is produced.
When linking C++ programs, this option will not resolve references to
constructors; to do that, use -Ur.
You could also do this with gcc:
gcc -Wl,-r foo.o bar.o -o foobar.o -nostdlib
Merging object files like this has some advantages over using an archive library: if merged files change very infrequently (compared to say main.c), your final executable links will be faster.
OTOH, with archived library, the linker will only use what it needs, so your executable may end up being smaller if e.g. window2.c ends up not being necessary.
I bunch of object files is a library. You can create a library with the ar
utility. The following example creates a library called mylib.a containing the files foo.o and bar.o
ar rvs mylib.a foo.o bar.o
You can then link with it by using it on the compiler command line:
gcc -o myexe main.c mylib.a
To create a library:
ar rvs somelib.a file1.o file2.o file3.o
To link it:
gcc -o program.exe file4.o somelib.a
Why does the order in which libraries are linked sometimes cause errors in GCC?
(See the history on this answer to get the more elaborate text, but I now think it's easier for the reader to see real command lines).
Common files shared by all below commands
// a depends on b, b depends on d
$ cat a.cpp
extern int a;
int main() {
return a;
}
$ cat b.cpp
extern int b;
int a = b;
$ cat d.cpp
int b;
Linking to static libraries
$ g++ -c b.cpp -o b.o
$ ar cr libb.a b.o
$ g++ -c d.cpp -o d.o
$ ar cr libd.a d.o
$ g++ -L. -ld -lb a.cpp # wrong order
$ g++ -L. -lb -ld a.cpp # wrong order
$ g++ a.cpp -L. -ld -lb # wrong order
$ g++ a.cpp -L. -lb -ld # right order
The linker searches from left to right, and notes unresolved symbols as it goes. If a library resolves the symbol, it takes the object files of that library to resolve the symbol (b.o out of libb.a in this case).
Dependencies of static libraries against each other work the same - the library that needs symbols must be first, then the library that resolves the symbol.
If a static library depends on another library, but the other library again depends on the former library, there is a cycle. You can resolve this by enclosing the cyclically dependent libraries by -( and -), such as -( -la -lb -) (you may need to escape the parens, such as -\( and -\)). The linker then searches those enclosed lib multiple times to ensure cycling dependencies are resolved. Alternatively, you can specify the libraries multiple times, so each is before one another: -la -lb -la.
Linking to dynamic libraries
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -L. -ld -o libb.so # specifies its dependency!
$ g++ -L. -lb a.cpp # wrong order (works on some distributions)
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong order
$ g++ -Wl,--as-needed a.cpp -L. -lb # right order
It's the same here - the libraries must follow the object files of the program. The difference here compared with static libraries is that you need not care about the dependencies of the libraries against each other, because dynamic libraries sort out their dependencies themselves.
Some recent distributions apparently default to using the --as-needed linker flag, which enforces that the program's object files come before the dynamic libraries. If that flag is passed, the linker will not link to libraries that are not actually needed by the executable (and it detects this from left to right). My recent archlinux distribution doesn't use this flag by default, so it didn't give an error for not following the correct order.
It is not correct to omit the dependency of b.so against d.so when creating the former. You will be required to specify the library when linking a then, but a doesn't really need the integer b itself, so it should not be made to care about b's own dependencies.
Here is an example of the implications if you miss specifying the dependencies for libb.so
$ export LD_LIBRARY_PATH=. # not needed if libs go to /usr/lib etc
$ g++ -fpic -shared d.cpp -o libd.so
$ g++ -fpic -shared b.cpp -o libb.so # wrong (but links)
$ g++ -L. -lb a.cpp # wrong, as above
$ g++ -Wl,--as-needed -L. -lb a.cpp # wrong, as above
$ g++ a.cpp -L. -lb # wrong, missing libd.so
$ g++ a.cpp -L. -ld -lb # wrong order (works on some distributions)
$ g++ -Wl,--as-needed a.cpp -L. -ld -lb # wrong order (like static libs)
$ g++ -Wl,--as-needed a.cpp -L. -lb -ld # "right"
If you now look into what dependencies the binary has, you note the binary itself depends also on libd, not just libb as it should. The binary will need to be relinked if libb later depends on another library, if you do it this way. And if someone else loads libb using dlopen at runtime (think of loading plugins dynamically), the call will fail as well. So the "right" really should be a wrong as well.
The GNU ld linker is a so-called smart linker. It will keep track of the functions used by preceding static libraries, permanently tossing out those functions that are not used from its lookup tables. The result is that if you link a static library too early, then the functions in that library are no longer available to static libraries later on the link line.
The typical UNIX linker works from left to right, so put all your dependent libraries on the left, and the ones that satisfy those dependencies on the right of the link line. You may find that some libraries depend on others while at the same time other libraries depend on them. This is where it gets complicated. When it comes to circular references, fix your code!
Here's an example to make it clear how things work with GCC when static libraries are involved. So let's assume we have the following scenario:
myprog.o - containing main() function, dependent on libmysqlclient
libmysqlclient - static, for the sake of the example (you'd prefer the shared library, of course, as the libmysqlclient is huge); in /usr/local/lib; and dependent on stuff from libz
libz (dynamic)
How do we link this? (Note: examples from compiling on Cygwin using gcc 4.3.4)
gcc -L/usr/local/lib -lmysqlclient myprog.o
# undefined reference to `_mysql_init'
# myprog depends on libmysqlclient
# so myprog has to come earlier on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# we have to link with libz, too
gcc myprog.o -lz -L/usr/local/lib -lmysqlclient
# undefined reference to `_uncompress'
# libz is needed by libmysqlclient
# so it has to appear *after* it on the command line
gcc myprog.o -L/usr/local/lib -lmysqlclient -lz
# this works
If you add -Wl,--start-group to the linker flags it does not care which order they're in or if there are circular dependencies.
On Qt this means adding:
QMAKE_LFLAGS += -Wl,--start-group
Saves loads of time messing about and it doesn't seem to slow down linking much (which takes far less time than compilation anyway).
Another alternative would be to specify the list of libraries twice:
gcc prog.o libA.a libB.a libA.a libB.a -o prog.x
Doing this, you don't have to bother with the right sequence since the reference will be resolved in the second block.
A quick tip that tripped me up: if you're invoking the linker as "gcc" or "g++", then using "--start-group" and "--end-group" won't pass those options through to the linker -- nor will it flag an error. It will just fail the link with undefined symbols if you had the library order wrong.
You need to write them as "-Wl,--start-group" etc. to tell GCC to pass the argument through to the linker.
You may can use -Xlinker option.
g++ -o foobar -Xlinker -start-group -Xlinker libA.a -Xlinker libB.a -Xlinker libC.a -Xlinker -end-group
is ALMOST equal to
g++ -o foobar -Xlinker -start-group -Xlinker libC.a -Xlinker libB.a -Xlinker libA.a -Xlinker -end-group
Careful !
The order within a group is important !
Here's an example: a debug library has a debug routine, but the non-debug
library has a weak version of the same. You must put the debug library
FIRST in the group or you will resolve to the non-debug version.
You need to precede each library in the group list with -Xlinker
Link order certainly does matter, at least on some platforms. I have seen crashes for applications linked with libraries in wrong order (where wrong means A linked before B but B depends on A).
I have seen this a lot, some of our modules link in excess of a 100 libraries of our code plus system & 3rd party libs.
Depending on different linkers HP/Intel/GCC/SUN/SGI/IBM/etc you can get unresolved functions/variables etc, on some platforms you have to list libraries twice.
For the most part we use structured hierarchy of libraries, core, platform, different layers of abstraction, but for some systems you still have to play with the order in the link command.
Once you hit upon a solution document it so the next developer does not have to work it out again.
My old lecturer used to say, "high cohesion & low coupling", it’s still true today.