I am using 3rd party libraries to build my executable. More specifically I have in ./extern/bin/win64
libfoo.dll
libfoo.dll.a
libbar.dll
Despite its name, it looks like libfoo.dll.a isn't really an import library, but rather an archive (static library). After scratching my head for some time, I came to this conclusion by looking at the output of lib /list libfoo.dll.a (from here) which gives me a bunch of .o files.
The command line I'm using to build my executable is always the same:
g++ -std=c++14 test.cpp -I./extern/include -L./extern/bin/win64 -lbaz -static-libgcc -static-libstdc++ -Wl,-Bstatic -lstdc++ -lpthread -Wl,-Bdynamic -lfoo -lbar -s -o test.exe
My understanding is that the final -Wl,-Bdynamic gives preference to dynamic linkage but doesn't insist on dynamic linkage. The linker should still be happy if only the static version of a library exist, but it should pick the dynamic version when both the static and the dynamic version exist. Correct ?
Now here is the part that puzzles me a little bit. If, in ./extern/bin/win64, I only keep libbar.dll and:
libfoo.dll then the executable has dependencies on libfoo.dll and libbar.dll (expected: dynamic linkage to libfoo is the only option)
libfoo.dll.a then the executable has a dependency on libbar.dll only (expected: static linkage to libfoo is the only option)
Both libfoo.dll and libfoo.dll.a then the executable has a dependency on libbar.dll only (surprising: static linkage seems to have been used but dynamic linkage was supposed to be preferred)
Does anyone have a theory to explain #3 ? Are my expectations incorrect ?
[Edit] As an extension to #3, I have also found that if I separate libfoo.dll and libfoo.dll.a into different folders, then the first folder to appear on the command line following -L determines which version of the library is selected. Again -Wl,-Bdynamic does not seem to have the expected effect....
PS: if anyone wants to have a detailed inspection of the libraries, they're distributed with Matlab, located in extern\bin\win64 as discussed. For completeness, the source code for my example can be found here but is this probably not relevant. Also using gcc 6.3 from here.
Related
When studying Scintilla's makefile for MinGW under windows, I noticed that it is passing -shared and -static together as LDFLAGS to gcc.
LDFLAGS=-shared -static -mwindows $(LDMINGW)
I googled, and only find some information from clang: https://reviews.llvm.org/D43811
[MinGW, CrossWindows] Allow passing -static together with -shared
In these combinations, link a DLL as usual, but pass -Bstatic instead of -Bdynamic to indicate prefering static libraries.
My question is: Would GCC do the same?
I haven't find any proof yet.
You can pass both -static and -shared in a GCC linkage. Their
combined effect is the same as you found described in your llvm link,
and this has always been the case for GCC.
-shared directs a GCC linkage to produce a shared library rather than a program,
which it achieves by passing on the option -shared to its invocation of
the linker.
-static directs a GCC linkage to ignore shared libraries when resolving
input library options -lname. By default -lname would be resolved by
searching the specified or default linker search directories for either
the shared library libname.so (on Windows, [lib]name.dll)
or the static library libname.a (on Windows also [lib]name.lib) and to prefer
the shared library if both of them are found in the same directory. -static
simply excludes all shared libraries from the search. GCC achieves this by passing the option -Bstatic
through to its invocation of the linker at a position in the generated linker
commandline that precedes all of the -lname options.
The GNU linker documentation of -Bstatic is explicit
that this option is consistent with -shared and that the effect is to produce a shared library
all of whose dependent libraries have been statically resolved.
-Bstatic
-dn
-non_shared
-static
Do not link against shared libraries. This is only meaningful on platforms for which shared libraries are supported.
The different variants of this option are for compatibility with various systems. You may use this option multiple times on the command line:
it affects library searching for -l options which follow it. This option also implies --unresolved-symbols=report-all.
This option can be used with -shared. Doing so means that a shared library is being created but that all of the library’s external references must be resolved by pulling in entries from static libraries.
(emphasis mine).
Although static linkage of shared library is in principle just a linkage restricted
in the same way as static linkage of a program, in practice it frequently encounters
a snag on Unix and Linux because all the object code linked into an ELF shared library
libname.so must be Position Independent Code,
as produced by the GCC compilation option -fPIC, whereas object files that are destined to be
archived in static libraries are customarily not compiled with -fPIC. Linkages using
-shared ... -static are thus apt to fail because necessary static libraries contain
non-PIC object files.
You do not have this worry with GCC on Windows, however, because there
is no such distinction as PIC v. non-PIC in Windows PE
object code.
I'm attempting to do a release of some software and am currently working through a script for the build process. I'm stuck on something I never thought I would be, statically linking LAPACK on x86_64 linux. During configuration AC_SEARCH_LIB([main],[lapack]) works, but compilation of the lapack units do not work, for example undefiend reference to 'dsyev_' --no lapack/blas routine goes unnoticed.
I've confirmed I have the libraries installed and even compiled them myself with the appropriate options to make them static with the same results.
Here is an example I had used in my first experience with LAPACK a few years ago that works dynamically, but not statically: http://pastebin.com/cMm3wcwF
The two methods I'm using to compile are the following,
gcc -llapack -o eigen eigen.c
gcc -static -llapack -o eigen eigen.c
Your linking order is wrong. Link libraries after the code that requires them, not before. Like this:
gcc -o eigen eigen.c -llapack
gcc -static -o eigen eigen.c -llapack
That should resolve the linkage problems.
To answer the subsequent question why this works, the GNU ld documentation say this:
It makes a difference where in the command you write this option; the
linker searches and processes libraries and object files in the order
they are specified. Thus, foo.o -lz bar.o' searches libraryz' after
file foo.o but before bar.o. If bar.o refers to functions in `z',
those functions may not be loaded.
........
Normally the files found this way are library files—archive files
whose members are object files. The linker handles an archive file by
scanning through it for members which define symbols that have so far
been referenced but not defined. But if the file that is found is an
ordinary object file, it is linked in the usual fashion.
ie. the linker is going to make one pass through a file looking for unresolved symbols, and it follows files in the order you provide them (ie. "left to right"). If you have not yet specified a dependency when a file is read, the linker will not be able to satisfy the dependency. Every object in the link list is parsed only once.
Note also that GNU ld can do reordering in cases where circular dependencies are detected when linking shared libraries or object files. But static libraries are only parsed for unknown symbols once.
My system is an older NAS running 2.6.32. I have found that when using -static for any subsequent library, it will also try to statically link any other library that I might need.
When I add the -Wl,-Bdynamic flag first and then explicitly name those libraries using -lc, such as "-Wl,-Bdynamic -lc -lstdc++" then it works. So what happens is that libc and others fail to be statically linked.
The static libc on the system is called /opt/lib/libc_nonshared.a.
The contents of /opt/lib/libc.so is this:
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/libc.so.6 /opt/lib/libc_nonshared.a )
The gcc version is 4.2.3. The current build command I am facing adds -dynamic at the end but this doesn't help much. When I add some static library directly using its .a name, and not using a -l flag, then there is no issue.
The problem seems to be that the dynamic library of libc came with the NAS, but the static version sits in /opt/lib.
I run:
gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -o hamming
I get:
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/bin/ld: cannot find -lc
collect2: ld returned 1 exit status
make: *** [hamming] Error 1
when I try to use static libc as is. Were I to perform a 'hack' to link libc_nonshared.a to libc.a, it suddenly does find it. But complains:
hamming.c:54: undefined reference to `malloc'
hamming.c:54: undefined reference to `memset'
And a zillion other errors of course. As mentioned above, /opt/libc.so contains the reference to both files (dynamic and static).
For libstdc++ only a .la file exists.
The -static linker flag does not take any argument. It is a boolean
flag that simply directs the linker to link no shared libraries, as
documented
-static
Do not link against shared libraries...
There is no need to explicitly direct the linker to link shared (dynamic)
libraries when it has a choice because that is the default bevaiour. If
you simply link, e.g.
gcc -o prog ... -lfoo ...
then the linker will link the first of libfoo.so (shared) or libfoo.a
(static) that it finds in any of the specified (-Ldir) or default
search directories, searched in commandline sequence. If it finds both
libfoo.so and libfoo.a in the same directory then it will choose
libfoo.so. Thus shared and static libraries may be freely intermixed
without any special options.
Specify -static only if you wish to link only static libraries.
If you wish to insist on linking a particular libfoo.a even when
libfoo.so is in the same directory and would be chosen by default,
use the explicit form of the -l option: -l:libfoo.a
Later
gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -o hamming
This command is failing with:
ld: cannot find -lc
because the linker (ld) cannot find a static library libc.a in
any of the specified linker search directories (-L. -L/opt/lib) or
the default linker search directories. If you wish instead to link
/opt/lib/libc_nonshared.a then your command should be:
>gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -lc_nonshared -o hamming
However, you have not explained why you want to link this program statically
(-static) in the first place, which is not the usual way and will require you to have installed
static versions of all libraries required for the linkage - both those
you explicitly link and the default libraries that gcc will add for C language
linkage (Standard C library, GCC runtime library).
Supposing you have a static library called (oddly) matrix.a (rather
than normally, libmatrix.a) that is located in /some/dir/, then the
normal way to compile and link your program would be:
gcc hamming.c -L/some/dir -l:matrix.a -o hamming
I suggest you start with that and deviate only as problems compel
you to.
The discovery of an /opt/lib/libc.so containing:
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/libc.so.6 /opt/lib/libc_nonshared.a )
is misleading you. This is not your shared libc. A shared library
is a binary. This is a linker script, and it says that your shared libc
is in fact /lib/libc.so.6. The linker will almost certainly find and use it by default.
I am trying to link a C++ module using GCC, essentially like this:
gcc -c hello.c
g++ -c world.cpp
gcc -ohello -lstdc++ hello.o world.o
Note that I use -lstdc++ to link the C++ module in, so that I can use gcc instead of g++. The problem is that I'm getting the error:
undefined reference to `operator new(unsigned long)'
(Assuming that world.cpp contains at least one call to new.)
This error is fixed if I put -lstdc++ at the end of the linker line, like this:
gcc -ohello hello.o world.o -lstdc++
I am aware that this question has been asked many times here, but I have a special requirement. I am not directly calling GCC. I am using a build system for a different programming language (Mercury) which is calling GCC on my behalf, and I can't easily modify the way it calls GCC (though I can specify additional libraries using the LDFLAGS environment variable). So I have two additional requirements:
I cannot use g++ to link (only gcc) -- that is why I am doing the -lstdc++ trick above rather than simply linking with g++).
I don't think that I can control the order of the linker commands -- Mercury will put the .o files on the command-line after any libraries.
I understand the basic reason why the order is important, but what is baffling me is why did this break now? I just updated to Ubuntu 11.10 / GCC 4.6.1. I have been successfully compiling this program for years using precisely the above technique (putting -lstdc++ first). Only now has this error come up. An unrelated program of mine links against OpenGL using -lgl and that too broke when I upgraded and I had to move -lgl to the end of the command-line. I'm probably going to discover that dozens of my programs no longer compile. Why did this change? Is there something wrong with my new system or is that the way it is now? Note that these are ordinary shared libraries, not statically linked.
Is there anything I can do to make GCC go back to the old way, where the order of libraries doesn't matter? Is there any other way I can convince GCC to link libstdc++ properly without moving it after the .o files on the command-line?
If Mercury puts object files after libraries, Mercury is broken. Libraries belong after object files - always. You may sometimes get away with the reverse order, but not reliably. (Static libraries must go after the object files that reference symbols in the static library. Sometimes, a linker will note the symbols defined by a shared library even when none of the symbols are used; sometimes, the linker will only note the shared library symbols if the shared library provides at least one symbol.)
I'm having a problem with my compiler telling me there is an 'undefined reference to' a function I want to use in a library. Let me share some info on the problem:
I'm cross compiling with gcc for C.
I am calling a library function which is accessed through an included header which includes another header, which contains the prototype.
I have included the headers directory using -I and i'm sure it's being found.
I'm first creating the .o files then linking them in a separate command.
So my thought is it might be the order in which I include the library files, but i'm not sure what is the correct way to order them. I tried with including the headers folder both before and after the .o file.
Some suggests would be great, and maybe and explanation of how the linker does its thing.
Thanks!
Response to answers
there is no .a library file, just .h and .c in the library, so -l isn't appropriate
my understanding of a library file is that it is just a collection of header and source files, but maybe it's a collection of .o files created from the source?!
there is no library object file being created, maybe there should be?? Yes seems I don't understand the difference between includes and libraries...i'll work on that :-)
Thanks for all the responses! I learned a lot about libraries. I'd like to put all the responses as the accepted answer :-)
Headers provide function declarations and function definitions. To allow the linker find the function's implementation (and get rid of the undefined reference) you need to ask the compiler driver (gcc) to link the specific library where the function resides using the -l flag. For instance, -lm will link the math library. A function's manual page typically specifies what library, if any, must be specified to find the function.
If the linker can't find a specified library you can add a library search path using the -L switch (for example, -L/usr/local/lib). You can also permanently affect the library path through the LIBRARY_PATH environment variable.
Here are some additional details to help you debug your problem. By convention the names of library files are prefixed with lib and (in their static form) have a .a extension. Thus, the statically linked version of the system's default math library (the one you link with -lm) typically resides in /usr/lib/libm.a. To see what symbols a given library defines you can run nm --defined-only on the library file. On my system, running the command on libm.a gives me output like the following.
e_atan2.o:
00000000 T atan2
e_asinf.o:
00000000 T asinf
e_asin.o:
00000000 T asin
To see the library path that your compiler uses and which libraries it loads by default you can invoke gcc with the -v option. Again on my system this gives the following output.
GNU assembler version 2.15 [FreeBSD] 2004-05-23 (i386-obrien-freebsd)
using BFD version 2.15 [FreeBSD] 2004-05-23
/usr/bin/ld -V -dynamic-linker /libexec/ld-elf.so.1 /usr/lib/crt1.o
/usr/lib/crti.o /usr/lib/crtbegin.o -L/usr/lib /var/tmp//ccIxJczl.o -lgcc -lc
-lgcc /usr/lib/crtend.o /usr/lib/crtn.o
It sounds like you are not compiling the .c file in the library to produce a .o file. The linker would look for the prototype's implementation in the .o file produced by compiling the library
Does your build process compile the library .c file?
Why do you call it a "library" if it's actually just source code?
I fear you mixed the library and header concepts.
Let's say you have a library libmylib.a that contains the function myfunc() and a corresponding header mylib.h that defines its prototype. In your source file myapp.c you include the header, either directly or including another header that includes it. For example:
/* myapp.h
** Here I will include and define my stuff
*/
...
#include "mylib.h"
...
your source file looks like:
/* myapp.c
** Here is my real code
*/
...
#include "myapp.h"
...
/* Here I can use the function */
myfunc(3,"XYZ");
Now you can compile it to obtain myapp.o:
gcc -c -I../mylib/includes myapp.c
Note that the -I just tells gcc where the headers files are, they have nothing to do with the library itself!
Now you can link your application with the real library:
gcc -o myapp -L../mylib/libs myapp.o -lmylib
Note that the -L switch tells gcc where the library is, and the -l tells it to link your code to the library.
If you don't do this last step, you may encounter the problem you described.
There might be other more complex cases but from your question, I hope this would be enough to solve your problem.
Post your makefile, and the library function you are trying to call. Even simple gcc makefiles usually have a line like this:
LIBFLAGS =-lc -lpthread -lrt -lstdc++ -lShared -L../shared
In this case, it means link the standard C library, among others
I guess you have to add the path where the linker can find the libraray. In gcc/ld you can do this with -L and libraray with -l.
-Ldir, --library-path=dir
Search directory dir before standard
search directories (this option must
precede the -l option that searches
that directory).
-larch, --library=archive
Include the archive file arch in the
list of files to link.
Response to answers - there is no .a library file, just .h and .c in the library, so -l isn't approriate
Then you may have to create the libraray first?
gcc -c mylib.c -o mylib.o
ar rcs libmylib.a mylib.o
I have encountered this problem when building a program with a new version of gcc. The problem was fixed by calling gcc with the -std=gnu89 option. Apparently this was due to inline function declarations. I have found this solution at https://gcc.gnu.org/gcc-5/porting_to.html