I'm using CMake in a project, and I'm trying to statically link some libraries.
I've set:
set(BUILD_SHARED_LIBS OFF)
set(CMAKE_EXE_LINKER_FLAGS "-static-libgcc -static-libstdc++ -static")
set_target_properties(icarus PROPERTIES LINK_SEARCH_END_STATIC 1)
And I've made sure when looking for the actual libraries that I have the *.a version of them.
Currently the project imports:
libPocoNet.a
libPocoUtil.a
libPocoXML.a
libPocoFoundation.a
libmysqlclient.a
libmysqlpp.a
libcrypto++.a
CUDA
All libraries are found, and when doing a dynamic/shared linkage, they work fine.
I have also tried to set compilation flags:
set(GCC_CXX_FLAGS ${GCC_CXX_FLAGS} "-static-libgcc -static-libstdc++ -static")
But to no avail.
While I get no issues while compiling, linkage is throwing alot of undefined reference errors for calls found in the above libraries, i.e:
undefined reference to `mysql_thread_init'
undefined reference to `mysql_real_query'
undefined reference to `pthread_mutex_unlock'
undefined reference to `Poco::ErrorHandler::handle()'
Not in that particular order, and numerous errors for each library.
Looking at the last line of GCC I see:
/usr/bin/c++ -g -g -static-libgcc -static-libstdc++ -static [list of *.cpp files]
-o icarus -rdynamic /usr/local/lib/libPocoFoundation.a /usr/local/lib/libPocoNet.a
/usr/local/lib/libPocoUtil.a /usr/local/lib/libPocoXML.a
-Wl,-Bstatic -lmysqlclient -lmysqlpp -lcrypto++
Which makes me wonder:
Why are Poco libraries linked as -rdynamic, and there is no -Wl -Bstatic flag? As if they are skipped/excluded from static linkage.
mysqlclient, mysqlpp and crypto++ seem to be set for static linkage, yet I still get errors
So, could someone please explain to me:
How do I setup for partial static linkage using CMake
Is CMAKE_EXE_LINKER_FLAGS the only one I need to set?
Should I be forcing static linking for mentioned libraries but not for entire project?
Please excuse me if those are too many or too localized questions, I haven't tried this before, and I can't seem to find much info on the net.
I've managed to solve my problem by using the following:
#Dynamic/Shared Libs
...
#Static start
set_target_properties(icarus PROPERTIES LINK_SEARCH_START_STATIC 1)
set_target_properties(icarus PROPERTIES LINK_SEARCH_END_STATIC 1)
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
#Static Libs
...
#Set Linker flags
set(CMAKE_EXE_LINKER_FLAGS "-static-libgcc -static-libstdc++")
This works without passing a -static which creates other big issues, and can essentially mix static and dynamic libraries.
As long as the order of static libraries is correct, and as long as dependencies of static libraries are satisfied, I get an ELF using some dynamic libraries (i.e. in my case mysqlclient, libmysql++) and the rest are all static libraries (crypto++, PocoNet, PocoUtil, PocoXML, PocoFoundation).
Bear in mind that static linked libraries have their own dependencies. Examining my debug application using readelf -d app, I see:
Dynamic section at offset 0x508f88 contains 28 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libmysqlpp.so.3]
0x0000000000000001 (NEEDED) Shared library: [libmysqlclient.so.18]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
I know that pthread is imported by Poco::Runnable, libm is for math operations, etc.
I am still unaware if this is the right way to use CMake for partial static linking.
In the case of Debian packaged libraries, such as crypto++, mysql++, mysqlclient, simply finding the *.a library worked, but in case of Poco Libraries, which only got me the full path and name of the library, but not a flag, -Bdynamic could only be turned off by using the above lines.
Note: Poco could not be linked statically, without -static-libstdc++
I hope this helps anyone stuck at something similar.
How do I setup for static linkage using CMake
Well... you don't :) That's not how CMake works: in CMake, you first find the absolute path of a library, then link to it with target_link_libraries.
So, if you want to link to a static library, you need to search for that static library:
find_library(SOMELIB libsomelib.a)
instead of:
find_library(SOMELIB somelib)
For recently verions of cmake, it smart enough to link lib as static if you specify full file name of .a file.
for example, the following line
TARGET_LINK_LIBRARIES(pfs ... liburing.a ...)
it will generate compile cmd line:
ld ... -Wl,-Bstatic -luring -Wl,-Bdynamic ...
while cmd line the syntax is GCC's syntax to link against static lib. https://stackoverflow.com/a/4500201/388354
Related
When studying Scintilla's makefile for MinGW under windows, I noticed that it is passing -shared and -static together as LDFLAGS to gcc.
LDFLAGS=-shared -static -mwindows $(LDMINGW)
I googled, and only find some information from clang: https://reviews.llvm.org/D43811
[MinGW, CrossWindows] Allow passing -static together with -shared
In these combinations, link a DLL as usual, but pass -Bstatic instead of -Bdynamic to indicate prefering static libraries.
My question is: Would GCC do the same?
I haven't find any proof yet.
You can pass both -static and -shared in a GCC linkage. Their
combined effect is the same as you found described in your llvm link,
and this has always been the case for GCC.
-shared directs a GCC linkage to produce a shared library rather than a program,
which it achieves by passing on the option -shared to its invocation of
the linker.
-static directs a GCC linkage to ignore shared libraries when resolving
input library options -lname. By default -lname would be resolved by
searching the specified or default linker search directories for either
the shared library libname.so (on Windows, [lib]name.dll)
or the static library libname.a (on Windows also [lib]name.lib) and to prefer
the shared library if both of them are found in the same directory. -static
simply excludes all shared libraries from the search. GCC achieves this by passing the option -Bstatic
through to its invocation of the linker at a position in the generated linker
commandline that precedes all of the -lname options.
The GNU linker documentation of -Bstatic is explicit
that this option is consistent with -shared and that the effect is to produce a shared library
all of whose dependent libraries have been statically resolved.
-Bstatic
-dn
-non_shared
-static
Do not link against shared libraries. This is only meaningful on platforms for which shared libraries are supported.
The different variants of this option are for compatibility with various systems. You may use this option multiple times on the command line:
it affects library searching for -l options which follow it. This option also implies --unresolved-symbols=report-all.
This option can be used with -shared. Doing so means that a shared library is being created but that all of the library’s external references must be resolved by pulling in entries from static libraries.
(emphasis mine).
Although static linkage of shared library is in principle just a linkage restricted
in the same way as static linkage of a program, in practice it frequently encounters
a snag on Unix and Linux because all the object code linked into an ELF shared library
libname.so must be Position Independent Code,
as produced by the GCC compilation option -fPIC, whereas object files that are destined to be
archived in static libraries are customarily not compiled with -fPIC. Linkages using
-shared ... -static are thus apt to fail because necessary static libraries contain
non-PIC object files.
You do not have this worry with GCC on Windows, however, because there
is no such distinction as PIC v. non-PIC in Windows PE
object code.
My system is an older NAS running 2.6.32. I have found that when using -static for any subsequent library, it will also try to statically link any other library that I might need.
When I add the -Wl,-Bdynamic flag first and then explicitly name those libraries using -lc, such as "-Wl,-Bdynamic -lc -lstdc++" then it works. So what happens is that libc and others fail to be statically linked.
The static libc on the system is called /opt/lib/libc_nonshared.a.
The contents of /opt/lib/libc.so is this:
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/libc.so.6 /opt/lib/libc_nonshared.a )
The gcc version is 4.2.3. The current build command I am facing adds -dynamic at the end but this doesn't help much. When I add some static library directly using its .a name, and not using a -l flag, then there is no issue.
The problem seems to be that the dynamic library of libc came with the NAS, but the static version sits in /opt/lib.
I run:
gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -o hamming
I get:
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/bin/ld: cannot find -lc
collect2: ld returned 1 exit status
make: *** [hamming] Error 1
when I try to use static libc as is. Were I to perform a 'hack' to link libc_nonshared.a to libc.a, it suddenly does find it. But complains:
hamming.c:54: undefined reference to `malloc'
hamming.c:54: undefined reference to `memset'
And a zillion other errors of course. As mentioned above, /opt/libc.so contains the reference to both files (dynamic and static).
For libstdc++ only a .la file exists.
The -static linker flag does not take any argument. It is a boolean
flag that simply directs the linker to link no shared libraries, as
documented
-static
Do not link against shared libraries...
There is no need to explicitly direct the linker to link shared (dynamic)
libraries when it has a choice because that is the default bevaiour. If
you simply link, e.g.
gcc -o prog ... -lfoo ...
then the linker will link the first of libfoo.so (shared) or libfoo.a
(static) that it finds in any of the specified (-Ldir) or default
search directories, searched in commandline sequence. If it finds both
libfoo.so and libfoo.a in the same directory then it will choose
libfoo.so. Thus shared and static libraries may be freely intermixed
without any special options.
Specify -static only if you wish to link only static libraries.
If you wish to insist on linking a particular libfoo.a even when
libfoo.so is in the same directory and would be chosen by default,
use the explicit form of the -l option: -l:libfoo.a
Later
gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -o hamming
This command is failing with:
ld: cannot find -lc
because the linker (ld) cannot find a static library libc.a in
any of the specified linker search directories (-L. -L/opt/lib) or
the default linker search directories. If you wish instead to link
/opt/lib/libc_nonshared.a then your command should be:
>gcc hamming.c -static -L. -L/opt/lib -l:matrix.a -lc_nonshared -o hamming
However, you have not explained why you want to link this program statically
(-static) in the first place, which is not the usual way and will require you to have installed
static versions of all libraries required for the linkage - both those
you explicitly link and the default libraries that gcc will add for C language
linkage (Standard C library, GCC runtime library).
Supposing you have a static library called (oddly) matrix.a (rather
than normally, libmatrix.a) that is located in /some/dir/, then the
normal way to compile and link your program would be:
gcc hamming.c -L/some/dir -l:matrix.a -o hamming
I suggest you start with that and deviate only as problems compel
you to.
The discovery of an /opt/lib/libc.so containing:
OUTPUT_FORMAT(elf32-littlearm)
GROUP ( /lib/libc.so.6 /opt/lib/libc_nonshared.a )
is misleading you. This is not your shared libc. A shared library
is a binary. This is a linker script, and it says that your shared libc
is in fact /lib/libc.so.6. The linker will almost certainly find and use it by default.
For example, I have 3 different instances:
Shared library with undefined reference "get_object" and definition
"x"
Static library with definitions "get_object", "y".
Binary file Object file with undefined references "x" and "y".
Am I right that linker can resolve all this symbols when I'll try to compile and link all this stuff together?
Main point of my worry is symbol "get_object".
Can linker resolve such references between shared and static libraries?
Static libraries are just an agglomeration of object files (their members), perhaps with a ranlib(1) generated index.
On Linux, if you link an object file (3) foo.o with a static library (2) libee.a and a shared library (1) libyz.so and if you pass -rdynamic at link time (i.e. gcc -rdynamic foo.o libee.a libyz.so -o myprog or gcc -rdynamic foo.o -lee -lyz -o myprog) then dynamic linker would resolve the get_object name at dynamic link time (in ld-linux.so)
Details are explained in ELF wikipage and Drepper's paper: How To Write Shared Libraries. Read also Levine's book: Linkers and loaders & ld(1) man page.
first I'll create a small program:
#include <gmpxx.h>
void function() {
//mpf_class num;
return;
}
int main() {}
Notice, that I am including a 3rd party library, but not using any of its contents.
I go to compile the program:
$ g++ main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Even though I linked two additional libraries to this program, I passed the flag --as-nedded which discovered that those two libraries were not actually needed at run-time. Therefore, readelf shows us that they were not actually linked.
(gcc actually links additional c and c++ libraries to your binary besides libc, but the --as-needed flag discovered that those too, were not needed.)
Here I reference the library code, but still don't actually need it:
#include <gmpxx.h>
void function() {
mpf_class num;
return;
}
int main() {}
$ g++ main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libgmp.so.10]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Question:
I was under the assumption that unused functions get thrown away, and don't become part of the binary. Assuming the compiler realizes that the "function" code is not needed, and can toss it, why can't it also realize that the linking of gmp is also no longer needed?
Interestingly, running the command with full optimization,
$ g++ -Ofast main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libgmp.so.10]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
results in some of gcc's implicitly linked libraries to be excluded when previously unoptimized were kept due to gmp's inclusion.
Question:
Is the compiler capable of excluding that 3rd party library given that the library code will never actually be executed?
If the compiler is absolutely sure some code won't ever be used, it is allowed to just throw it away. If so, the call won't show up in the object, and the library ommited.
In the example function that isn't called, it is by default extern, so the compiler can't know if it is called from another file, and isn't allowed to get rid of it. But most compilers just compile a function at a time, and do little intra-function analysis. So I doubt any current compiler will compile that file (with a static void function()), notice the function isn't called, and delete it. Perhaps a smart linker (which does whole-program analysis) will do so, but I haven't looked too closely at that.
I'm having trouble building a little program that uses Boost.Test on my Mac with a Boost installed by MacPorts at /opt/local/lib/
Here's my minimal source file, test.cpp:
#define BOOST_TEST_MODULE MyTest
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_CASE(test1) {
}
and my CMakeLists.txt:
cmake_minimum_required(VERSION 2.6)
project (test)
find_package(Boost COMPONENTS unit_test_framework REQUIRED)
add_executable(test test.cpp)
and an excerpt of from make VERBOSE=1:
[100%] Building CXX object CMakeFiles/test.dir/test.cpp.o
g++ -o CMakeFiles/test.dir/test.cpp.o -c /Users/exclipy/Code/cpp/inline_variant/question/test.cpp
Linking CXX executable test
"/Applications/CMake 2.8-5.app/Contents/bin/cmake" -E cmake_link_script CMakeFiles/test.dir/link.txt --verbose=1
g++ -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/test.dir/test.cpp.o-o test
Undefined symbols for architecture x86_64:
"_main", referenced from:
start in crt1.10.6.o
"vtable for boost::unit_test::unit_test_log_t", referenced from:
boost::unit_test::unit_test_log_t::unit_test_log_t() in test.cpp.o
boost::unit_test::unit_test_log_t::~unit_test_log_t() in test.cpp.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
As you can see, it doesn't know how to link to Boost library. So I try adding to CMakeLists.txt:
target_link_libraries(test boost_unit_test_framework)
But I just get:
g++ -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/test.dir/test.cpp.o-o test -lboost_unit_test_framework
ld: library not found for -lboost_unit_test_framework
From lots of trial and error, I've found that manually running this works:
$ g++ test.cpp -L/opt/local/lib -lboost_unit_test_framework -DBOOST_TEST_DYN_LINK
But after hours of fiddling, I can't get it to build from CMake. I don't care whether it links dynamically or statically, I just want it to work.
You need to tell CMake where to find the boost libraries (the -L/opt/local/lib in your g++ line). You can accomplish this by adding the following line (if you had no problem with find_package):
link_directories ( ${Boost_LIBRARY_DIRS} )
before add_executable.
Another alternative is using the single-header variant of the UTF. This variant is really simple (you only need to include <boost/test/included/unit_test.hpp> but it has a major drawback in its considerable increase in build time.
The find_package(Boost COMPONENTS ...) call collects the required link libraries for the searched Boost components (e.g.,unit_test_framework) in the CMake variable Boost_LIBRARIES.
To get rid of the link error, add:
target_link_libraries(test ${Boost_LIBRARIES})
EDIT 2020-02
The details of building a Boost.Test module are on the documentation here and covered with many examples on the documentation. Usually if main is not found, this might be due to:
mixing static and shared library versions of Boost.Test (the linker prefers shared libraries)
improper definitions of the BOOST_TEST_MODULE and/or BOOST_TEST_DYN_LINK macros: depending on those, the Boost.Test framework will define (correctly) a main or not.
Previous (wrong) answer
Well here, the problem is not that cmake does not find the boost_unit_test_framework library, but rather that this specific library does not contain the main entry point for running the binary.
In fact, you should link against ${Boost_TEST_EXEC_MONITOR_LIBRARY} since it contains the proper definitions. You should also avoid defining the macro BOOST_TEST_DYN_LINK.