I have inherited a Makefile which builds a .so file. It is linking with -lcrypto from OpenSSL on Ubuntu with gcc 4.7.4. Critically, it is NOT linking with -lssl nor -ldl, and when I run nm -g thelib.so, it only has the ~15 symbols from openssl crypto. However, they are all U (undefined).
I'm refactoring the Makefile on another Ubuntu machine. When I link with -lcrypto, it fails due to undefined symbols needed from dl. When I add linking to -ldl, those errors go away and linking succeeds. However, my .so file is 1.5 MB bigger than the original, and there are at least a hundred symbols related to SSL, which are all T (defined), which seem to indicate that -lssl is happening implicitly somehow.
While it would seem prudent and good that they are all defined in my case, I need to figure out how to produce the same result just as it is.
So, my question is, how does one get GCC to allow the linking of a .so file and accept undefined references? I've compared our commands, and there are little differences which I've tried to eliminate, but nothing seems to work. I read that it might be related to -Wl,--no-as-needed, but i'm using that. Here's my linker flags.
g++ -shared -o mylib.so myobjs.o -fPIC -lstdc++ -lm -z defs -Wl,-soname,mylib -Wl,--no-as-needed -lpthread -lcrypto -lz
On the other system (the one with the larger result), OpenSSL has apparently not been built as a shared object, only as a static library (but maybe as PIC, so that you can link the result into a shared object). You will have to install the packages that provide the shared object and the corresponding .so symbolic link.
Related
I have written a small code that I want to compile with a combination of static and dynamic libs. The code uses functions from hdf5 and exodusII (a specialist CAE lib) as well as math, and of course good-old stdio.
To make the binary highly portable, I wanted to link hdf5 and exodusII statically into the code, but leave math and libc as shared, so that the code is optimised on different platforms.
I cannot work out what the correct method is to compile something like this. I already have tried:
gcc -lm -lc -fPIC test1.c /usr/lib/libexodus.a /usr/lib/libhdf5.a -Wl,-pie
This gives the error:
/usr/lib64/crt1.o: relocation R_X86_64_32S against '__libc_csu_fini' can not be used when making a shared object; recompile with -fPIC
/usr/lib64/crt1.o: could not read symbols: Bad value
I have also tried:
gcc -c test1.c (WORKS!)
ld /usr/lib/libhdf5.a /usr/lib/libexodus.a -lm -lc test1.o
Which gives a warning of not being able to find an entry symbol _start followed by a whole lot of undefined reference errors to the libexodus functions in test1.c. (I have checked libexodus.a with nm, and the functions being reported do actually exist in the archive.
I would really appreciate a hand in this. I am not overly experienced in using static libs, but for this application, I think it is the best choice, so long as I can work out a reliable way of compiling and linking.
The error was in the linking order. I have now learned that the linking order works like babushka dolls where the library at the top of the dependency tree comes first, and the most general library comes last.
For future reference, in the end, the working build command was as follows:
gcc test1.c -lexodus -lnetcdf -lhdf5_hl -hdf5 -lcurl -ldl -lm
What I don't really understand is that when I had built the exodus library as a shared library, I only had to link against the shared library, and the dependency libraries (-lnetcdf -lhdf5_hl -lhdf5 -lcurl) were not specified; however, with the static compilation of the exact same library, I now need to link all the libraries explicitly.
If someone has an answer to this behaviour, it would be helpful for my understanding and very appreciated, but as I can continue coding with this current build method, it is not an urgent matter.
I have a C project using several object files that needs to be linked in a specific order to find all needed symbols.
For example this command works fine (lib2.o depends on lib1.o etc.)
gcc -o my_app main.o lib1.o lib2.o lib3.o -lm
but
gcc -o my_app main.o lib3.o lib2.o lib1.o -lm
ends with undefined reference to `my_variable' errors.
This is a known behavior and can be solved for example by adding these objects to GROUP section in a linker script.
Now I'd like to share these object as a static library with my colleagues. So...
ar -rcs mylib.a lib1.o lib2.o lib3.o
gcc -o my_app main.o mylib.a -lm
Unfortunately this gives the same undefined reference errors like the specifying the objects in incorrect order.
I have not found any linker or archiver options to make it working and also no solution by googling even if I think that this problem should be relatively common.
Do please anybody know a solution?
regards
Jan
This might be a link order problem. When the GNU linker sees a library, it discards all symbols that it doesn't need. It also does that in the sequential order form left to right.
Recent versions of gcc/ld default to linking with --as-needed flag.
This means if you write -lmylib.a before the C file the library will automatically get excluded (the order matters when testing if things are "needed" like this)
You can fix this with either:
gcc -L. -o example example.c -lmylib.a
gcc -L. -Wl,--no-as-needed -o example example.c -lmylib.a
The latter of which passes --no-as-needed to the linker, which would cause the library to still be linked, even if you didn't call any function external from it.
Your error implies that the problem is in one of your lib?.o files [lib{later}.o depends on lib{earlier}.o]
How did you manage to compile them?
Were there any compilation warnings?
It has been a while since I used C, but I think that you will need to include dependent libraries within the library that has the dependency - this may be the reason why you can't find too many references to the problem, because it does not really exist.
I am trying to link a C++ module using GCC, essentially like this:
gcc -c hello.c
g++ -c world.cpp
gcc -ohello -lstdc++ hello.o world.o
Note that I use -lstdc++ to link the C++ module in, so that I can use gcc instead of g++. The problem is that I'm getting the error:
undefined reference to `operator new(unsigned long)'
(Assuming that world.cpp contains at least one call to new.)
This error is fixed if I put -lstdc++ at the end of the linker line, like this:
gcc -ohello hello.o world.o -lstdc++
I am aware that this question has been asked many times here, but I have a special requirement. I am not directly calling GCC. I am using a build system for a different programming language (Mercury) which is calling GCC on my behalf, and I can't easily modify the way it calls GCC (though I can specify additional libraries using the LDFLAGS environment variable). So I have two additional requirements:
I cannot use g++ to link (only gcc) -- that is why I am doing the -lstdc++ trick above rather than simply linking with g++).
I don't think that I can control the order of the linker commands -- Mercury will put the .o files on the command-line after any libraries.
I understand the basic reason why the order is important, but what is baffling me is why did this break now? I just updated to Ubuntu 11.10 / GCC 4.6.1. I have been successfully compiling this program for years using precisely the above technique (putting -lstdc++ first). Only now has this error come up. An unrelated program of mine links against OpenGL using -lgl and that too broke when I upgraded and I had to move -lgl to the end of the command-line. I'm probably going to discover that dozens of my programs no longer compile. Why did this change? Is there something wrong with my new system or is that the way it is now? Note that these are ordinary shared libraries, not statically linked.
Is there anything I can do to make GCC go back to the old way, where the order of libraries doesn't matter? Is there any other way I can convince GCC to link libstdc++ properly without moving it after the .o files on the command-line?
If Mercury puts object files after libraries, Mercury is broken. Libraries belong after object files - always. You may sometimes get away with the reverse order, but not reliably. (Static libraries must go after the object files that reference symbols in the static library. Sometimes, a linker will note the symbols defined by a shared library even when none of the symbols are used; sometimes, the linker will only note the shared library symbols if the shared library provides at least one symbol.)
and sorry for my not really good english. I'll try my best :)
I am trying to compile a addin for my Casio graphic calculator in C. This works without problems when using the official SDK. Because it is only available for Windows, I want to use gcc.
So I got sh-rtems-gcc and it's binutils from macports and tried to compile my program according to this instructions. I copy-pasted the described addin.ld and crt0.s and placed my main.c and libfxsys.a (from the same guys as the instructions mentioned above) in the same directory. The sub-dir include contains fxsys' headers. I verified the presence of all the functions of the library in the .a file with nm.
When using this command for compilation:
sh-rtems-gcc-4.2.3 -m3 -mb -nostdlib -I./include -c crt0.s main.c
Everything works fine. But then im trying to link:
sh-rtems-gcc-4.2.3 -m3 -mb -nostdlib -L. -o myaddin.elf -Taddin.ld crt0.o main.o -lfxsys
and get the following error:
main.o: In function `__main':
main.c:(.text+0x248): undefined reference to `_Bdisp_AllClr_VRAM'
...
... (--- cut 16 other errors in the same format ---)
...
main.c:(.text+0x360): undefined reference to `_Sleep'
./libfxsys.a(locate.o): In function `_locate':
locate.c:(.text+0x28): undefined reference to `_locate_OS'
collect2: ld gab 1 als Ende-Status zurück
All the missing symbols are in the libfxsys.a. I have verified this with nm.
I have already played with the positions of the library in the command, as this is often mentioned as a source of failure in other posts found in google, but without success. I also tried adding and removing the -lgcc option that is used in the above mentioned instructions, without success.
My Host-Machine is a Intel Mac, OS X 10.6
Because I have no idea how to solve this problem, and get to compile my program, I have to ask: What am I doing wrong? How can I compile my program without using the SDK?
Thanks in advance,
xythobuz
Edit:
I have also tried linking with:
sh-rtems-ld -EB -L. -o myaddin.elf -Taddin.ld crt0.o --start-group main.o libfxsys.a --end-group
But it produces the same output as above.
I can't say the exact problem, but would investigate like this:
Find the library that contains the missing symbols. Use nm to see symbol names
Once you know which library contains the symbols make sure you're linking to it, and in the correct order. Try using recursive symbol resolution options -( -) with your linker.
I am trying to write a matlab mex function which uses libhdf5; My Linux install provides libhdf5-1.8 shared libraries and headers. However, my version of Matlab, r2007b, provides a libhdf5.so from the 1.6 release. (Matlab .mat files bootstrap hdf5, evidently). When I compile the mex, it segfaults in Matlab. If I downgrade my version of libhdf5 to 1.6 (not a long-term option), the code compiles and runs fine.
question: how do I solve this problem? how do I tell the mex compilation process to link against /usr/lib64/libhdf5.so.6 instead of /opt/matlab/bin/glnxa64/libhdf5.so.0 ? When I try to do this using -Wl,-rpath-link,/usr/lib64 in my compilation, I get errors like:
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libhdf5.so.0, needed by /opt/matlab/matlab75/bin/glnxa64/libmat.so, may conflict with libhdf5.so.6
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status
mex: link of 'hdf5_read_strings.mexa64' failed.
make: *** [hdf5_read_strings.mexa64] Error 1
ack. the last resort would be to download a local copy of the hdf5-1.6.5 headers and be done with it, but this is not future proof (a Matlab version upgrade is in my future.). any ideas?
EDIT: per Ramashalanka's excellent suggestions, I
A) called mex -v to get the 3 gcc commands; the last is the linker command;
B) called that linker command with a -v to get the collect command;
C) called that collect2 -v -t and the rest of the flags.
The relevant parts of my output:
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crti.o
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/crtbeginS.o
hdf5_read_strings.o
mexversion.o
-lmx (/opt/matlab/matlab75/bin/glnxa64/libmx.so)
-lmex (/opt/matlab/matlab75/bin/glnxa64/libmex.so)
-lhdf5 (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/libhdf5.so)
/lib64/libz.so
-lm (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/libm.so)
-lstdc++ (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libstdc++.so)
-lgcc_s (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libgcc_s.so)
/lib64/libpthread.so.0
/lib64/libc.so.6
/lib64/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libgcc_s.so)
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/crtendS.o
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crtn.o
So, in fact the libhdf5.so from /usr/lib64 is being referenced. However, this is being overriden, I believe, by the environment variable LD_LIBRARY_PATH, which my version of Matlab automagically sets at run-time so it can locate its own versions of e.g. libmex.so, etc.
I am thinking that the crt_file.c example works either b/c it does not use the functions I am using (H5DOpen, which had a signature change in the move from 1.6 to 1.8 (yes, I am using -DH5_USE_16_API)), or, less likely, b/c it does not hit the parts of Matlab internals that need hdf5. ack.
The following worked on my system:
Install hdf5 version 1.8.4 (you've already done this: I installed the source and compiled to ensure it is compatible with my system, that I get gcc versions and that I get the static libraries - e.g. the binaries offered for my system are icc specific).
Make a target file. You already have your own file. I used the simple h5_crtfile.c from here (a good idea to start with this simple file first a look for warnings). I changed main to mexFunction with the usual args and included mex.h.
Specify the static 1.8.4 library you want to load explicitly (the full path with no -L for it necessary) and don't include -lhdf5 in the LDFLAGS. Include a -t option so you can ensure that there is no dynamic hdf5 library being loaded. You also need -lz, with zlib installed. For darwin we also need a -bundle in LDFLAGS:
mex CFLAGS='-I/usr/local/hdf5/include' LDFLAGS='-t /usr/local/hdf5/lib/libhdf5.a -lz -bundle' h5_crtfile.c -v
For linux, you need an equivalent position-independent call, e.g. fPIC and maybe -shared, but I don't have a linux system with a matlab license, so I can't check:
mex CFLAGS='-fPIC -I/usr/local/hdf5/include' LDFLAGS='-t /usr/local/hdf5/lib/libhdf5.a -lz -shared' h5_crtfile.c -v
Run the h5_crtfile mex file. This runs without problems on my machine. It just does a H5Fcreate and H5Fclose to create "file.h5" in the current directory, and when I call file file.h5 I get file.h5: Hierarchical Data Format (version 5) data.
Note that if I include a -lhdf5 above in step 3, then matlab aborts when I try to run the executable (because it then uses matlab's dynamic libraries which for me are version 1.6.5), so this is definitely solving the problem on my system.
Thanks for the question. My solution above is definitely much easier for me than what I was doing before. Hopefully the above works for you.
I am accepting Ramashalanka's answer because it led me to the exact solution which I will post here for completeness only:
download the hdf5-1.6.5 library from the hdf5 website, and install the header files in a local directory;
tell mex to look for "hdf5.h" in this local directory, rather than in the standard location (e.g. /usr/include.)
tell mex to compile my code and the shared object library provided by matlab, and do not use the -ldfh5 flag in LDFLAGS.
the command I used is, essentially:
/opt/matlab/matlab_default/bin/mex -v CC#gcc CXX#g++ CFLAGS#"-Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include" CXXFLAGS#"-Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include " -O -lmwblas -largeArrayDims -L/usr/lib64 hdf5_read_strings.c /opt/matlab/matlab_default/bin/glnxa64/libhdf5.so.0
this gets translated by mex into the commands:
gcc -c -I/opt/matlab/matlab75/extern/include -DMATLAB_MEX_FILE -Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include -O -DNDEBUG hdf5_read_strings.c
gcc -c -I/opt/matlab/matlab75/extern/include -DMATLAB_MEX_FILE -Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include -O -DNDEBUG /opt/matlab/matlab75/extern/src/mexversion.c
gcc -O -pthread -shared -Wl,--version-script,/opt/matlab/matlab75/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined -o hdf5_read_strings.mexa64 hdf5_read_strings.o mexversion.o -lmwblas -L/usr/lib64 /opt/matlab/matlab_default/bin/glnxa64/libhdf5.so.0 -Wl,-rpath-link,/opt/matlab/matlab_default/bin/glnxa64 -L/opt/matlab/matlab_default/bin/glnxa64 -lmx -lmex -lmat -lm -lstdc++
this solution should work on all my various target machines and at least until I upgrade to matlab r2009a, which I believe uses hdf5-1.8. thanks for all the help, sorry for being so dense with this--I think I was overly-committed to using the packaged version of hdf5, rather than a local set of header files.
Note this would all have been trivial if Mathworks had provided a set of the header files with the Matlab distribution...