GCC differently treats an object and a static library regarding undefined symbols - gcc

Recently I discoved that Linux linker does not fail due to undefined symbols from static libraries, however does fail due to the same undefined symbols if I link directly with te object files. Here is a simple example:
Source code:
$ cat main.c
int main() { return 0; }
$ cat src.c
int outerUnusedFunc() {
return innerUndefinedFunc();
}
int innerUndefinedFunc();
Creating *.o and *.a from it, comparing using "nm":
$ gcc -c -o main.o main.c
$ gcc -c -o src.o src.c
$ ar r src.a src.o
ar: creating src.a
$ nm src.o
U innerUndefinedFunc
0000000000000000 T outerUnusedFunc
$ nm src.a
src.o:
U innerUndefinedFunc
0000000000000000 T outerUnusedFunc
(Here we clearly see that both *.o and *.a contain the equal symbols list)
And now...
$ ld -o exe main.o src.o
src.o: In function `outerUnusedFunc':
src.c:(.text+0xa): undefined reference to `innerUndefinedFunc'
$ echo $?
1
$ ld -o exe main.o src.a
$ echo $?
0
What is the reason for GCC to treat it differenty?

If you read the static-libraries tag wiki
it will explain why no object files from src.a are linked into your program, and therefore
why it doesn't matter what undefined symbols are referenced in them.
The difference between an object file foo.o and a static library libfoo.a, as linker inputs, is
that an object file is always linked into your program, unconditionally, whereas the same object file
in a static library library, libfoo.a(foo.o), is extracted from libfoo.a and linked into
the program only if the linker needs it to carry on the linkage, as explained by the tag wiki.
Naturally, the linker will give errors only for undefined references in object files that are linked into the program.
The behaviour you are observing is behaviour of the linker, whether or not you invoke it via a GCC
front-end.
Giving the linker foo.o tells it: I want this in the program. Giving the linker
libfoo.a tells it: Here are some object files that you might or might not need.

In the second case — with static library — command line with says "build exe from main.o and add all required things from src.a".
ld just ignores the library because no external symbols required for main.o (outerUnusedFunc is not referenced from main.o).
But in the first case command line says "build exe from main.o and src.o".
ld should place src.o content into output file.
Hence, it obligate to analyze src.o module, add outerUnusedFunc into output file and resolve all symbols for outerUnusedFunc despite it is unused.
You can enable garbage collection for code sections
gcc --function-sections -Wl,--gc-sections -o exe main.c src.c
In this case outerUnusedFunc (as well as all other functions) will be placed
in separate section. ld will see that this section unused (no symbols referenced). It will remove all the section from output file so that innerUndefinedFunc would not be referenced and the symbol should not be resolved — the same result as for library case.
On the other hand, you can manually reference outerUnusedFunc as "undefined" so that ld should find it in library and add to output file.
ld -o exe main.o -u outerUnusedFunc src.a
in this case the same error (undefined reference to innerUndefinedFunc) will be produced.

Related

How to see the object file contents of a .so file

how to see what .o files constitute .so file?
Means how to notice what are the object files are used to build the .so file from the .so file (If I have only the .so file)
You can't know, given just a shared library, what object files were
compiled into it. If you're lucky, you may be able to make a reasonable guess.
A shared library is made, by the linker, from object files and
possibly other shared libraries, but it does not contain the object files
or shared libraries from which it was made. A static library, on the other hand, which
is made by the archiver ar, does contain object
files: it is just an ar archive of object files.
If a shared library has not been stripped of debugging information, then
for debugging purposes its symbol table will contain the names of the source files
from which the object files were compiled that were linked in the shared library - at least those source files which were compiled with debugging information.
From the names of those source files you can infer the names of the object files
with reasonable confidence, but not with certainty.
For example, here we make a shared library from source files foo.c and bar.c.
Compile the source files to object files:
$ gcc -Wall -fPIC -c -o foo.o foo.c
$ gcc -Wall -fPIC -c -o bar.o bar.c
Link the object files to make a shared library:
$ gcc -shared -o libfoobar.so foo.o bar.o
Then:
$ readelf -s libfoobar.so | grep FILE
26: 0000000000000000 0 FILE LOCAL DEFAULT ABS crtstuff.c
35: 0000000000000000 0 FILE LOCAL DEFAULT ABS foo.c
37: 0000000000000000 0 FILE LOCAL DEFAULT ABS bar.c
39: 0000000000000000 0 FILE LOCAL DEFAULT ABS crtstuff.c
42: 0000000000000000 0 FILE LOCAL DEFAULT ABS
indicates that three source files have contributed debugging info to the
library, and we'd infer that the object files to which they were compiled
were likely to be:
crtstuff.o
foo.o
bar.o
Note that crtstuff.c is not one of the source files that we compiled. It
happens to contain program initialization and finalization code from the C runtime library, which has got into
our library from a C runtime object file that is linked by default.
This inference could be wrong about any of the files, since:
$ gcc -Wall -fPIC -c -o abc.o foo.c
$ gcc -Wall -fPIC -c -o xyz.o bar.c
$ gcc -shared -o libfoobar.so abc.o xyz.o
is also a perfectly possible way of compiling and linking the library.
If debugging information has been stripped from the library:
$ strip -g libfoobar.so
then we are out of luck:
$ readelf -s libfoobar.so | grep FILE
$
No more FILE symbols.

linker option to ignore unused dependencies

I would like to remove all unused symbols from my compiled C++ binary. I saw this, which gives an overview using gcc, which is the toolchain I'm using: How to remove unused C/C++ symbols with GCC and ld?
However, on my system, the linking option (-Wl,--gc-sections) is rejected:
$ gcc -fdata-sections -ffunction-sections a.c -o a.o -Wl,--gc-sections
ld: fatal: unrecognized option '--'
ld: fatal: use the -z help option for usage information
collect2: error: ld returned 1 exit status
I'm running on illumos, which is a (relatively) recent fork of Solaris, with GCC 4.7. Anybody know what the correct linker option to use here is?
Edit: searching the man pages more closely turned up "-zignore":
-z ignore | record
Ignores, or records, dynamic dependencies that are not
referenced as part of the link-edit. Ignores, or
records, unreferenced ELF sections from the relocatable
objects that are read as part of the link-edit. By
default, -z record is in effect.
If an ELF section is ignored, the section is eliminated
from the output file being generated. A section is
ignored when three conditions are true. The eliminated
section must contribute to an allocatable segment. The
eliminated section must provide no global symbols. No
other section from any object that contributes to the
link-edit, must reference an eliminated section.
However the following sequence still puts FUNCTION_SHOULD_BE_REMOVED in the ELF section .text.FUNCTION:
$ cat a.c
int main() {
return 0;
}
$ cat b.c
int FUNCTION_SHOULD_BE_REMOVED() {
return 0;
}
$ gcc -fdata-sections -ffunction-sections -c a.c -Wl,-zignore
$ gcc -fdata-sections -ffunction-sections -c b.c -Wl,-zignore
$ gcc -fdata-sections -ffunction-sections a.o b.o -Wl,-zignore
$ elfdump -s a.out # I removed a lot of output for brevity
Symbol Table Section: .dynsym
[2] 0x08050e72 0x0000000a FUNC GLOB D 1 .text.FUNCTION FUNCTION_SHOULD_BE_REMOVED
Symbol Table Section: .symtab
[71] 0x08050e72 0x0000000a FUNC GLOB D 0 .text.FUNCTION FUNCTION_SHOULD_BE_REMOVED
Because the man pages say "no global symbols", I tried making the function "static" and that had the same end result.
The ld '-z ignore' option is positional, it applies to those input objects which occur after it on the command line. The example you gave:
gcc a.o b.o -Wl,-zignore
Applies the option to no objects -- so nothing is done.
gcc -Wl,-zignore a.o b.o
Should work

Creating shared object from static library whose object files were linked with -fPIC

For a project we are trying to create a shared object file that exports a set of functions specified in libname.exports. Of course we know that the object files from which the .so file gets linked have to be created using -fPIC, so that has been taken care of. We then combined the object files into an archive named libname.a. This should now be the basis for the .so file to be created - or so was the idea.
We're passing libname.exports to --retain-symbols-file, so the expected behavior was that the linker would pull in any of the .a members relevant to those symbols.
However, the output of nm libname.so is empty. On the other hand grepping in nm libname.a shows that the relevant symbols named in libname.exports exist in the .a members.
Now I stumbled over --whole-archive and thus adjusted the command line from:
gcc -o libname.so -shared -Wl,-z,defs,--retain-symbols-file,libname.exports,-L. libname.a -lc
to:
gcc -o libname.so -shared -Wl,-z,defs,--retain-symbols-file,libname.exports,-L.,--whole-archive,libname.a,--no-whole-archive -lc
which appears to have the intended effect of including all the object files from the .a (although the size difference is strange). However, nm libname.so still gives me no output.
How can I use the archive file to create a shared object with only the symbols named in libname.exports visible?
Unfortunately How to create a shared object file from static library doesn't quite answer my question.
Note: before you ask. The idea behind using the .a file as input is because it makes it easy to use a pattern rule in GNUmakefile and because the .a file with -fPIC is needed regardless. There shouldn't be any difference between linking the individual object files versus the archive file.
You could use the -u SYMBOL option to force objects to be read in from an archive.
% cc -c -fPIC a.c
% nm a.o
00000000 T a
% ar rv liba.a a.o
ar: creating liba.a
a - a.o
% gcc -o liba.so -shared -u a liba.a
% nm liba.so | awk '$3 == "a" { print }'
0000042c T a
One thing to check would be the spellings of the symbols being specified with --retain-symbols-file. For example, symbol names in objects compiled from C++ code are likely to be mangled:
% g++ -c -fPIC a.c
% nm a.o | awk '$2 == "T" { print }'
00000000 T _Z1av

GCC 4.5 vs 4.4 linking with dependencies

I am observing a difference when trying to do the same operation on GCC 4.4 and GCC 4.5. Because the code I am doing this with is proprietary, I am unable to provide it, but I am observing a similar failure with this simple test case.
What I am basically trying to do is have one shared library (libb) depend on another shared library (liba). When loading libb, I assume that liba should be loaded as well - even though libb is not necessarily using the symbols in liba.
What I am observing is when I compile with GCC 4.4, I observe that the liba is loaded, but if I compile with GCC 4.5, libb is not loaded.
I have a small test case that consists of two files, a.c and b.c . The contents of the files:
//a.c
int a(){
return 0;
}
//b.c
int b(){
return 0;
}
//c.c
#include <stdio.h>
int a();
int b();
int main()
{
printf("%d\n", a()+b());
return 0;
}
//test.sh
$CC -o liba.so a.c -shared
$CC -o libb.so b.c -shared -L. -la -Wl,-rpath-link .
$CC c.c -L. -lb -Wl,-rpath-link .
LD_LIBRARY_PATH=. ./a.out
This is my output with different versions of GCC
$ CC=gcc-4.4 ./test.sh
1
$ CC=gcc-4.5 ./test.sh
/tmp/cceJhAqy.o: In function `main':
c.c:(.text+0xf): undefined reference to `a'
collect2: ld returned 1 exit status
./test.sh: line 4: ./a.out: No such file or directory
$ CC=gcc-4.6 ./test.sh
/tmp/ccoovR0x.o: In function `main':
c.c:(.text+0xf): undefined reference to `a'
collect2: ld returned 1 exit status
./test.sh: line 4: ./a.out: No such file or directory
$
Can anyone explain what is happening? Another extra bit of information is that ldd on libb.so does show liba.so on GCC 4.4 but not on GCC 4.5.
EDIT
I changed test.sh to the following:
$CC -shared -o liba.so a.c
$CC -L. -Wl,--no-as-needed -Wl,--copy-dt-needed-entries -la -shared -o libb.so b.c -Wl,-rpath-link .
$CC -L. c.c -lb -Wl,-rpath-link .
LD_LIBRARY_PATH=. ./a.out
This gave the following output with GCC 4.5:
/usr/bin/ld: /tmp/cc5IJ8Ks.o: undefined reference to symbol 'a'
/usr/bin/ld: note: 'a' is defined in DSO ./liba.so so try adding it to the linker command line
./liba.so: could not read symbols: Invalid operation
collect2: ld returned 1 exit status
./test.sh: line 4: ./a.out: No such file or directory
There seems to have been changes in how DT_NEEDED libraries are treated during linking by ld. Here's the relevant part of current man ld:
With --copy-dt-needed-entries dynamic libraries mentioned on the command
line will be recursively searched, following their DT_NEEDED tags to other libraries, in order to resolve symbols required by the output binary. With the
default setting however the searching of dynamic libraries that follow it will stop with the dynamic library itself. No DT_NEEDED links will be traversed
to resolve symbols.
(part of the --copy-dt-needed-entries section).
Some time between GCC 4.4 and GCC 4.5 (apparently, see some reference here - can't find anything really authoritative), the default was changed from the recursive search, to no recursive search (as you are seeing with the newer GCCs).
In any case, you can (and should) fix it by specifying liba in your final link step:
$CC c.c -L. -lb -la -Wl,-rpath-link .
You can check that this linker setting is indeed (at least part of) the issue by running with your newer compilers and this command line:
$CC c.c -L. -Wl,--copy-dt-needed-entries -lb -Wl,--no-copy-dt-needed-entries \
-Wl,-rpath-link .

is it possible to create an object file from other object files in gcc?

I was trying to do something like this in a makefile:
program.exe: ui.o main.o
gcc ......etc
ui.o: window1.o window2.o
gcc -c window1.o window2.o -o ui.o #this doesn't want to work
window1.o: window1.c window1.h window1_events.c window1_controls.c ...
gcc -c window1.c window1_events.c window1_controls.c... -o window1.o
window2.o: ...
gcc ...
main.o: ...
gcc ...
but when I compile like this, it gives the error "input file unused because linking not done," and then I get a bunch of unresolved externs, etc--problems which are resolved by changing
program.exe: ui.o main.o
gcc ...
to
program.exe: window1.o window2.o main.o
gcc ...
so is it possible to just link object files together, to avoid having mile-long lines in a makefile and break down the build process a little more?
Yes: to merge several object files into one, use ld -r or ld -Ur:
From "man ld" on Linux:
-r
--relocatable
Generate relocatable output---i.e., generate an output file that can
in turn serve as input to ld. This is often called partial linking.
As a side effect, in environments that support standard Unix magic
numbers, this option also sets the output file’s magic number to
"OMAGIC".
If this option is not specified, an absolute file is produced.
When linking C++ programs, this option will not resolve references to
constructors; to do that, use -Ur.
You could also do this with gcc:
gcc -Wl,-r foo.o bar.o -o foobar.o -nostdlib
Merging object files like this has some advantages over using an archive library: if merged files change very infrequently (compared to say main.c), your final executable links will be faster.
OTOH, with archived library, the linker will only use what it needs, so your executable may end up being smaller if e.g. window2.c ends up not being necessary.
I bunch of object files is a library. You can create a library with the ar
utility. The following example creates a library called mylib.a containing the files foo.o and bar.o
ar rvs mylib.a foo.o bar.o
You can then link with it by using it on the compiler command line:
gcc -o myexe main.c mylib.a
To create a library:
ar rvs somelib.a file1.o file2.o file3.o
To link it:
gcc -o program.exe file4.o somelib.a

Resources