Undefined symbols: for -fvisibility=hidden - xcode

I am having a linker problem that I can't fix (using MacOS and xcode).
First off, when I compile with -fvisibility=hidden all is fine. If I omit that flag, I get
"XYZ::IPlugView::iid", referenced from:
__ZN9XYZ9IPlugView3iidE$non_lazy_ptr in pluginview.o
(maybe you meant: __ZN9XYZ9IPlugView3iidE$non_lazy_ptr)
I don't know if this is related, but before that are a couple warnings like
ld: warning: non-virtual thunk to XYZ...::release()has different visibility (hidden) in xyz/foo.o and (default) in xyz/bar.o
Any Ideas would be greatly appreciated....
Thanks!

The warnings might in fact be related. What these warnings are trying to tell you is that there is a symbol named XYZ...::release() and this symbol is defined twice, once in the file xyz/foo.o (probably compiled from xyz/foo.cc) and one in the file xyz/bar.o (probably compiled from xyz/bar.cc), however, the symbol is not twice defined with the same visibility, one symbol is default (visible outside the current binary/library you are compiling) and one is hidden (only visible within the current binary/library you are compiling). The same symbol cannot have two different visibilities, of course.
By using -fvisibility=hidden you tell the compiler: Whenever there is no code annotation that would otherwise define the visibility of a symbol, make this symbol hidden. This probably solves your problem as the duplicate symbols that used to be once hidden and once default are now twice hidden, because the definition that used to be default was probably implicitly made default and now its implicitly made hidden, which means the symbol is twice hidden and this resolves the conflict, since the two symbols are probably treated like a single one by the linker.
Within your source code, there are two ways to set the visibility of a symbol, either by attribute or by pragma. To make a symbol hidden, you can either append __attribute__ ((visibility ("hidden"))) in front of its definition or you can place a pragma #pragma GCC visibility push(visibility) somewhere in your source file, causing all symbol definitions below it to be set hidden until you either use the pragma #pragma GCC visibility pop to return to the previous visibility state (or back to the default one if there wasn't a previous one), use another pragma to change the visibility to something else or to the end of the file, whatever comes first.
I think in your case the symbol in question is once defined in foo.cc with an explicit visibility (which is hidden) and once in bar.cc with no explicit visibility, so it's either default or hidden, depending on your -fvisibility flag.

Related

How can I keep a symbol during gcc strip?

Using arm-linux-gnueabi-gcc, I'm working with a shared object that has an undefined reference to "main". I'm using dlopen() to open this library. However, when my program is compiled with "-Wl,-s", the symbol main is removed, which causes dlopen() to fail with an undefined symbol "main".
In gcc, is there any way I can perform a strip, but keep the symbol main?
Edit: even better, can I put just the symbol "main" in the dynamic section (similar to -rdynamic)?
The shared object with an undefined reference to main appears to use the dynamic symbol table to search for the reference to "main". So, there are two options here:
Compile with "-Wl,-rdynamic". This will put all of the symbols into the dynamic symbol table. This works, but it results in a lot of extra symbols being put in there, too.
Compile with "-Wl,--dynamic-list=[path to my list file]". Then list "main" in the dynamic file to retain only main in the dynamic symbols table. This has the drawback that all dynamic references must now be managed, but for simple programs, this is adequate.
Example of (2):
Invocation:
arm-linux-gnueabi-gcc -fPIC mycode.c -Wl,-s,--dynamic-list=./dynamic_symbols.txt
dynamic_symbols.txt:
{ main; };

Using Linker Symbol from C++ code as a fixed constant (NOT relocated) in a shared library (DLL)

Sorry if the title is not very clear. I am using MinGW with GCC 6.3.0 to build a x86 32-bit DLL on Windows (so far). I'll spare you the details why I need hacky offsets amongst its sections accessible from code, so please do not ask if it's useful or not (because I don't want to bother explaining that).
So, if I can get the following testcase to work, I'm good. Here's my problem:
In a C++ file, I want to access a linker symbol as an absolute numeric value, not relocated, directly. Remember that I am building a 32-bit DLL which requires a .reloc section for relocations, but in this case I do NOT want relocation, in fact a relocation would screw it up completely.
Here's an example: retrieve the offset of say __imp__MessageBoxW#16 relative to __IAT_start__, in case you don't know what they are, __imp__MessageBoxW#16 is the relocated pointer to the actual function at runtime, and __IAT_start__ is a linker symbol in the default script file. Here's where it is defined:
.idata BLOCK(__section_alignment__) :
{
/* This cannot currently be handled with grouped sections.
See pe.em:sort_sections. */
KEEP (SORT(*)(.idata$2))
KEEP (SORT(*)(.idata$3))
/* These zeroes mark the end of the import list. */
LONG (0); LONG (0); LONG (0); LONG (0); LONG (0);
KEEP (SORT(*)(.idata$4))
__IAT_start__ = .;
KEEP (SORT(*)(.idata$5))
__IAT_end__ = .;
KEEP (SORT(*)(.idata$6))
KEEP (SORT(*)(.idata$7))
}
So far, no problem. Because GAS doesn't allow me to "subtract" two externally defined symbols (both symbols are defined in the linker), I have to define the symbol in the linker script, so at the end of the linker script I have this:
test_symbol = ABSOLUTE("__imp__MessageBoxW#16" - __IAT_start__);
Then in C++ I use this little inline asm to retrieve this relative difference which is supposed to be a fixed value once linked:
asm("movl $test_symbol, %0":"=r"(var));
Now var should contain that fixed number right? Wrong!
Because test_symbol is an "undefined" symbol as far as the assembler is concerned, it makes it relocated. Or I don't know why, but I tried so many things to force it to be an "absolute constant value symbol" instead of a "relocated symbol" to no avail. Even editing the linker script with many things like LD_FEATURE("SANE_EXPR") and others, doesn't work at all.
Its value is correct only if the DLL does not get relocated.
You see, either GNU LD or the assembler adds an entry in the .reloc section for that movl instruction, which is WRONG!
Is there a way to force it to treat an external/undefined symbol as a fixed CONSTANT and apply no relocation to it whatsoever? Basically, omit it from the .reloc section.
I am going crazy with this, please tell me there's something easy I overlooked, I searched for hours!
In other words, is there a way to use a Linker Symbol from within inline asm/C++ without having it relocated whatsoever? No entry to the .reloc section or anything, basically same as a constant like $1234. So if a DLL gets loaded into another base address, that constant would be the same everytime.
UPDATE: I forgot about this question but decided to bring an update, since it seems it's likely not possible as nobody even commented. For anyone else in the same boat as me, I presume this is a limitation of the COFF object format itself. In other words, external symbols are implicitly relocated, and it doesn't seem there's a way against this.
I didn't "fix" it the way I wanted, I did it in a very hacky way though. If anyone is interested, here's my ugly "hack":
First I put a special "custom" instruction in the inline assembly where I reference this external symbol from C++. This "custom" instruction holds a placeholder instruction that grabs the symbol (normal x86 asm instruction with a dummy constant, e.g. 1234) and a way to identify it. Then let GCC generate the assembly files (.S files), then I parse the assembly with a simple script and when I find that "custom" instruction I insert a label for the linker (make it .global) and at the same time add a directive to a custom "on-the-fly" generated linker script that gets included from my main linker script at the end.
This places data in a temporary section in the resulting DLL with absolute offsets to the custom instruction that I need, but without relocation.
Next, I parse the binary DLL itself, in particular that temporary section I added with all this hack. I take the offsets from there, convert them to file offsets, and modify the DLL's .text section directly where those offsets point (remember those placeholder instructions? it is replacing their immediate constants 1234 with the respective value from the linker's non-relocated constant). Then I strip the temporary section from the DLL, and it's done. Of course, all of this is done automatically by a helper program and script
It's an insane hack, but it works and it's fully automatic now that I got it going. If my assumption is correct that COFF doesn't support non-relocated external symbols, then it's really the only way to use linker constants from C++ without them being relocated, which would be a disaster.

nm symbol output t vs T in a shared so library

I have added a new function (fuse_lowlevel_notify_inval_directory) in user space fuse library.
The compilation and creation of libfuse.so is finished without error.
But when my application tries to use this new function, the linker is throwing error: undefined reference to `fuse_lowlevel_notify_inval_directory'
collect2: ld returned 1 exit status
When I checked with nm
nm ../libfuse.so | grep inval
00000000000154ed T fuse_invalidate
**000000000001e142 t fuse_lowlevel_notify_inval_directory**
000000000001e26c T fuse_lowlevel_notify_inval_entry
000000000001e1cb T fuse_lowlevel_notify_inval_inode
T/t means the symbol is present in text section. if uppercase, the symbol is global (external). I suspect this is the issue. The new added function is showing lowercase t while other older functions are having upper case T. Any idea about what I might be doing wrong?
Any idea about what I might be doing wrong?
The t function is indeed local to the library. This could happen due to a number or reasons. The most likely ones are:
You declared the function static, or
You compiled the library with -fvisibility=hidden and did not have __attribute__((visibility("default"))) on the function, or
You linked this library with a linker version script (i.e. with --version-script=libfoo.version flag) that hides all functions, except those which are explicitly exported, and you didn't add your function to that list.
See this example of using --version-script to limit symbol visibility.

how can I verify that dead code was stripped from the binary?

My c/obj-c code (an iOS app built with clang) has some functions excluded by #ifdefs. I want to make sure that code that gets called from those functions, but not from others (dead code) gets stripped out (eliminated) at link time.
I tried:
Adding a local literal char[] in a function that should be eliminated; the string is still visible when running strings on the executable.
Adding a function that should be eliminated; the function name is still visible when running strings.
Before you ask, I'm building for release, and all strip settings (including dead-code stripping, obviously) are enabled.
The question is not really xcode/apple/iOS specific; I assume the answer should be pretty much the same on any POSIX development platform.
(EDIT)
In binutils, ld has the --gc-sections option which does what you want for sections on object level. You have several options:
use gcc's flags -ffunction-sections and -fdata-sections to isolate each symbol into its own section, then use --gc-sections;
put all candidates for removal into a separate file and the linker will be able to strip the whole section;
disassemble the resulting binary, remove dead code, assemble again;
use strip with appropriate -N options to discard the offending symbols from the
symbol table - this will leave the code and data there, but it won't show up in the symbol table.

OSX 10.5 Leopard Symbol Mangling with $non_lazy_ptr

Why does Leopard mangle some symbols with $non_lazy_ptr? More importantly what is the best method to fix undefined symbol errors because a symbol has been mangled with $non_lazy_ptr?
From: Developer Connection - Indirect Addressing
Indirect addressing is the name of the code generation technique that allows symbols defined in one file to be referenced from another file, without requiring the referencing file to have explicit knowledge of the layout of the file that defines the symbol. Therefore, the defining file can be modified independently of the referencing file. Indirect addressing minimizes the number of locations that must be modified by the dynamic linker, which facilitates code sharing and improves performance.
When a file uses data that is defined in another file, it creates symbol references. A symbol reference identifies the file from which a symbol is imported and the referenced symbol. There are two types of symbol references: nonlazy and lazy.
Nonlazy symbol references are resolved (bound to their definitions) by the dynamic linker when a module is loaded.
A nonlazy symbol reference is essentially a symbol pointer—a pointer-sized piece of data. The compiler generates nonlazy symbol references for data symbols or function addresses.
Lazy symbol references are resolved by the dynamic linker the first time they are used (not at load time). Subsequent calls to the referenced symbol jump directly to the symbol’s definition.
Lazy symbol references are made up of a symbol pointer and a symbol stub, a small amount of code that directly dereferences and jumps through the symbol pointer. The compiler generates lazy symbol references when it encounters a call to a function defined in another file.
In human-speak: the compiler generates stubs with $non_lazy_ptr appended to them to speed up linking. You're probably seeing that function Foo referenced from _Foo$non_lazy_ptr is undefined, or something like that - these are not the same thing. Make sure that the symbol is actually declared and exported in the object files/libraries you're linking your app to. At least that was my problem, I also thought it's a weird linker thing until I found that my problem was elsewhere - there are several other possible causes found on Google.
ranlib -c libwhatever.a
is a solid fix for the issue. I had the same problem when building the PJSIP library for iOS. This library sort-of uses an autoconf based make system, but needs a little tweaking to various files to make everything alright for iOS. In the process of doing that I managed to remove the ranlib line in the rule for libraries and then started getting an error in the link of my project about _PJ_NO_MEMORY_EXCEPTION referenced from _PJ_NO_MEMORY_EXCEPTION$non_lazy_ptr being undefined.
Adding the ranlib line back to the library file solved it. Now my full entry for LIBS in rules.mak is
$(LIB): $(OBJDIRS) $(OBJS) $($(APP)_EXTRA_DEP)
if test ! -d $(LIBDIR); then $(subst ##,$(subst /,$(HOST_PSEP),$(LIBDIR)),$(HOST_MKDIR)); fi
$(LIBTOOL) -o $(LIB) $(OBJS)
$(RANLIB) -c $(LIB)
Hope this helps others as well trying to use general UNIX configured external libraries with iPhone or iOS.
If someone else stumbles the same problem I had:
Had a extern NSString* const someString; in the header file, but forgot to put it the implementation file. as NSString* const someString=#"someString";
This solved it.
ranlib -c on your library file fixes the problem

Resources