Compile a binary file for linking OSX - macos

I'm trying to compile a binary file into a MACH_O object file so that it can be linked it into a dylib. The dylib is written in c/c++.
On linux the following command is used:
ld -r -b binary -o foo.o foo.bin
I have tried various option on OSX but to no avail:
ld -r foo.bin -o foo.o
gives:
ld: warning: -arch not specified
ld: warning: ignoring file foo.bin, file was built for unsupported file format which is not the architecture being linked (x86_64)
An empty .o file is created
ld -arch x86_64 -r foo.bin -o foo.o
ld: warning: ignoring file foo.bin, file was built for unsupported file format which is not the architecture being linked (x86_64)
Again and empty .o file is created. Checking the files with nm gives:
nm foo.o
nm: no name list
The binary file is actually, firmware that will be downloaded to an external device.
Thanks for looking

Here's the closest translation to the Linux linker command to perform binary embedding with the OSX linker:
touch stub.c
gcc -o stub.o -c stub.c
ld -r -o foo.o -sectcreate binary foo_bin foo.bin stub.o
foo.bin will be stored in segment binary, section foo_bin (both names are arbitrary but chosen to mimic GNU ld for ELF on Linux) of the foo.o object.
stub is necessary because ld refuses to create just a custom segment/section. You don't need it if you link directly with a real code object.
To get data back from the section, use getsectbyname (struct is defined in mach-o/loader.h):
#include <mach-o/getsect.h>
const struct section_64 *sect = getsectbyname("binary", "foo_bin");
char *buffer = calloc(1, sect->size+1);
memcpy(buffer, sect->addr, sect->size); // whatever
or getsectdata:
#include <mach-o/getsect.h>
size_t size;
char *data = getsectdata("binary", "foo_bin", &size);
char *buffer = calloc(1, size+1);
memcpy(buffer, data, size); // whatever
(I used it to store text data, hence the stringification via calloc zeroing of size+1 plus blob copying)
Warning: Since 10.7, ASLR got stronger and messes badly with getsect* functions, resulting in segfaults. set disable-aslr off in GDB before running to reproduce EXC_BAD_ACCESS (SIGSEGV) in debug conditions. People had to jump through inordinate hoops to find the real address and get this working again.
A simple workaround is to get the offset and size, open the binary and read the data straight from disk. Here is a working example:
// main.c, build with gcc -o main main.c foo.o
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <mach-o/getsect.h>
int main() {
// finding the filename of the running binary is left as an exercise to the reader
char *filename = "main";
const struct section_64 *sect = getsectbyname("binary", "foo_bin");
if (sect == NULL) {
exit(1);
}
char *buffer = calloc(1, sect->size+1);
int fd = open(filename, O_RDONLY);
if (fd < 0) {
exit(1);
}
lseek(fd, sect->offset, SEEK_SET);
if (read(fd, buffer, sect->size) != sect->size) {
close(fd);
exit(1);
}
printf("%s", buffer);
}

Related

GCC (minGW - Windows 10) report that there is no reference to symbol defined in .o file when I'm trying to embed the binary into executable

creating .o file
ld -r -b binary -o baked.o baked.txt
λ nm baked.o
0000000f D _binary_baked_txt_end
0000000f A _binary_baked_txt_size
00000000 D _binary_baked_txt_start
Code:
#include <stdlib.h>
#include <stdio.h>
extern char *_binary_baked_txt_end;
extern char _binary_baked_txt_size;
extern char *_binary_baked_txt_start;
int main(void) {
printf("the baked is %s", _binary_baked_txt_start);
return 0;
}
compiling:
λ gcc -o main baked.o main.c
C:\Users\566\AppData\Local\Temp\cc4um6Mn.o:main.c:(.text+0x12): undefined reference to `_binary_baked_txt_start'
collect2.exe: error: ld returned 1 exit status
The question is, what do I do wrong? I see the symbols into .o file, I compile it with main.c file, what could go wrong?
I should use just:
binary_baked_txt_end;
binary_baked_txt_size;
binary_baked_txt_start;
because of the COFF format on Windows machines.
and the right way to do will be:
extern char binary_baked_txt_end[];
extern char binary_baked_txt_size;
extern char binary_baked_txt_start[];
because in that way it's clear that binary_baked_txt_end is a adress that points to a char (in my case) and it will be safer, nobody won't be able to assign new value to it.

Undefined function from static library

I am trying to build a static library using MinGW.
Everything was going fine until I tried to use the library and got an error saying that add_numbers is an undefined function.
Many other people have had this problem and sorted it out by moving their library to be linked after the source files were included, but that was how I had written my batch file anyway, so that was not of much help.
Here are my sources.
mylib.h
#ifndef MYLIB_H
#define MYLIB_H
int add_numbers(int a, int b, int c);
#endif
mylib.c
#include "mylib.h"
int add_numbers(int a, int b, int c)
{
return a+b+c;
}
I'm building my .a file with the following commands
gcc --std=c89 -c mylib.c -o mylib.o
ar rcs libmylib.a mylib.o
I've also tried with out specifying the standard.
There are no errors or warnings when running this command.
Next, my test program looks like this.
#include <stdio.h>
#include "mylib.h"
int main()
{
printf("The sum of 1, 2, and 3 is %d", add_numbers(1, 2, 3));
getchar();
return 0;
}
And lastly, we build the test with this command.
gcc mylibtest.c -L -lmylib -o test.exe
I've tried moving around those commands into many many different sequences, but always receiving the following error:
C:\Users\Aaron\AppData\Local\Temp\cc0ERpBi.o:mylibtest.c:(.text+0x26): undefined
reference to `add_numbers'
collect2.exe: error: ld returned 1 exit status
E:\my_first_static_library>
Any help would be very appreciated, I've read every tutorial I could find on the art of writing static libraries, as well as a good ten stackoverflow questions.
You are missing a dot after -L:
gcc mylibtest.c -L . -lmylib -o test.exe

Where is the __eprintf symbol defined in OS X 10.11?

I am compiling some asserts with -arch i386 -isysroot /Applications/Xcode-7.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.4. Therefore, the assert code doesn't use __assert_rtn but __eprintf instead.
Relevant snippet from assert.h:
#if defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__) && ((__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__-0) < 1070)
#define __assert(e, file, line) \
__eprintf ("%s:%u: failed assertion `%s'\n", file, line, e)
#else
/* 8462256: modified __assert_rtn() replaces deprecated __eprintf() */
#define __assert(e, file, line) \
__assert_rtn ((const char *)-1L, file, line, e)
#endif
So far so good, except when linking time arrives, it doesn't find __eprintf. In which library is this defined ?
A way to reproduce getting __eprintf into the assert:
cat <<EOF >/tmp/x.c
#include <assert.h>
#ifdef __clang__
# if defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__) && ((__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__-0) <= 1040)
# if ! __DARWIN_UNIX03
# warning "compiling for 10.4 (not __DARWIN_UNIX03), with __eprintf"
# endif
# endif
#endif
int xxx( int a)
{
assert( a);
return( a);
}
EOF
clang -E -arch i386 -mmacosx-version-min=10.4 -isysroot /Applications/Xcode-7.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -o /tmp/x.txt /tmp/x.c
Using above, make a dylib and observe the problem:
clang -c -arch i386 -mmacosx-version-min=10.4 -isysroot /Applications/Xcode-7.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -O0 -o /tmp/x.o /tmp/x.c
ld -arch i386 -macosx_version_min 10.4.0 -syslibroot /Applications/Xcode-7.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -t -o /tmp/x.dylib -ldylib1.o /tmp/x.o -lSystem -lgcc_s.10.4
These calls are destilled down from what xcodebuild produces.
Thanks for the helpful responses.
The answer turns out to be /Applications/Xcode-7.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/7.0.2/lib/darwin/libclang_rt.10.4.a in my particular case.
This is a library that gets added by clang itself behind the scenes (DarwinClang::AddLinkRuntimeLibArgs). Note that this library is only for 10.4 code and seems to be tied to the compiler version.
I don't know by which magic (don't see it in the invocation) the linker decides which compiler version it needs to link for.
The easiest solution is to define ones own __eprintf, instead of messing around with the toolchain:
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#if defined(__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__) && ((__ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__-0) <= 1040)
# if ! __DARWIN_UNIX03
__attribute__((visibility("hidden")))
void __eprintf( const char* format, const char* file,
unsigned line, const char *expr)
{
fprintf( stderr, format, file, line, expr);
abort();
}
# endif
#endif
int main( int argc, char *argv[])
{
assert( argc == 2);
return( 0);
}

Mac OSX ld report 32-bit RIP relative reference out of range error for Absolute symbol

I'm trying to combine objcopy with clang toolchain.
Because objcopy of binutils 2.25 generates broken Mach-O object file, I edit generated object file using my shell script.
$ objcopy-comp.sh -I binary -O mach-o-x86-64 test test.o
$ nm test.o
000000000000000b D _binary_test_end
000000000000000b A _binary_test_size
0000000000000000 D _binary_test_start
However, link against a C code fails with this error message.
$ clang main.c test.o
ld: 32-bit RIP relative reference out of range (-4294971146 max is +/-4GB):
from _main (0x100000EA0) to _binary_test_size (0x0000000B)
in '_main' from main.o for architecture x86_64
(Newlines are inserted for readbility)
Here is main.c.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
extern const unsigned char binary_test_start[];
extern const unsigned char binary_test_end[];
extern const unsigned char binary_test_size[];
int main(int argc, char *argv[])
{
size_t len = binary_test_end - binary_test_start;
char *data = calloc(len + 1, sizeof(char));
memcpy(data, binary_test_start, len);
data[len] = 0;
printf("%s %ld %d\n", data, len, (int)binary_test_size);
return 0;
}
According to nlist document,
N_ABS (0x2)—The symbol is absolute. The linker does not change the value of an absolute symbol.
but the error message suggests that linker does try to change the value.
How to protect Absolute value from linker?
The Mach-OABI utilizes relative addressing with x86_64. The compiler interprets the address you've used as 32-bit which is out range, nor will it be absolute. Try compiling your code as i386 only and you might have a better chance of success.
Specifically how you're changing the symbol types is unknown since you haven't shown the commands you've used with objcopy.

Linking libusb in Mac OS X

I have this very simple piece of code that I'm trying to compile. I'm fairly new to GCC from the command line, so please forgive me. I've tried a quite few different things with GCC, but I'm still unable to get it to compile. I do have libusb installed. How can I get this piece of code to compile?
Libusb:
anything:usb mymac$ brew list libusb
/usr/local/Cellar/libusb/1.0.9/include/libusb-1.0/libusb.h
/usr/local/Cellar/libusb/1.0.9/lib/libusb-1.0.0.dylib
/usr/local/Cellar/libusb/1.0.9/lib/pkgconfig/libusb-1.0.pc
/usr/local/Cellar/libusb/1.0.9/lib/ (2 other files)
anything:usb mymac$
GCC attempts (all failed):
gcc -o xout usbtest.c
gcc -o xout usbtest.c -lusb-1.0
gcc -L/usr/local/Cellar/libusb/1.0.9/lib -o xout usbtest.c -lusb-1.0
Error for all attempts:
usbtest.c:3:10: fatal error: 'libusb.h' file not found
#include <libusb.h>
Code:
#include <stdio.h>
#include <stdlib.h>
#include <libusb.h>
int main(int argc, const char * argv[])
{
libusb_device **devs;
libusb_context *context = NULL;
size_t list;
//size_t i;
int ret;
ret = libusb_init(&context);
if(ret < 0)
{
perror("libusb_init");
exit(1);
}
list = libusb_get_device_list(context, &devs);
printf("There are %zd devices found\n", list);
return 0;
}
So I had a similar issue, for some reason gcc doesnt include /usr/local/lib in its default search path on OS X. The quick fix is to add:
-lusb-1.0
to the gcc commands and it should compile.
You are not telling gcc where to look for the header files. This is done by the -I option on the gcc command line for compiling.
e.g.
gcc -I /usr/local/include -o xout usbtest.c
I think Homebrew does provide a symbolic link frominside the Cellar to /usr/local

Resources