I'm trying to understand the process by which Mac OS Catalina determines the locations of dynamic libraries and resolves name conflicts at runtime. The information on this website:
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryUsageGuidelines.html#//apple_ref/doc/uid/TP40001928-SW12
implies that there is a search hierarchy that starts with the contents of LD_LIBRARY_PATH and moves through a number of other environment variables; that one of these, DYLD_FALLBACK_LIBRARY_PATH, has default values if not explicitly set that include locations like /usr/lib .
Then I see that if I look at a library's dependencies with ldd -d, it shows the dependencies for each library with path information:
nasmac3465:libs ptenenba$ ldd -d libtess-mod.dylib
libtess-mod.dylib:
Contents of (__DATA,__data) section
000000000001cb80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000000000001cb90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000000000001cba0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
/Users/ptenenba/git/tess/spoc/code/dist/libs/libtess-mod.dylib (compatibility version 0.0.0, current version 0.0.0)
/opt/local/lib/libgcc/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.28.0)
/Users/ptenenba/git/tess/spoc/code/dist/libs/libtess-common.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
/opt/local/lib/libgcc/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
nasmac3465:libs ptenenba$
My question: why do I need to specify any library search paths using LD_LIBRARY_PATH or DYLD_LIBRARY_PATH when each library has both the name and the path of its dependencies?
Normally it is unnecessary and undesirable to use the environment variables with dyld the dynamic library loader. The paths should instead be set during the link phase, or modified later with install_name_tool.
Related
I've got a weird issue with compiling GCC that I can't seem to figure out.
What I'm trying to compile: gcc v12.2.0 (via make bootstrap)
What I'm compiling with: gcc v11.2.0
Configure command: ../configure --prefix=/usr --libdir=/usr/lib64 --mandir=/usr/man --infodir=/usr/info --enable-shared --enable-bootstrap --enable-languages=ada,c,c++,d,fortran,go,lto,objc,obj-c++ --enable-threads=posix --enable-checking=release --enable-objc-gc --with-system-zlib --enable-libstdcxx-dual-abi --with-default-libstdcxx-abi=new --disable-libstdcxx-pch --disable-libunwind-exceptions --enable-__cxa_atexit --disable-libssp --enable-gnu-unique-object --enable-plugin --enable-lto --disable-install-libiberty --disable-werror --with-gnu-ld --with-isl --verbose --with-arch-directory=amd64 --disable-gtktest --enable-clocale=gnu --disable-multilib --target=x86_64-slackware-linux --build=x86_64-slackware-linux --host=x86_64-slackware-linux
Build environment: Docker image (Slackware x86_64 v15.0)
Host environment: I've managed to reproduce this on two different hosts: one is running Ubuntu (x86_64, v20.04.5); the other is running Windows 10.
The issue: Sometimes (about half the time), the build fails with something similar to the following:
make[4]: Entering directory '/tmp/gcc-12.2.0/gcc.build.lnx/prev-gcc'
rm -f stamp-gnatlib2-rts stamp-tools
d/.deps/opover.Po:1: *** target pattern contains no '%'. Stop.
make[4]: Leaving directory '/tmp/gcc-12.2.0/gcc.build.lnx/prev-gcc'
make[3]: *** [Makefile:3279: stmp-fixinc] Error 2
make[3]: *** Waiting for unfinished jobs....
rm gcc.pod gfortran.pod gccgo.pod gdc.pod
make[3]: Leaving directory '/tmp/gcc-12.2.0/gcc.build.lnx/gcc'
make[2]: *** [Makefile:5005: all-stage2-gcc] Error 2
make[2]: Leaving directory '/tmp/gcc-12.2.0/gcc.build.lnx'
make[1]: *** [Makefile:30918: stage2-bubble] Error 2
make[1]: Leaving directory '/tmp/gcc-12.2.0/gcc.build.lnx'
make: *** [Makefile:31130: bootstrap] Error 2
The offending file will be different each time, although it's always under d/.deps/. Additionally, sometimes the error message is *** target pattern contains no '%'. Stop.; sometimes it's *** missing separator. Stop..
If I go and inspect the offending file, I can see that the filename has been corrupted (notice the errant f2 3a fd 7f after opover.o and before the :):
00000000 64 2f 6f 70 6f 76 65 72 2e 6f f2 3a fd 7f 3a 20 |d/opover.o.:..: |
00000010 2e 2e 2f 2e 2e 2f 67 63 63 2f 64 2f 64 6d 64 2f |../../gcc/d/dmd/|
00000020 6f 70 6f 76 65 72 2e 64 20 5c 0a 20 2f 74 6d 70 |opover.d \. /tmp|
00000030 2f 67 63 63 2d 31 32 2e 32 2e 30 2f 67 63 63 2f |/gcc-12.2.0/gcc/|
00000040 64 2f 64 6d 64 2f 63 6c 6f 6e 65 2e 64 20 2f 74 |d/dmd/clone.d /t|
I don't see these extraneous characters in the output being spit out by make -- so I don't know where they're coming from.
I'm almost ready to call this a bug in GCC. But before I do, what else would cause this to happen?
I have a program that generates self-modifying code (see https://tigress.wtf/selfModify.html in case you're interested). It runs on x86 Darwin and Linux. On Darwin, I compile with
gcc -g -segprot __TEXT rwx rwx self_modifying.c -o self_modifying.exe
Recently, this seems not to work, I get
dyld: malformed mach-o image: __TEXT segment maps start of file but is writable
when I run the program.
I'm running clang version 6.0.1 on MacOS 10.15.3. Any help would be appreciated.
#AlexDenisov is pretty close, but the restriction does not apply only to executables running on Catalina with min macOS 10.15.0 and higher.
There are 2 formats of Mach-O load command that indicate the min MacOS the executable itself can use:
- LC_BUILD_VERSION(the new one introduced around 10.14 if I recall correctly)
- LC_VERSION_MIN_MACOSX (the legacy one)
Even with fallbacking to older MacOS version using LC_VERSION_MIN_MACOSX:
gcc -segprot __TEXT rwx rwx -mmacosx-version-min=10.6 self_modifying.c
we run into the same problem.
To bypass the check one solution I found to work is to get rid of min macos version altogether. I'm not aware though of any gcc or ld flag that can achieve this. Fortunately we can do the processing after linking the executable with the help of Jonathan Levin's jtool2
So the chain of commands become:
gcc -segprot __TEXT rwx rwx self_modifying.c
jtool2 -l a.out
LC 00: LC_SEGMENT_64 Mem: 0x000000000-0x100000000 __PAGEZERO
LC 01: LC_SEGMENT_64 Mem: 0x100000000-0x100001000 __TEXT
Mem: 0x100000f60-0x100000f83 __TEXT.__text (Normal)
Mem: 0x100000f84-0x100000f8a __TEXT.__stubs (Symbol Stubs)
Mem: 0x100000f8c-0x100000fa6 __TEXT.__stub_helper (Normal)
Mem: 0x100000fa6-0x100000fb2 __TEXT.__cstring (C-String Literals)
Mem: 0x100000fb4-0x100000ffc __TEXT.__unwind_info
LC 02: LC_SEGMENT_64 Mem: 0x100001000-0x100002000 __DATA_CONST
Mem: 0x100001000-0x100001008 __DATA_CONST.__got (Non-Lazy Symbol Ptrs)
LC 03: LC_SEGMENT_64 Mem: 0x100002000-0x100003000 __DATA
Mem: 0x100002000-0x100002008 __DATA.__la_symbol_ptr (Lazy Symbol Ptrs)
Mem: 0x100002008-0x100002010 __DATA.__data
LC 04: LC_SEGMENT_64 Mem: 0x100003000-0x100004000 __LINKEDIT
LC 05: LC_DYLD_INFO
Rebase info: 8 bytes at offset 12288 (0x3000-0x3008)
Bind info: 24 bytes at offset 12296 (0x3008-0x3020)
No Weak info
Lazy info: 16 bytes at offset 12320 (0x3020-0x3030)
Export info: 48 bytes at offset 12336 (0x3030-0x3060)
LC 06: LC_SYMTAB
LC 07: LC_DYSYMTAB
1 local symbols at index 0
2 external symbols at index 1
2 undefined symbols at index 3
No TOC
No modtab
3 Indirect symbols at offset 0x30b8
LC 08: LC_LOAD_DYLINKER /usr/lib/dyld
LC 09: LC_UUID UUID: 6AE91487-DB61-3FA8-8DBE-686FEC1DA8FC
LC 10: LC_BUILD_VERSION Build Version: Platform: MacOS 10.15.0 SDK: 10
LC 11: LC_SOURCE_VERSION Source Version: 0.0.0.0.0
LC 12: LC_MAIN Entry Point: 0xf60 (Mem: 0x100000f60)
LC 13: LC_LOAD_DYLIB /usr/lib/libSystem.B.dylib
LC 14: LC_FUNCTION_STARTS Offset: 12384, Size: 8 (0x3060-0x3068)
LC 15: LC_DATA_IN_CODE Offset: 12392, Size: 0 (0x3068-0x3068)
jtool2 -rc 10 --inplace a.out
Now your a.out should launch correctly :-)
The problem you are observing is the restriction of macOS Catalina and not related to your compiler.
Looking at the dyld source code (can be found here https://opensource.apple.com/release/macos-1015.html) the error message is coming from this code:
if ( (segCmd->initprot & VM_PROT_WRITE) == VM_PROT_WRITE ) {
if ( context.strictMachORequired )
dyld::throwf("malformed mach-o image: %s segment maps start of file but is writable", segCmd->segname);
}
The exception is thrown only when strictMachORequired, which is always true on macOS 10.15 or higher, based on the other snippet from the dyld sources:
#if __MAC_OS_X_VERSION_MIN_REQUIRED
gLinkContext.strictMachORequired = false;
// <rdar://problem/22805519> be less strict about old macOS mach-o binaries
((dyld3::MachOFile*)mainExecutableMH)->forEachSupportedPlatform(^(dyld3::Platform platform, uint32_t minOS, uint32_t sdk) {
if ( (platform == dyld3::Platform::macOS) && (sdk >= DYLD_PACKED_VERSION(10,15,0)) ) {
gLinkContext.strictMachORequired = true;
}
});
if ( gLinkContext.iOSonMac )
gLinkContext.strictMachORequired = true;
#else
// simulators, iOS, tvOS, watchOS, are always strict
gLinkContext.strictMachORequired = true;
#endif
I'm trying to use rpath to link in some libraries for an app. Some of the rpaths work others don't.
I have two dynamic libraries I'm linking to libgunrock and libtd both in different directories.
My link like looks like the following:
gcc -L/Users/aterrel/workspace/apps/gunrock/build/lib/ -Wl,-rpath -Wl,/Users/aterrel/workspace/apps/gunrock/build/lib/ -lgunrock -L/Users/aterrel/workspace/apps/xdata/xlang/thunderdome -Wl,-rpath -Wl,/Users/aterrel/workspace/apps/xdata/xlang/thunderdome -ltd app-c-gunrock-bokeh.o -o app
The load paths are correct. The relavant parts of the otool -l app:
Load command 16
cmd LC_RPATH
cmdsize 64
path /Users/aterrel/workspace/apps/gunrock/build/lib/ (offset 12)
Load command 17
cmd LC_RPATH
cmdsize 72
path /Users/aterrel/workspace/apps/xdata/xlang/thunderdome (offset 12)
Load command 18
Yet I still get the error:
dyld: Library not loaded: libtd.dylib
Referenced from: /Users/aterrel/workspace/apps/xdata/xlang/webgraph/apps/c-gunrock-bokeh/./app
Reason: image not found
Trace/BPT trap: 5
If I run with DYLD_LIBRARY_PATH=/Users/aterrel/workspace/apps/xdata/xlang/thunderdome app works fine.
I don't understand why libgunrock.dylib is loaded fine but libtd.dylib is not.
When building rpm's for centos I continue to get this error
ERROR 0020: file '/usr/local/probe/lib64/libasan.so.0.0.0' contains an rpath referencing '..'
According to the docs I should be able to ignore this by calling:
QA_RPATHS=$[ 0x0020 ] rpmbuild -v -bb --target=x86_64 ~/rpmbuild/SPECS/$PACKAGE.spec
But it still gives me the same error.
the QA_RPATHS did solve the problem but our spec file was exporting a different QA_RPATH in the script. Once I removed that it was resolved.
I was following OpenGL tutorials from this site - www.arcsynthesis.org/gltut/ and in the Building Tutorials section, i downloaded the 0.3.8 version of the tutorials. I extracted its contents in a folder in the desktop... I downloaded premake4, then I placed that in the folder where i have extracted the contents. Then I opened up cmd and I cd to the directory where I extracted the contents. Since, I'm using CodeBlocks, i typed premake4 gmake and then it started building stuff... I copied the contents of the framework folder to its respective directory. Then I went to Tut 01 Hello Triangle directory via command prompt and executed the premake4 command. After that, I opened up tut1.cpp with CodeBlocks and after compiling i get a number of these errors:
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleCreateShader'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleShaderSource'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleCompileShader'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetShaderiv'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetShaderiv'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetShaderInfoLog'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleCreateProgram'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleAttachShader'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleLinkProgram'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetProgramiv'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetProgramiv'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGetProgramInfoLog'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleDetachShader'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleDeleteShader'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGenBuffers'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleBindBuffer'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleBufferData'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleBindBuffer'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleGenVertexArrays'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleBindVertexArray'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleClearColor'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleClear'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleUseProgram'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleBindBuffer'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleEnableVertexAttribArray'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleVertexAttribPointer'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleDrawArrays'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleDisableVertexAttribArray'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleUseProgram'
C:\Users\PB9X\Desktop\Tutorial 0.3.8\Tut 01 Hello Triangle\tut1.o:tut1.cpp|| undefined reference to `__gleViewport'
||=== Build finished: 30 errors, 0 warnings (0 minutes, 1 seconds) ===|
Where the hell did I go wrong? I already have GLEW, FreeGLUT, glimg, glload, glm, glmesh and gutil in its respective directories...
Since, I'm using CodeBlocks, i typed premake4 gmake
Codeblocks, as stated on the Premake4 site, has it's own command: premake4 codeblocks. If you're using Codeblocks, you're supposed to use the Codeblocks build system, not try to juryrig the gmake makefile into Codeblocks.