Basic UNIX configure/make/sudo make install question - macos

I have learned to compile my own source over the years but always left some of the process to "mystery". I'm running into a bitch of a time getting ImageMagick and its myriad dependencies to work correctly on a PowerPC Mac OS X Server.
In setting flags for ./compile where does one typically uncover the various options (flags) that can be set and the descriptions of what each does?!?
For example, in the case of libwmf I'm using:
./configure --without-expat --with-xml --with-png=/usr/X11
...but not really knowing what each flag does. Now I'm having a png compatibility problem and want to find out why --with-png=/usr/X11 flag is being specified in install script I'm following...but can't find any documentation, etc. Any help?
Even a general response of how the UNIX Guru approaches this problem would be helpful.

./configure --help will give you the list of options that a configure script supports. To find out details about each option or the arguments you have to supply in particular cases, you will have to read the installation documentation of the particular package, if it has one. There are certain conventions that you will pick up over time, but they are just conventions. If you build a rather complicated package such as ImageMagick that pulls in dozens of library dependencies, and you use a less-than-common platform such as Mac with hand-compiled stuff, you might really have to dig deep and hard in some cases.

On my Mac, /usr/X11/lib contains a lot of libraries, including libpng.dylib and libpng.3.44.0.dylib in particular. Do you have those libraries? If not, you need to get X11 installed onto your Mac, and things are likely to go more smoothly.
MacOS X for Intel
I've downloaded ImageMagick 6.6.9-9 and configured, built and checked (but not installed) it with minimal issues. I have XCode 4 on my machine - and the only special option I used with ./configure was:
CC=/usr/bin/gcc ./configure
to ensure it picked up the system-provided GCC (4.2.1), not my private version of GCC 4.6.0. (A configuration test failed - for the sizeof(off_t) - because of a library issue. Using the system C compiler avoided that problem.) The ImageMagick configuration printed out:
ImageMagick is configured as follows. Please verify that this configuration
matches your expectations.
Host system type: x86_64-apple-darwin10.7.0
Build system type: x86_64-apple-darwin10.7.0
Option Value
-------------------------------------------------------------------------------
Shared libraries --enable-shared=yes yes
Static libraries --enable-static=yes yes
Module support --with-modules=no no
GNU ld --with-gnu-ld=no no
Quantum depth --with-quantum-depth=16 16
High Dynamic Range Imagery
--enable-hdri=no no
Delegate Configuration:
BZLIB --with-bzlib=yes yes
Autotrace --with-autotrace=no no
Dejavu fonts --with-dejavu-font-dir=default none
DJVU --with-djvu=yes no
DPS --with-dps=yes no
FFTW --with-fftw=yes no
FlashPIX --with-fpx=yes no
FontConfig --with-fontconfig=yes no
FreeType --with-freetype=yes yes
GhostPCL None pcl6 (unknown)
GhostXPS None gxps (unknown)
Ghostscript None gs (unknown)
Ghostscript fonts --with-gs-font-dir=default none
Ghostscript lib --with-gslib=no no
Graphviz --with-gvc=yes no
JBIG --with-jbig=yes no
JPEG v1 --with-jpeg=yes no (failed tests)
JPEG-2000 --with-jp2=yes no
LCMS v1 --with-lcms=yes no
LCMS v2 --with-lcms2=yes no
LQR --with-lqr=yes no
LZMA --with-lzma=yes no
Magick++ --with-magick-plus-plus=yes yes
OpenEXR --with-openexr=yes no
PERL --with-perl=no no
PNG --with-png=yes yes
RSVG --with-rsvg=yes no
TIFF --with-tiff=yes no
WEBP --with-webp=yes no
Windows fonts --with-windows-font-dir= none
WMF --with-wmf=yes no
X11 --with-x= yes
XML --with-xml=yes yes
ZLIB --with-zlib=yes yes
X11 Configuration:
X_CFLAGS =
X_PRE_LIBS = -lSM -lICE
X_LIBS = -L/usr/X11/lib -R/usr/X11/lib
X_EXTRA_LIBS =
Options used to compile and link:
PREFIX = /usr/local
EXEC-PREFIX = /usr/local
VERSION = 6.6.9
CC = /usr/bin/gcc -std=gnu99 -std=gnu99
CFLAGS = -D_THREAD_SAFE -D_THREAD_SAFE -pthread -fopenmp -g -O2 -Wall -D_THREAD_SAFE -pthread
CPPFLAGS = -I/usr/local/include/ImageMagick
PCFLAGS = -fopenmp
DEFS = -DHAVE_CONFIG_H
LDFLAGS = -L/usr/X11/lib -R/usr/X11/lib
MAGICK_LDFLAGS = -L/usr/local/lib -L/usr/X11/lib -R/usr/X11/lib
LIBS = -lMagickCore -lfreetype -lpng -lXext -lXt -lSM -lICE -lX11 -lbz2 -lxml2 -lz -lm -lgomp -lclparser -Wl,-framework,OpenCL -L/System/Library/Frameworks/OpenCL.framework/Versions/A/Libraries -lm -lpthread
CXX = g++
CXXFLAGS = -g -O2 -D_THREAD_SAFE -pthread
FEATURES = OpenMP
Some of the tests failed - I expect that was because I was not actually running an X11 server on the machine at the time. I'm not immediately going to investigate - you are running into problems at a much earlier stage.
MacOS X for PowerPC
As you can see, this is for Snow Leopard (10.6.7) on Intel x86/64. If you are running with Leopard (10.5.x) on PowerPC, you probably have an older XCode and libraries etc, but it would be surprising if the configuration process does not work if you have the XCode on the machine - and X11 installed.

Addressing your question about the flags in the invocation:
./configure --without-expat --with-xml --with-png=/usr/X11
In general, you cannot be certain what each flag does because the
maintainer of the package is completely free to have the flags do whatever
they want. There are certain conventions, but many are not well
understood by the package maintainers so you cannot fully rely on them.
Generally, the --with and --without are simply used to determine
what features to compile in. For example, if you do not have libexpat
installed, you can set --without-expat to complete the build which would otherwise fail.
One would suspect that the default for the package is to attempt
to build with certain features that require libexpat, and the
configury will fail if libexpat is not present, so you set
--without-expat to turn off those features.
The --with-png option with an argument is, IMO, a completely
untrustworthy kludge. I've never seen a package where it
actually works. Typically, all the configure script does
is take your /usr/X11 argument and add it (with "-L"
prepended) to LDFLAGS. The idea is that this will cause
the linker to use /usr/X11/libpng.so and not some other
version of libpng, but the simple fact is that if you
have libpng somewhere else on your system and that other
location is in your LDFLAGS, it will be used. In other
words, if you have /usr/lib/foo.so and /usr/lib/bar.so
and /usr/other/lib/foo.so and /usr/other/lib/bar.so,
there is simply no way to use /usr/lib/foo.so and /usr/lib/other/bar.so
short of deleting one of the libraries. Rather than specifying
a path as an argument to --with-png, you are probably better
off doing "--with-png LDFLAGS='-L/path -L/path2'", or just
ensure that your system is built so that the desired paths are
searched by the linker. (eg, specify the paths in /etc/ld.so.conf
or in /etc/ld.so.conf.d)

Related

Link a "toy" OS using llvm/clang

Is it possible (within reason) to build a "toy" OS on a mac using llmv/clang (and the other "normal" build tools)? By "toy" OS, I mean the simple, "Hello, World" examples found on OSDev (http://wiki.osdev.org/Bare_Bones) and x86 Bare Metal (https://github.com/cirosantilli/x86-bare-metal-examples).
My main problem is I can't figure out how to specify precisely where the linker should place the code (i.e., that the starting point should be 0x7c00, that bytes 510 and 511 need to be 0xaa55, etc.).
I would say yes it is possible within reason, at least if you consider waiting for a build of lld (and its dependency llvm) reasonable. Instructions to build lld can be found on their website or as part of this answer.
Compiling and linking for a different target than the host is relatively easy with clang. You just have to set a target, for example -target i386-none-elf for an ELF binary. Cross-compilation using clang is explained in more detail here.
As for macOS, as Micheal Petch noted, you have to use another linker than the standard ld installed. You could in theory install binutils to get an ELF ld but then you have to compile it yourself to set the target. My recommendation is to use lld which can target many architectures without the need to recompile.
With clang and a lld in place we can compile sources with
clang -c -o file.o file.c -target i386-none-elf # freestanding flags omitted
and then link them with
clang -o kernel.bin file.o -target i386-linux-elf -nostdlib -Wl,linkerscript.ld -fuse-ld=lld
Note that for linking I am using i386-linux-elf because there is a bug in clang where they just forward their input to gcc. But when using -nostdlib it is essentially the same.
If you want to see a complete example ready to build, you can take a look at https://github.com/Henje/x86-Toy-OS.

Cannot get FreeType library to work in Mac OS X Mountain Lion

I have an OpenGL project which works/compiles fine in Windows.
I wanted to port the application to Mac OS.
I got the application up and running on Mac, but the text inside the project is not visible. So I decided to use a 3rd party library for text rendering in Mac, I came across FreeType, which has many advantages such as anti-aliasing and UNICODE support.
So, I downloaded the library on my Mac, './configure'd it, did 'make' and 'make install' as I would normally do.
Then in Xcode I set search paths for both include and library directories,
/usr/local/include and /usr/local/lib respectively. Then I added 'other linker flags' in Xcode, freetype-configure --libs gave me following flags- -L/usr/local/lib -lfreetype -lz -lbz2, I added them in Xcode.
Now, whenever I include a freetype header there is no problem, but when I call any method from the freetype library, it gives me following linker error.
After looking up on google I found out that I have to set the build targets accordingly, I did that, now my application builds for i386 x86 the issue still persist.
I also tried following flags while configuring freetype ./configure CC="gcc -arch i386" CXX="g++ -arch i386"
which did not help either.
I am relatively new to Mac OS X/Unix environment, I have previously got freetype working on windows with VS 2008. Any help would be greatly appreciated.
What:
It seems you have more than one libfreetype.dylib file in your library search paths.
/master_repository/......../lib contains one and /usr/local/lib contains another
-lfreetype is a convenient way of telling the linker to look for file named libfreetype.dylib in all the directories specified with -L flags. Because -L/master_repository... comes before -L/usr/local/lib in the linker argument list, the linker uses the first instance it finds and attempts linking to that one.
Fix:
Reorganising your -L flags so that -L/usr/local/lib comes before the other. Avoid this option if you can.
Removing the extra search path, leaving only the relevant one.
Specifying the library explicitly instead of relying on the convenient -l by replacing -lfreetype with a full path to a library: /usr/local/lib/libfreetype.dylib

How to link to a specific version of the standard library (with gcc)

I've installed GCC 3.4 to /opt/gcc-3.4, and I'm using it to compile legacy code which is incompatible with GCC 4. This also means old versions of the C(++) standard libraries, binutils, and utility libraries.
It works fine for some libraries, but fails when compiling libtiff, because it picks up the system libraries in /usr/lib (see output below). This might be an autotools/configure issue, but I'm not sure. I can't find a configure switch or environment variable, and I'd rather not modify my system /usr/lib/libc.so .
So how to make sure it links to the standard library in /opt/gcc-3.4.4/lib, and ignores /lib and /usr/lib completely?
Output of make (excerpt):
libtool: link: g++ -shared -nostdlib /usr/lib/crti.o /opt/gcc-3.4.3/lib/gcc/i686-pc-linux-gnu/3.4.3/crtbeginS.o .libs/tif_stream.o -Wl,--whole-archive ../port/.libs/libport.a -Wl,--no-whole-archive -Wl,-rpath -Wl,/home/jason/d0src34/prereq/tiff-3.9.4/libtiff/.libs -Wl,-rpath -Wl,/opt/gcc-3.4.3/lib -Wl,-rpath -Wl,/home/jason/d0src34/prereq/usr/lib -Wl,-rpath -Wl,/opt/gcc-3.4.3/lib ../libtiff/.libs/libtiff.so -L/usr/lib /usr/lib/libjpeg.so -lz -L/opt/gcc-3.4.3/lib/gcc/i686-pc-linux-gnu/3.4.3 -L/opt/gcc-3.4.3/lib/gcc/i686-pc-linux-gnu/3.4.3/../../.. /opt/gcc-3.4.3/lib/libstdc++.so -L/home/jason/Downloads/gcc-3.4.3/build/i686-pc-linux-gnu/libstdc++-v3/src -L/home/jason/Downloads/gcc-3.4.3/build/i686-pc-linux-gnu/libstdc++-v3/src/.libs -L/home/jason/Downloads/gcc-3.4.3/build/gcc -lm -lc -lgcc_s /opt/gcc-3.4.3/lib/gcc/i686-pc-linux-gnu/3.4.3/crtendS.o /usr/lib/crtn.o -Wl,-soname -Wl,libtiffxx.so.3 -o .libs/libtiffxx.so.3.9.4
/home/jason/d0src34/prereq/usr/bin/ld:/usr/lib/libc.so: file format not recognized; treating as linker script
/home/jason/d0src34/prereq/usr/bin/ld:/usr/lib/libc.so:5: parse error
I've found a (hackish) answer to my own question:
I've been using binutils 2.15, because later versions have an incompatibility with GCC 3.4. In more recent versions, the format of /usr/lib/libc.so has changed, and the old binutils can't parse it.
I temporarily commented out the last line (with "GROUP"), and my code compiled:
/* GNU ld script
Use the shared library, but some functions are only in
the static library, so try that secondarily. */
OUTPUT_FORMAT(elf32-i386)
/* GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.2 ) ) */
However, I'm not really satisfied, since I can hardly tell other people who want to use the code to edit their system files. Also, I'm not sure if I've linked it to the correct glibc version, since the system /usr/lib is still in the search path, so I can't tell for sure if the binaries will work on other systems.

How to compile for a freestanding environment with GCC?

The code I'm working on is supposed to be possible to build for both hosted and freestanding environments, providing private implementations for some stdlib functions for the latter case.
Can I reliably test this with just GCC on a normal workstation/build server? Compile for freestanding environment with GCC
The "-ffreestanding" option looked promising, but it seems that it "only" disables built-ins and sets the STDC_HOSTED macro properly, it still provides all system headers.
The option "-nostdinc" is too restrictive; I still want to use the headers required for a freestanding implementation (in particular stddef.h and limits.h).
What am I missing here?
Oh, and I'm using GCC 4.4.3 for the moment, will upgrade to 4.5.0 "soon".
Well, since no answer is given yet I'd might as well describe how I made this work. It's pretty simple although depending on the target system it can be tedious.
Using "-nostdinc" means that the standard system include paths will be skipped; other include-paths given with "-I" will of course still be searched for headers.
So, for the freestanding build target I create a folder 'include-freestanding-c89' and link the relevant system headers -- float.h, iso646.h, limits.h, stdarg.h and stddef.h -- there. Other headers might be included in these, depending on your platform, so you might have to do some research and set up more links (hence the tediousness if you need to do this for several target platforms).
The C89 directory can then be used as base for 'include-freestanding-c99', the extra headers to link are stdbool.h and stdint.h
The command-line to use is then
gcc -std=c89 -nostdinc -nostdlib -ffreestanding -I include-freestanding-c89
or
gcc -std=c99 -nostdinc -nostdlib -ffreestanding -I include-freestanding-c99
This Xen Makefile uses gcc -print-search-dirs to get the directory with stddef.h and similar, adds it with -isystem, then uses -nostdinc to build:
https://github.com/mirage/xen/blob/2676bc915157ab474ee478d929b0928cf696b385/stubdom/Makefile#L35

shared library locations for matlab mex files:

I am trying to write a matlab mex function which uses libhdf5; My Linux install provides libhdf5-1.8 shared libraries and headers. However, my version of Matlab, r2007b, provides a libhdf5.so from the 1.6 release. (Matlab .mat files bootstrap hdf5, evidently). When I compile the mex, it segfaults in Matlab. If I downgrade my version of libhdf5 to 1.6 (not a long-term option), the code compiles and runs fine.
question: how do I solve this problem? how do I tell the mex compilation process to link against /usr/lib64/libhdf5.so.6 instead of /opt/matlab/bin/glnxa64/libhdf5.so.0 ? When I try to do this using -Wl,-rpath-link,/usr/lib64 in my compilation, I get errors like:
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../x86_64-pc-linux-gnu/bin/ld: warning: libhdf5.so.0, needed by /opt/matlab/matlab75/bin/glnxa64/libmat.so, may conflict with libhdf5.so.6
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: ld returned 1 exit status
mex: link of 'hdf5_read_strings.mexa64' failed.
make: *** [hdf5_read_strings.mexa64] Error 1
ack. the last resort would be to download a local copy of the hdf5-1.6.5 headers and be done with it, but this is not future proof (a Matlab version upgrade is in my future.). any ideas?
EDIT: per Ramashalanka's excellent suggestions, I
A) called mex -v to get the 3 gcc commands; the last is the linker command;
B) called that linker command with a -v to get the collect command;
C) called that collect2 -v -t and the rest of the flags.
The relevant parts of my output:
/usr/bin/ld: mode elf_x86_64
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crti.o
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/crtbeginS.o
hdf5_read_strings.o
mexversion.o
-lmx (/opt/matlab/matlab75/bin/glnxa64/libmx.so)
-lmex (/opt/matlab/matlab75/bin/glnxa64/libmex.so)
-lhdf5 (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/libhdf5.so)
/lib64/libz.so
-lm (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/libm.so)
-lstdc++ (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libstdc++.so)
-lgcc_s (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libgcc_s.so)
/lib64/libpthread.so.0
/lib64/libc.so.6
/lib64/ld-linux-x86-64.so.2
-lgcc_s (/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/libgcc_s.so)
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/crtendS.o
/usr/lib/gcc/x86_64-pc-linux-gnu/4.3.4/../../../../lib64/crtn.o
So, in fact the libhdf5.so from /usr/lib64 is being referenced. However, this is being overriden, I believe, by the environment variable LD_LIBRARY_PATH, which my version of Matlab automagically sets at run-time so it can locate its own versions of e.g. libmex.so, etc.
I am thinking that the crt_file.c example works either b/c it does not use the functions I am using (H5DOpen, which had a signature change in the move from 1.6 to 1.8 (yes, I am using -DH5_USE_16_API)), or, less likely, b/c it does not hit the parts of Matlab internals that need hdf5. ack.
The following worked on my system:
Install hdf5 version 1.8.4 (you've already done this: I installed the source and compiled to ensure it is compatible with my system, that I get gcc versions and that I get the static libraries - e.g. the binaries offered for my system are icc specific).
Make a target file. You already have your own file. I used the simple h5_crtfile.c from here (a good idea to start with this simple file first a look for warnings). I changed main to mexFunction with the usual args and included mex.h.
Specify the static 1.8.4 library you want to load explicitly (the full path with no -L for it necessary) and don't include -lhdf5 in the LDFLAGS. Include a -t option so you can ensure that there is no dynamic hdf5 library being loaded. You also need -lz, with zlib installed. For darwin we also need a -bundle in LDFLAGS:
mex CFLAGS='-I/usr/local/hdf5/include' LDFLAGS='-t /usr/local/hdf5/lib/libhdf5.a -lz -bundle' h5_crtfile.c -v
For linux, you need an equivalent position-independent call, e.g. fPIC and maybe -shared, but I don't have a linux system with a matlab license, so I can't check:
mex CFLAGS='-fPIC -I/usr/local/hdf5/include' LDFLAGS='-t /usr/local/hdf5/lib/libhdf5.a -lz -shared' h5_crtfile.c -v
Run the h5_crtfile mex file. This runs without problems on my machine. It just does a H5Fcreate and H5Fclose to create "file.h5" in the current directory, and when I call file file.h5 I get file.h5: Hierarchical Data Format (version 5) data.
Note that if I include a -lhdf5 above in step 3, then matlab aborts when I try to run the executable (because it then uses matlab's dynamic libraries which for me are version 1.6.5), so this is definitely solving the problem on my system.
Thanks for the question. My solution above is definitely much easier for me than what I was doing before. Hopefully the above works for you.
I am accepting Ramashalanka's answer because it led me to the exact solution which I will post here for completeness only:
download the hdf5-1.6.5 library from the hdf5 website, and install the header files in a local directory;
tell mex to look for "hdf5.h" in this local directory, rather than in the standard location (e.g. /usr/include.)
tell mex to compile my code and the shared object library provided by matlab, and do not use the -ldfh5 flag in LDFLAGS.
the command I used is, essentially:
/opt/matlab/matlab_default/bin/mex -v CC#gcc CXX#g++ CFLAGS#"-Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include" CXXFLAGS#"-Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include " -O -lmwblas -largeArrayDims -L/usr/lib64 hdf5_read_strings.c /opt/matlab/matlab_default/bin/glnxa64/libhdf5.so.0
this gets translated by mex into the commands:
gcc -c -I/opt/matlab/matlab75/extern/include -DMATLAB_MEX_FILE -Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include -O -DNDEBUG hdf5_read_strings.c
gcc -c -I/opt/matlab/matlab75/extern/include -DMATLAB_MEX_FILE -Wall -O3 -fPIC -I./hdf5_1.6.5/src -I/usr/include -I/opt/matlab/matlab_default/extern/include -O -DNDEBUG /opt/matlab/matlab75/extern/src/mexversion.c
gcc -O -pthread -shared -Wl,--version-script,/opt/matlab/matlab75/extern/lib/glnxa64/mexFunction.map -Wl,--no-undefined -o hdf5_read_strings.mexa64 hdf5_read_strings.o mexversion.o -lmwblas -L/usr/lib64 /opt/matlab/matlab_default/bin/glnxa64/libhdf5.so.0 -Wl,-rpath-link,/opt/matlab/matlab_default/bin/glnxa64 -L/opt/matlab/matlab_default/bin/glnxa64 -lmx -lmex -lmat -lm -lstdc++
this solution should work on all my various target machines and at least until I upgrade to matlab r2009a, which I believe uses hdf5-1.8. thanks for all the help, sorry for being so dense with this--I think I was overly-committed to using the packaged version of hdf5, rather than a local set of header files.
Note this would all have been trivial if Mathworks had provided a set of the header files with the Matlab distribution...

Resources