Building MariaDB with musl: /usr/bin/ld cannot find -lgcc_s - gcc

I am trying to build MariaDB v10.3 with a musl tool chain on x86_64 Debian kernel v4.19. I have mainly been using the musl-gcc gcc wrapper to achieve this. The relevant packages I installed are as follows:
musl (1.1.21-2): standard C library
musl-dev (1.1.21-2): standard C library development files
musl-tools (1.1.21-2): standard C library tools
To build MariaDB, I first run:
CC=/usr/bin/musl-gcc cmake ../ -DWITHOUT_TOKUDB=1
which exits cleanly, and then I follow that up with:
make CC=/usr/bin/musl-gcc
which error with the following message:
Scanning dependencies of target strings-t
[ 12%] Building C object unittest/strings/CMakeFiles/strings-t.dir/strings-t.c.o
[ 12%] Linking CXX executable strings-t
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
collect2: error: ld returned 1 exit status
make[2]: *** [unittest/strings/CMakeFiles/strings-t.dir/build.make:94: unittest/strings/strings-t] Error 1
make[1]: *** [CMakeFiles/Makefile2:731: unittest/strings/CMakeFiles/strings-t.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
Now I know the library that musl is looking for (libgcc_s.so) is located in /lib/gcc/x86_64-linux-gnu/8/ but my attempts to include the library using LDFLAGS or symlinking the library into /usr/lib/x86_64-linux-musl/ have failed.
Am I going about compiling MariaDB the right way? I imagine I am doing something wrong as Alpine Linux can run it.

Thus why don't looking how alpine is building it ?
https://git.alpinelinux.org/aports/tree/main/mariadb/APKBUILD?id=3ca8e70b047f37a01df42e3244014a6635893abc
seems they disable test
-DSKIP_TESTS=ON
ref: https://git.alpinelinux.org/aports/tree/main/mariadb/APKBUILD?id=3ca8e70b047f37a01df42e3244014a6635893abc#n186
And their ppc-glibc patch ?
https://git.alpinelinux.org/aports/tree/main/mariadb/ppc-remove-glibc-dep.patch?id=3ca8e70b047f37a01df42e3244014a6635893abc

I will update this answer when I become completely successful, but the solution thus far has been to use musl-cross-make to compile all libraries and such to specifically target musl. Since getting musl-cross-make I have been building all the dependencies from scratch (which is not fun :)). Thus far, I have gotten a more-or-less successful configuration and I am working on compilation (hammering out the last few dependencies).
I am using the following script to build things:
#!/bin/bash
set -euo pipefail
# musl paths
MUSL_PREFIX='/usr/local/x86_64-linux-musl'
MUSL_INC="$MUSL_PREFIX/include"
MUSL_LIB="$MUSL_PREFIX/lib"
CC='/usr/local/bin/x86_64-linux-musl-gcc'
CXX='/usr/local/bin/x86_64-linux-musl-g++'
#
# CMake couldn't locate lz4 when I installed it manually, so we bundle
# it in with the MariaDB build
#
wget https://github.com/lz4/lz4/archive/v1.7.5.tar.gz
tar -xzf v1.7.5.tar.gz
rm v1.7.5.tar.gz
mv lz4-1.7.5 /home/ajg/mariadb/storage/mroonga/vendor/groonga/vendor/
# Configure the build
CC="$CC" \
CXX="$CXX" \
LDFLAGS="-L$MUSL_LIB -Wl,-rpath,$MUSL_LIB" \
CFLAGS="-I$MUSL_INC" \
CXXFLAGS="-I$MUSL_INC" \
CPPFLAGS="-I$MUSL_INC" \
CMAKE_PREFIX_PATH="$MUSL_PREFIX" \
cmake . -DWITHOUT_TOKUDB=1 -DGRN_WITH_BUNDLED_LZ4=ON
# Make it
make \
CC="$CC" \
CXX="$CXX" \
LDFLAGS="-L$MUSL_LIB -Wl,-rpath,$MUSL_LIB" \
CFLAGS="-I$MUSL_INC" \
CXXFLAGS="-I$MUSL_INC" \
CPPFLAGS="-I$MUSL_INC"
I hope this helps someone else out in the future :)

Related

Compiling OpenFOAM's scotch on macOS

I am trying to compile the scotch library embedded into the OpenFOAM.org third-party repository here. I ran the command
make -C ./ThirdParty-dev/scotch_6.0.9/src/
and I get the below error message:
(cd libscotch ; make VERSION=6 RELEASE=0 PATCHLEVEL=9 scotch && make install)
make \
CC="gcc" \
CCD="gcc" \
scotch.h \
scotchf.h \
libscotch.so \
libscotcherr.so \
libscotcherrexit.so
gcc -O3 -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -Drestrict=__restrict -DSCOTCH_VERSION_NUM=6 -DSCOTCH_RELEASE_NUM=0 -DSCOTCH_PATCHLEVEL_NUM=9 dummysizes.c -o dummysizes -Xlinker --no-as-needed -lz -lm -lrt
ld: unknown option: --no-as-needed
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [dummysizes] Error 1
make[1]: *** [scotch] Error 2
make: *** [libscotch] Error 2
I am not sure what this error message means. If it is complaining about scotch not being available, that's why I'm compiling it in the first place. Out of desperation, I also tried to install it via brew install scotch to no avail. I would appreciate it if you could help me understand the above error message and resolve the issue.
The scotch build is a bit different in that they manage all of the OS/compiler-specific bits separately via a src/Makefile.inc that the user is responsible for providing. Of course they also provide a number of examples in the src/Make.inc/ directory, but they may not properly cover your particular OS/compiler requirements.
Since you grabbed the scotch source files from a third-party source instead of from the pristine upstream sources, you also have someone else's src/Makefile.inc that happens to be a Linux-specific version. So no surprise that it has incorrect link (or even compile) options.
The Darwin-specific makefile adjustments that are used by openfoam.com:
# Linux:
LIB = .so
ARFLAGS = $(WM_CFLAGS) -shared -o
LDFLAGS = -Xlinker --no-as-needed $(WM_LDFLAGS) -lm -lrt
# Darwin:
LIB = .dylib
ARFLAGS = $(WM_CFLAGS) -dynamiclib -undefined dynamic_lookup -o
LDFLAGS = $(WM_LDFLAGS) -lm
Without worrying about any other source of differences (in the OpenFOAM WM_CFLAGS and WM_LDFLAGS variables), it would appear that you are using Linux (gcc only?) link options for Darwin - so should be no surprise that they don't work.
The location for the pristine scotch sources move around a bit (seems to be related to their filer) but a reasonably uptodate reference is always included in the OpenFOAM ThirdParty BUILD.md. The URLs are provided as links, but also listed near the bottom of the file for easy grepping.
The current scotch link : https://gforge.inria.fr/frs/download.php/file/38352/scotch_6.1.0.tar.gz
The newest scotch is actually scotch-6.1.2 but there appears to be a regression in the dgraph calculation (the distributed graph in ptscotch) so probably better to stick with 6.1.0 for now.
Here is the information for the scotch repo itself (https://gitlab.inria.fr/scotch/scotch) - should be the most reliable source of information.

Spurious ld error on macOS (M1 processor)

I have a C project which uses libcairo. On my intel Mac, compiling Cairo with the attached script produces a x86_64 static library, which can be used without issue.
On a M1 Mac (Mac mini, though I doubt that's a factor), I've made sure to set -arch x86_64 everywhere (the output binary is used on Intel). Cairo builds successfully and produces a static library. When building a program that relies on it however, I get the following error message at link stage:
ld: warning: ignoring file (...)/lib/libcairo.a, building for macOS-x86_64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture x86_64:
"_cairo_create", referenced from:
_av_display_new in libavionics.a(display.c.o)
(...)
I've made sure to set the CMAKE_OSX_ARCHITECTURES flag to x86_64 in CMake. I can replicate the problem by creating a dummy project in Xcode and building for x86_64. A dummy project using freetype2 (built using the same method), builds & run without errors. I'm at a bit of a loss here, given that the error message says that the architectures don't match (and yet are both macOS-x86_64.
lipo -info libcairo.a returns Non-fat file: libcairo.a is architecture: x86_64.
My Cairo build script is the following:
CAIRO_CONFOPTS_COMMON="--enable-static --disable-egl --disable-glesv2 \
--disable-glesv3 --disable-glx --disable-gl --disable-valgrind \
--disable-xlib --enable-ft --disable-shared --disable-xlib-xrender \
--disable-xcb --disable-svg --disable-full-testing --disable-interpreter \
--disable-gallium --disable-beos --disable-cogl --disable-directfb \
--disable-fc --disable-ps --disable-glesv2 --disable-win32 \
--disable-win32-font --disable-drm --disable-png --disable-script --disable-quartz \
--disable-wgl --disable-gobject --disable-trace --disable-symbol-lookup --disable-zlib"
ARCH_FLAGS="-arch\\ x86_64\\ -mmacosx-version-min=10.9"
export GNUMAKEFLAGS=--no-print-directory
cd cairo
# Make sure we use the libraries we built, and not the system ones
export pixman_CFLAGS="-I$OUTDIR/include/pixman-1"
export pixman_LIBS="$OUTDIR/lib/libpixman-1.a"
export FREETYPE_CFLAGS="-I$OUTDIR/include/freetype2"
export FREETYPE_LIBS="$OUTDIR/lib/libfreetype.a"
export PKG_CONFIG_PATH="$OUTDIR/lib/pkgconfig"
eval LDFAGS="-fvisibility=hidden\\ $ARCH_FLAGS" \
CFLAGS="-fvisibility=hidden\\ $ARCH_FLAGS" \
CXXFLAGS="-fvisibility=hidden\\ $ARCH_FLAGS" \
./configure --prefix=$OUTDIR $CAIRO_CONFOPTS_COMMON $ADDCONF || exit 1
make -j9 || exit 1
make install || exit 1

Static linked vips (libvips) binary

I have been attempting to create a statically linked version of vips but have been unable to. Is it possible to create a statically linked vips command?
The platform I am compiling on is Ubuntu 16.04.
The make command I am running:
make LDFLAGS=-all-static
I am not configuring it to use python or imagemagick, (those show "no" in the config output). The error I am getting is:
/usr/bin/ld: cannot find -lgdk_pixbuf-2.0
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgio-2.0.a(libgio_2_0_la-glocalfileinfo.o): In function `lookup_gid_name':
(.text+0x11d7): warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgio-2.0.a(libgio_2_0_la-glocalvfs.o): In function `g_local_vfs_parse_name':
(.text+0x1cd): warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libglib-2.0.a(libglib_2_0_la-gutils.o): In function `g_get_user_database_entry':
(.text+0x249): warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libglib-2.0.a(libglib_2_0_la-gutils.o): In function `g_get_user_database_entry':
(.text+0xcf): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgio-2.0.a(libgio_2_0_la-glocalfileinfo.o): In function `lookup_uid_data':
(.text+0x1054): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libxml2.a(nanohttp.o): In function `xmlNanoHTTPConnectHost':
(.text+0x924): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libxml2.a(nanohttp.o): In function `xmlNanoHTTPConnectHost':
(.text+0x9f4): warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgio-2.0.a(libgio_2_0_la-gnetworkaddress.o): In function `g_network_address_parse':
(.text+0xc39): warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libgio-2.0.a(libgio_2_0_la-gnetworkaddress.o): In function `g_network_address_parse':
(.text+0xc4e): warning: Using 'endservent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
collect2: error: ld returned 1 exit status
Makefile:597: recipe for target 'vips' failed
make[2]: *** [vips] Error 1
make[2]: Leaving directory '/usr/local/src/vips-8.4.1/tools'
Makefile:631: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/usr/local/src/vips-8.4.1'
Makefile:536: recipe for target 'all' failed
make: *** [all] Error 2
From what I've found on, for example here:
Create statically-linked binary that uses getaddrinfo?
is that this is an issue with libnss. But in the ./configure --help output there is no --enable-static-flag. Even so I tried that and it didn't fix my problem.
The libraries I am linking against (according to the config output, I've truncated it):
build radiance support: yes
build analyze support: yes
build PPM support: yes
use fftw3 for FFT: yes
accelerate loops with orc: yes
ICC profile support with lcms: yes (lcms2)
SVG import with librsvg-2.0: yes
zlib: yes
file import/export with libwebp: yes
file import/export with libpng: yes (pkg-config libpng >= 1.2.9)
file import/export with libtiff: yes (pkg-config libtiff-4)
file import/export with giflib: yes (found by search)
file import/export with libjpeg: yes
use libexif to load/save JPEG metadata: yes
Is there a particular library I am linking against that is causing the problem?
After failing to convince the build mechanism to statically link, I was able to successfully create a working static vips executable with staticx, after reporting an issue to staticx and seeing that it was fixed. See here for how I'm building.
Just in case that link of mines dies in the future, here are the relevant parts:
RUN curl -sL https://github.com/libvips/libvips/releases/download/v8.9.2/vips-8.9.2.tar.gz | tar -xz -f- --strip-components=1 -C .
# TODO: Add --disable-deprecated
# Blocked by https://github.com/libvips/libvips/pull/1593
# XXX: -static doesn't work here, I'm using staticx to make the final vips binary static.
RUN CFLAGS="-O3 -flto -pipe" CXXFLAGS="-O3 -flto -pipe" \
./configure \
--disable-shared \
--disable-static \
--disable-dependency-tracking
# This is the fastest easiest way I found to compile the
# CLI as fast as possible. You can probably get more optimal,
# but it'd be a lot harder wrestling autotools.
RUN cd libvips \
&& make -j"$(nproc)"
RUN cd tools \
&& make -j"$(nproc)" vips
RUN cd tools \
&& staticx vips ../vips
Since configure automatically feature-detects, here are the Debian 10 (June 22) packages I installed:
libglib2.0-dev \
libexpat1-dev \
libjpeg-dev \
libpng-dev \
libimagequant-dev \
libexif-dev \
liborc-0.4-dev
It weighs 3.3M, which is fairly impressive and a heck of a lot smaller than distribution packages, which is why I set out to do this in the first place.
$ file vips-glibc-gcc
vips-glibc-gcc: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
$ ls -lah vips-glibc-gcc
-rwxr-xr-x 1 josh josh 3.3M Jun 23 02:51 vips-glibc-gcc
I tried like this:
$ CFLAGS="-static" CXXFLAGS="-static" ./configure --prefix=/home/john/vips --without-python --without-magick
And it seems to work:
$ ls ~/vips/lib
girepository-1.0 libvipsCC.a libvips-cpp.a libvips.la python2.7
libvips.a libvipsCC.la libvips-cpp.la pkgconfig
$ which vips
/home/john/vips/bin/vips
$ ls -l ~/vips/bin/vips
-rwxr-xr-x 1 john john 6373864 Sep 27 13:16 /home/john/vips/bin/vips
$ vips invert /data/john/pics/k2.jpg x.jpg
$ eog x.jpg
I've not tested it much though, and I suspect it's not very static. If you run ldd on the vips binary, for example, you get a long list. True static binaries do not really exist any more.
Why do you want a static binary? If it's to ease distribution, things like flatpack and snappy might be better. You can also sort-of make your own --- for example, vips comes with a simple wrapper script which can make the shared binary relocatable.

checking for suffix of object files... configure: error: cannot compute suffix of object files: cannot compile

While building ARM toolchain , I got the following error
checking for suffix of object files... configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
make[1]: *** [configure-target-libgcc] Error 1
make[1]: Leaving directory `<path>/gcc-4.3.2-arm-elf'
make: *** [all] Error 2
what might be the problem?
Did you read http://gcc.gnu.org/wiki/FAQ#configure_suffix ?
Have you installed GMP, MPFR and MPC? Are they in your library search path?
See http://gcc.gnu.org/wiki/InstallingGCC and make sure you've followed the basic instructions. By far the simplest way to build GCC (including as a cross compiler) is to follow these instructions:
Alternatively, after extracting the GCC source archive, simply run the ./contrib/download_prerequisites script in the GCC source directory. That will download the support libraries and create symlinks, causing them to be built automatically as part of the GCC build process.
"*Building GCC is not trivial, but is not difficult if you follow the instructions carefully.
Many people rush into trying to build it without reading the installation docs properly and make one or more of these common mistakes:
1) do not run ./configure from gcc src dir (this is not supported) => you need to run configure from outside the gcc source directory
2) Note: if GCC links dynamically to the prerequisite libs (GMP/MPFR/MPC) then the shared libraries must be in the dynamic linker's path (LD_LIBRARY_PATH), both when building gcc and when using the installed compiler.*"
Simple example (without dynamic link to GMP/MPFR/MPC):
tar xzf gcc-4.8.0.tar.gz
cd gcc-4.8.0
./contrib/download_prerequisites
cd ..
mkdir objdir
cd objdir
$PWD/../gcc-4.8.0/configure --prefix=/opt/gcc-4.8.0
make
make install
Sources:
Advogato Doc -
GNU Doc
export LD_LIBRARY_PATH=/path/for/libraries:$LD_LIBRARY_PATH
path/for/libraries is where the GMP MPFR and MPC libraries are present.
I was compiling GCC on ubuntu 12.04 and these linraries present in the path /usr/local/lib

Building GCC cross compiler (from "Linux" to "Windows")

I want to build "gcc cross-compiler" to compile "c/c++" applications on "Linux" environment but for "Windows" target.
I have made this so far:
Installed the necessary tools and packages for building GCC listed on "Prerequisites for GCC" page.
Downloaded required sources:
"gcc-core-4.4.1", "gcc-g++-4.4.1", "binutils-2.19.1", "w32api-3.13-mingw32", "mingwrt-3.16-mingw32"
Created this directory hierarchy:
"${HOME}/gcc/" - for final cross-compiler
"${HOME}/src/" - for sources
"${HOME}/src/build-binutils/i386-mingw32/" - for building binutils to "i386-mingw32" target
"${HOME}/src/build-gcc/i386-mingw32/" - for building gcc to "i386-mingw32" target
Builded binutils package:
cd "${HOME}/src/build-binutils/i386-mingw32/"
../../binutils-2.19.1/configure --prefix="${HOME}/gcc" --target=i386-mingw32 --disable-nls
make
make install
Copied "w32api" and "mingwrt" headers to the install directory:
cp -R "${HOME}/src/w32api-3.13-mingw32/include" "${HOME}/gcc/i386-mingw32"
cp -R "${HOME}/src/mingwrt-3.16-mingw32/include" "${HOME}/gcc/i386-mingw32"
And now when I am trying to build the "c (only) cross-compiler":
cd "${HOME}/src/build-gcc/i386-mingw32/"
../../gcc-4.4.1/configure --prefix="${HOME}/gcc" --target=i386-mingw32 --enable-languages=c --with-headers="${HOME}/gcc/i386-mingw32/include" --disable-nls
make<br>
it was building something about 4 minutes and then gives me these errors:
${HOME}/gcc/i386-mingw32/bin/ld: dllcrt2.o: No such file: No such file or directory
collect2: ld returned 1 exit status
make[2]: *** [libgcc_s.dll] Error 1
make[2]: Leaving directory `${HOME}/src/build-gcc/i386-mingw32/i386-mingw32/libgcc'
make[1]: *** [all-target-libgcc] Error 2
make[1]: Leaving directory `${HOME}/src/build-gcc/i386-mingw32'
make: *** [all] Error 2
From that error message I really don't know what to do now :-((( .
Does anybody know where is the problem?
Thanks.
That's actually OK: the way things go, you need to
build binutils
install headers
build the a partial C compiler: enough to create object files, but not enough to link
build the win32api and mingw runtime (which includes your missing dllcrt2.o)
build a complete C compiler (and other front-ends, such as C++, Fortran, Ada, whatever, if you want them)
You have successful performed step 3 above; it fails building libgcc (which is a GCC support library), but that means the C compiler core is functionnal (although it won't be able to link, it can still create valid object files). You can check that by looking at the gcc/xgcc file in your GCC build directory.
So, you need to go to the next step, not worrying about your current failure.
(To actuall install the partial C compiler, you should run make with the -k option, to have it do it best, even in the face of errors. For example, use make -k install.)
There are precompiled cross-compilers of MinGW-w64 available.
This allows to compile native 32- and 64-bit Windows binaries from Linux, a two minute tutorial is available at http://www.blogcompiler.com/2010/07/11/compile-for-windows-on-linux/
Just in case you don't want to spend a lot of time trying to build it yourself.
I grepped through the MinGW sources, and found that dllcrt2.o is something built off the mingwrt package. I assume you have to compile and install that, not just copy the headers?

Resources