Library fails to build if "autoreconf -i" is run before ./configure - macos

I'm working with a tool that is supposed to simplify the build process of Unix-based apps. One of the things it does automatically is run "autoreconf -i" before doing "./configure". However, when it tries to build the expat library on OSX (Lion), the build fails:
$ tar xfz expat-2.0.1.tar.gz; cd expat-2.0.1
$ autoreconf -i
glibtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `conftools'.
glibtoolize: copying file `conftools/ltmain.sh'
glibtoolize: You should add the contents of the following files to `aclocal.m4':
glibtoolize: `/usr/bin/../share/aclocal/libtool.m4'
glibtoolize: `/usr/bin/../share/aclocal/ltoptions.m4'
glibtoolize: `/usr/bin/../share/aclocal/ltversion.m4'
glibtoolize: `/usr/bin/../share/aclocal/ltsugar.m4'
glibtoolize: `/usr/bin/../share/aclocal/lt~obsolete.m4'
glibtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and
glibtoolize: rerunning glibtoolize, to keep the correct libtool macros in-tree.
glibtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
$ ./configure
(Lots of output here)
configure: creating ./config.status
config.status: creating Makefile
config.status: WARNING: Makefile.in seems to ignore the --datarootdir setting
config.status: creating expat_config.h
$ make
bin/sh ./libtool --silent --mode=compile gcc -std=gnu99 -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o lib/xmlparse.lo -c lib/xmlparse.c
./libtool: line 473: CDPATH: command not found
./libtool: line 1297: func_opt_split: command not found
libtool: Version mismatch error. This is libtool 2.2.10, but the
libtool: definition of this LT_INIT comes from an older release.
libtool: You should recreate aclocal.m4 with macros from libtool 2.2.10
libtool: and run autoconf again.
On the other hand, if I don't do autoreconf first, it build just fiine:
$ cd ..; rm -r expat-2.0.1; tar xfz expat-2.0.1.tar.gz; cd expat-2.0.1
$ ./configure
(Lots of output here)
configure: creating ./config.status
config.status: creating Makefile
config.status: creating expat_config.h
$ make
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o lib/xmlparse.lo -c lib/xmlparse.c
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o lib/xmltok.lo -c lib/xmltok.c
/bin/sh ./libtool --silent --mode=compile gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o lib/xmlrole.lo -c lib/xmlrole.c
/bin/sh ./libtool --silent --mode=link gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -no-undefined -version-info 6:2:5 -rpath /usr/local/lib -o libexpat.la lib/xmlparse.lo lib/xmltok.lo lib/xmlrole.lo
gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlwf.o -c xmlwf/xmlwf.c
gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlfile.o -c xmlwf/xmlfile.c
gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o xmlwf/codepage.o -c xmlwf/codepage.c
gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o xmlwf/unixfilemap.o -c xmlwf/unixfilemap.c
/bin/sh ./libtool --silent --mode=link gcc -I./lib -I. -g -O2 -Wall -Wmissing-prototypes -Wstrict-prototypes -fexceptions -DHAVE_EXPAT_CONFIG_H -o xmlwf/xmlwf xmlwf/xmlwf.o xmlwf/xmlfile.o xmlwf/codepage.o xmlwf/unixfilemap.o libexpat.la
What is "autoreconf -i" doing, and why is it causing the build to fail?

I think the problem is here:
libtool: Version mismatch error. This is libtool 2.2.10, but the
libtool: definition of this LT_INIT comes from an older release.
libtool: You should recreate aclocal.m4 with macros from libtool 2.2.10
libtool: and run autoconf again.
In my experience with OS X Leopard and Snow Leopard (I have not tried Lion yet), the autotools actually installed with OS X are not recent enough to actually reconfigure anything for the system. So I usually grab the latest autotools from a GNU mirror and make a local version of them if I need to reconfigure.
"autoreconf -i" copies files from the installed GNU autotools to AC_CONFIG_AUX_DIR, an in this case is probably ltmain.sh.

A user should never run autoreconf. As ldav1s points out (+1), the autotools installed on your box do not match the tools used to generate the tarball. There are many packages in the wild that recommend (or even require) the user to run autoconf in some form to build the package; those packages are broken. A properly built package should build on a machine that does not have any of the autotools installed, and unless you are a package maintainer you should probably just uninstall them. IMO, they should not be included in default installations.
If you use a package that requires the autotools to build, please report that as a bug to the package maintainer. If a version control system (eg git, hg, svn ) is being used as the distribution mechanism, then you will either need to run the autotools to build the package or the package will have to include autotools generated files in the repository. This is why version control systems should not be used as a distribution tool for packages that use the autotools.

Related

EC2 GRPC install hangs after 12 hours still no install

I'm pretty sure it's because I am using t2.nano and not something a little more beefy.
But I have used laravel forge to provision an ec2 server, I can't deploy my application however because I need to install GRPC.
I have followed these instructions: https://cloud.google.com/php/grpc#using-pecl
And when I run: sudo pecl install grpc it runs for around 10 mins and then just gets stuck.
Seems to be running the same thing over and over again, can't quite workout the full stack trace or more importantly where it begins, but I'll post below.
DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"PHP\" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"1.41.0\" -c /tmp/pear/temp/grpc/src/core/ext/filters/client_channel/channel_connectivity.cc -fPIC -DPIC -o src/core/ext/filters/client_channel/.libs/channel_connectivity.o
/bin/bash /tmp/pear/temp/pear-build-rootRBppUP/grpc-1.41.0/libtool --mode=compile g++ -I. -I/tmp/pear/temp/grpc -I/tmp/pear/temp/pear-build-rootRBppUP/grpc-1.41.0/include -I/tmp/pear/temp/pear-build-rootRBppUP/grpc-1.41.0/main -I/tmp/pear/temp/grpc -I/usr/include/php/20200930 -I/usr/include/php/20200930/main -I/usr/include/php/20200930/TSRM -I/usr/include/php/20200930/Zend -I/usr/include/php/20200930/ext -I/usr/include/php/20200930/ext/date/lib -I/tmp/pear/temp/grpc/include -I/tmp/pear/temp/grpc/src/core/ext/upb-generated -I/tmp/pear/temp/grpc/src/core/ext/upbdefs-generated -I/tmp/pear/temp/grpc/src/php/ext/grpc -I/tmp/pear/temp/grpc/third_party/abseil-cpp -I/tmp/pear/temp/grpc/third_party/address_sorting/include -I/tmp/pear/temp/grpc/third_party/boringssl-with-bazel/src/include -I/tmp/pear/temp/grpc/third_party/re2 -I/tmp/pear/temp/grpc/third_party/upb -I/tmp/pear/temp/grpc/third_party/xxhash -DHAVE_CONFIG_H -std=c++11 -fno-exceptions -fno-rtti -g -O2 -fvisibility=hidden -DOPENSSL_NO_ASM -D_GNU_SOUR
CE -DWIN32_LEAN_AND_MEAN -D_HAS_EXCEPTIONS=0 -DNOMINMAX -DGRPC_ARES=0 -DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX='"PHP"' -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX='"1.41.0"' -c /tmp/pear/temp/grpc/src/core/ext/filters/client_channel/client_channel.cc -o src/core/ext/filters/client_channel/client_channel.lo
libtool: compile: g++ -I. -I/tmp/pear/temp/grpc -I/tmp/pear/temp/pear-build-rootRBppUP/grpc-1.41.0/include -I/tmp/pear/temp/pear-build-rootRBppUP/grpc-1.41.0/main -I/tmp/pear/temp/grpc -I/usr/include/php/20200930 -I/usr/include/php/20200930/main -I/usr/include/php/20200930/TSRM -I/usr/include/php/20200930/Zend -I/usr/include/php/20200930/ext -I/usr/include/php/20200930/ext/date/lib -I/tmp/pear/temp/grpc/include -I/tmp/pear/temp/grpc/src/core/ext/upb-generated -I/tmp/pear/temp/grpc/src/core/ext/upbdefs-generated -I/tmp/pear/temp/grpc/src/php/ext/grpc -I/tmp/pear/temp/grpc/third_party/abseil-cpp -I/tmp/pear/temp/grpc/third_party/address_sorting/include -I/tmp/pear/temp/grpc/third_party/boringssl-with-bazel/src/include -I/tmp/pear/temp/grpc/third_party/re2 -I/tmp/pear/temp/grpc/third_party/upb -I/tmp/pear/temp/grpc/third_party/xxhash -DHAVE_CONFIG_H -std=c++11 -fno-exceptions -fno-rtti -g -O2 -fvisibility=hidden -DOPENSSL_NO_ASM -D_GNU_SOURCE -DWIN32_LEAN_AND_MEAN -D_HAS_EXCEPTIONS=0 -DNOMINMAX -DGRPC_ARES=0 -
DGRPC_POSIX_FORK_ALLOW_PTHREAD_ATFORK=1 -DGRPC_XDS_USER_AGENT_NAME_SUFFIX=\"PHP\" -DGRPC_XDS_USER_AGENT_VERSION_SUFFIX=\"1.41.0\" -c /tmp/pear/temp/grpc/src/core/ext/filters/client_channel/client_channel.cc -fPIC -DPIC -o src/core/ext/filters/client_channel/.libs/client_channel.o
If there are any logs I can grab please drop the command in to tail or cat and I'll attach them to the question!
Upgrade to a bigger tier than the t2.nano and it should work. I think it's because of the RAM limit. I had the same issue with some instances in Digital Ocean.

Has anyone built gcc 8.2.0 on OpenBSD 6.3?

I am getting the following error when I try to build gcc 8.2.0 on OpenBSD 6.3:
/bin/sh ./libtool --tag=CC --mode=compile /home/shane/gcc-8.2.0/host-i386-unknown-openbsd6.3/gcc/xgcc -B/home/shane/gcc-8.2.0/host-i386-unknown-openbsd6.3/gcc/ -B/opt/gcc-8.2.0/i386-unknown-openbsd6.3/bin/ -B/opt/gcc-8.2.0/i386-unknown-openbsd6.3/lib/ -isystem /opt/gcc-8.2.0/i386-unknown-openbsd6.3/include -isystem /opt/gcc-8.2.0/i386-unknown-openbsd6.3/sys-include -DHAVE_CONFIG_H -I../.././libatomic/config/x86 -I../.././libatomic/config/posix -I../.././libatomic -I. -march=i486 -mtune=i386 -fomit-frame-pointer -Wall -Werror -pthread -g -O2 -MT fence.lo -MD -MP -MF .deps/fence.Tpo -c -o fence.lo ../.././libatomic/fence.c
libtool: compile: /home/shane/gcc-8.2.0/host-i386-unknown-openbsd6.3/gcc/xgcc -B/home/shane/gcc-8.2.0/host-i386-unknown-openbsd6.3/gcc/ -B/opt/gcc-8.2.0/i386-unknown-openbsd6.3/bin/ -B/opt/gcc-8.2.0/i386-unknown-openbsd6.3/lib/ -isystem /opt/gcc-8.2.0/i386-unknown-openbsd6.3/include -isystem /opt/gcc-8.2.0/i386-unknown-openbsd6.3/sys-include -DHAVE_CONFIG_H -I../.././libatomic/config/x86 -I../.././libatomic/config/posix -I../.././libatomic -I. -march=i486 -mtune=i386 -fomit-frame-pointer -Wall -Werror -pthread -g -O2 -MT fence.lo -MD -MP -MF .deps/fence.Tpo -c ../.././libatomic/fence.c -fPIC -DPIC -o .libs/fence.o
../.././libatomic/fence.c:26:10: fatal error: stdatomic.h: No such file or directory
#include <stdatomic.h>
^~~~~~~~~~~~~
compilation terminated.
gmake[4]: *** [Makefile:491: fence.lo] Error 1
gmake[4]: Leaving directory '/home/shane/gcc-8.2.0/i386-unknown-openbsd6.3/libatomic'
gmake[3]: *** [Makefile:524: all-recursive] Error 1
gmake[3]: Leaving directory '/home/shane/gcc-8.2.0/i386-unknown openbsd6.3/libatomic'
gmake[2]: *** [Makefile:360: all] Error 2
gmake[2]: Leaving directory '/home/shane/gcc-8.2.0/i386-unknown-openbsd6.3/libatomic'
gmake[1]: *** [Makefile:21680: all-target-libatomic] Error 2
gmake[1]: Leaving directory '/home/shane/gcc-8.2.0'
gmake: *** [Makefile:943: all] Error 2
My process:
Install OpenBSD (using defaults)
Run syspatch
Add some packages:
# pkg_add wget
# pkg_add gmp mpfr libmpc
# pkg_add gmake
Get the source:
$ wget ftp://ftp.nluug.nl/mirror/languages/gcc/releases/gcc-8.2.0/gcc-8.2.0.tar.gz
$ tar xzf gcc-8.2.0.tar.gz
$ cd gcc-8.2.0
Build:
$ ./configure --prefix=/opt/gcc-8.2.0 --with-gmp=/usr/local --enable-languages=c,c++
$ gmake
This also fails with gcc 8.1.0 in a similar way.
I also tried using clang rather than gcc to build, like this:
$ CC=cc CXX=c++ ./configure --prefix=/opt/gcc-8.2.0 --with-gmp=/usr/local --enable-languages=c,c++
That made no difference, which makes some sense as apparently it is failing in stage 3 (gcc compiling itself again for a final check).
Annoyingly, there is a stdatomic.h file in the tarball:
$ find gcc-8.2.0 -name stdatomic.h
gcc-8.2.0/gcc/ginclude/stdatomic.h
So I have no idea what I am missing.
Has anyone successfully built this? How?

gcc 4.9.2 build error on RHEL6

I'm trying to build gcc 4.9.2 on RHEL6, as follows:
unpack the source tarball (to ~/gcc-4.9.2)
in the source dir:
./contrib/download_prerequisites
create a separate build directory ~/gcc-4.9.2-build
in the build dir:
~/gcc-4.9.2/configure --prefix=/usr/local/gcc-4.9.2
make
This fails with:
make[6]: Entering directory `/root/gcc-4.9.2-build-try0/x86_64-unknown-linux-gnu/libjava/classpath/native/jni/java-math'
/bin/sh ../../../libtool --tag=CC --mode=compile /root/gcc-4.9.2-build-try0/./gcc/xgcc -B/root/gcc-4.9.2-build-try0/./gcc/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/bin/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/lib/ -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/include -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/sys-include -DHAVE_CONFIG_H -I. -I/root/gcc-4.9.2/libjava/classpath/native/jni/java-math -I../../../include -I/root/gcc-4.9.2/libjava/classpath/include -I/root/gcc-4.9.2/libjava/classpath/native/jni/classpath -I/root/gcc-4.9.2/libjava/classpath/native/jni/native-lib -W -Wall -Wmissing-declarations -Wwrite-strings -Wmissing-prototypes -Wno-long-long -I/usr/include -g -O2 -MT gnu_java_math_GMP.lo -MD -MP -MF .deps/gnu_java_math_GMP.Tpo -c -o gnu_java_math_GMP.lo /root/gcc-4.9.2/libjava/classpath/native/jni/java-math/gnu_java_math_GMP.c
libtool: compile: /root/gcc-4.9.2-build-try0/./gcc/xgcc -B/root/gcc-4.9.2-build-try0/./gcc/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/bin/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/lib/ -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/include -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/sys-include -DHAVE_CONFIG_H -I. -I/root/gcc-4.9.2/libjava/classpath/native/jni/java-math -I../../../include -I/root/gcc-4.9.2/libjava/classpath/include -I/root/gcc-4.9.2/libjava/classpath/native/jni/classpath -I/root/gcc-4.9.2/libjava/classpath/native/jni/native-lib -W -Wall -Wmissing-declarations -Wwrite-strings -Wmissing-prototypes -Wno-long-long -I/usr/include -g -O2 -MT gnu_java_math_GMP.lo -MD -MP -MF .deps/gnu_java_math_GMP.Tpo -c /root/gcc-4.9.2/libjava/classpath/native/jni/java-math/gnu_java_math_GMP.c -fPIC -DPIC -o .libs/gnu_java_math_GMP.o
mv -f .deps/gnu_java_math_GMP.Tpo .deps/gnu_java_math_GMP.Plo
/bin/sh ../../../libtool --tag=CC --mode=link /root/gcc-4.9.2-build-try0/./gcc/xgcc -B/root/gcc-4.9.2-build-try0/./gcc/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/bin/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/lib/ -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/include -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/sys-include -W -Wall -Wmissing-declarations -Wwrite-strings -Wmissing-prototypes -Wno-long-long -I/usr/include -g -O2 -module -version-info 0:0:0 -no-undefined -lgmp -avoid-version -o libjavamath.la -rpath /usr/local/gcc-4.9.2/lib/../lib64/gcj-4.9.2-15 gnu_java_math_GMP.lo ../../../native/jni/classpath/jcl.lo -lrt
libtool: link: /root/gcc-4.9.2-build-try0/./gcc/xgcc -B/root/gcc-4.9.2-build-try0/./gcc/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/bin/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/lib/ -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/include -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/sys-include -shared .libs/gnu_java_math_GMP.o ../../../native/jni/classpath/.libs/jcl.o /root/gcc-4.9.2-build-try0/./gmp/.libs/libgmp.a -lrt -Wl,-soname -Wl,libjavamath.so -o .libs/libjavamath.so
/usr/bin/ld: /root/gcc-4.9.2-build-try0/./gmp/.libs/libgmp.a(mp_set_fns.o): relocation R_X86_64_32 against `__gmp_default_allocate' can not be used when making a shared object; recompile with -fPIC
/root/gcc-4.9.2-build-try0/./gmp/.libs/libgmp.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
make[6]: *** [libjavamath.la] Error 1
make[6]: Leaving directory `/root/gcc-4.9.2-build-try0/x86_64-unknown-linux-gnu/libjava/classpath/native/jni/java-math'
A little googling pointed me at the --disable-shared option, which sort of helped, but now:
in the build dir:
~/gcc-4.9.2/configure --prefix=/usr/local/gcc-4.9.2 --disable-shared
make
yielded a slightly different error:
make[8]: Entering directory `/root/gcc-4.9.2-build/x86_64-unknown-linux-gnu/32/libjava/classpath/native/jni/classpath'
/bin/sh ../../../libtool --tag=CC --mode=compile /root/gcc-4.9.2-build/./gcc/xgcc -B/root/gcc-4.9.2-build/./gcc/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/bin/ -B/usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/lib/ -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/include -isystem /usr/local/gcc-4.9.2/x86_64-unknown-linux-gnu/sys-include -m32 -DHAVE_CONFIG_H -I. -I/root/gcc-4.9.2/libjava/classpath/native/jni/classpath -I../../../include -I/root/gcc-4.9.2/libjava/classpath/include -I/root/gcc-4.9.2/libjava/classpath/native/jni/classpath -I/root/gcc-4.9.2/libjava/classpath/native/jni/native-lib -W -Wall -Wmissing-declarations -Wwrite-strings -Wmissing-prototypes -Wno-long-long -Wstrict-prototypes -pedantic -fexceptions -fasynchronous-unwind-tables -g -O2 -m32 -MT jcl.lo -MD -MP -MF .deps/jcl.Tpo -c -o jcl.lo /root/gcc-4.9.2/libjava/classpath/native/jni/classpath/jcl.c
libtool: compile: not configured to build any kind of library
libtool: compile: See the libtool documentation for more information.
libtool: compile: Fatal configuration error.
make[8]: *** [jcl.lo] Error 1
make[8]: Leaving directory `/root/gcc-4.9.2-build/x86_64-unknown-linux-gnu/32/libjava/classpath/native/jni/classpath'
Adding the --enable-static option made no difference.
Any suggestions on what might be wrong?
BTW, gcc 4.9.1 gave the same errors.
System: Linux 2.6.32-431.29.2.el6.x86_64 #1 SMP Sun Jul 27 15:55:46 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
System is RHEL6, fully updated via Redhat, and has no custom RPMs that I know of.
Just completed building 4.9.2 on a CentOS 6.4 VM using 4.4.7 gcc. Ran into an out of space issue, which caused the build to fail, but once resolved, everything else went smoothly.
Looks like your build is failing in the Java side of town? Not sure if you need the Java support, so have you tried compiling just the c/c++ language support?
~/gcc-4.9.2/configure --enable-languages=c,c++ ...
I also don't require 32-bit binaries, so I add --disable-multilib to the list of configure parameters.

Compile Linux Kernel Modules into LLVM .bc bitcode

Background
I'm trying to compile certain drivers within the Linux kernel: drm (drivers/gpu/drm/drm_drv.o) and radeon (drivers/gpu/drm/radeon/) gpu drivers. I'm using LLVM for the purposes of static analysis (tracking the arguments used in copy_to/from_user() invocations).
So far, I'm able to compile the actual modules using the Makefile as shown below:
make CC=clang CFLAGS=-emit-llvm drivers/gpu/drm/radeon/
But this does not actually emit any llvm bitcode -- I need the .bc files to run my pass with opt.
I only know how to generate .bc files when using clang directly (like below), but not with Makefiles...
clang -emit-llvm hello.c -c -o hello.bc
Since that worked, I grabbed the verbose output of the GNU make operation, changed gcc to clang, and ran it to create the .bc file, which also worked:
clang -emit-llvm [[tons of CFLAGS]] -c -o drm_drv.bc drivers/gpu/drm/drm_drv.c
The only problem with that is I can only process a single C file in the kernel module at a time. Also it's very tedious to do this approach...
Main Problem
Which brings me to my main question: How would you go about emitting llvm .bc bitcode files using the kernel's Makefiles?
Or, if .bc bitcode creation must be done on a per-file basis, then how would I link them all together at the end so that I can run an LLVM opt pass on the aggregate of all the .bc files in a kernel module?
The best way to get LLVM IR in bitcode form out of Clang is to do LTO with the -flto command line flag.
If you have multiple translation units you can combine them together using the llvm-lto tool to "link" the bitcode files. Typically it generates code, but you can get it to drop the merged LLVM IR module with the -save-merged-module flag.
But none of this is really a supported interface. If this is a significantly useful workflow, you should talk to the LLVM developers about supporting something more akin to ld's -r.
I used clang to instrument ext2 module. There are 3 things that you want to do:
1) convert .c into .bc
2) run your optimiser on .bc file, and create an ext2-opt.o file
3) create ext2-instrumented.ko file from the ext2-opt.o file.
along with this, you also require a linux version compiled with clang. I could get linux 4.17 compiled with clang version 3.8.1 after disabling few modules. you can get it here
now, lets move to step 1 - as you said, run Make in verbose mode
make V=1 M=fs/ext2
and grab all options that are spit out by Makefile. the default compiler will be gcc. replace gcc with clang, add --emit-llvm command, and replace .o with .bc. the resultant compilataion command on my machine looks like so:
cd ../../ && clang -emit-llvm -Wp,-MD,fs/ext2/.all.o.d -nostdinc -isystem /usr/local/bin/../lib/clang/3.8.1/include -I./arch/x86/include -Iarch/x86/include/generated/uapi -Iarch/x86/include/generated -Iinclude -I./arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -Qunused-arguments -Wno-unknown-warning-option -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -no-integrated-as -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -O2 -Wframe-larger-than=1024 -fno-stack-protector -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-asm-operand-widths -Wno-initializer-overrides -fno-builtin -Wno-tautological-compare -mno-global-merge -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-unknown-warning-option -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(all)" -D"KBUILD_MODNAME=KBUILD_STR(ext2)" -mcmodel=kernel -c -o fs/ext2_instrumented/all.bc fs/ext2/all.c && cd -
this will create a .bc file for you. note that I have changed the resultant .bc codes directory to ext2_instrumented, i.e. my bc file is not created in fs/ext2 but in another folder called fs/ext2_instrumented. this is my work folder. I want this folder to not have any .c files, only .bc files. I need this because the default KBuild system looks for .c files in a folder. More on this later.
Step 2: Run all optimisation passes on your resultant ext2-instrumented.bc file using opt command like so:
opt -load $(DIR)/build/FSlice.so -constprop -sccp -mergereturn -sink -licm -reg2mem all.bc -o all.inst.bc
llvm-link -o all.inst2.bc $(DIR)/build/libFSlice.bc all.inst.bc
clang -mcmodel=kernel -c all.inst2.bc -o all.o
this will result into a .o file, which we will now compile into a .ko file in the next step:
Step 3:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) ext2instrumented.ko
Kbuild system is tough to understand, and even tougher to make modifications on. I would recommend use this 2 folder hack (1st folder to create .bc file and second folder to create .o and .ko files). To summarise, here is my resultant makefile in ext2_instrumented folder:
DIR=/home/fslice/fslice
obj-m += ext2instrumented.o
ext2instrumented-objs := all.o
all:
cd ../../ && clang -emit-llvm -Wp,-MD,fs/ext2/.all.o.d -nostdinc -isystem /usr/local/bin/../lib/clang/3.8.1/include -I./arch/x86/include -Iarch/x86/include/generated/uapi -Iarch/x86/include/generated -Iinclude -I./arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -Qunused-arguments -Wno-unknown-warning-option -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -no-integrated-as -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -O2 -Wframe-larger-than=1024 -fno-stack-protector -Wno-unused-variable -Wno-format-invalid-specifier -Wno-gnu -Wno-asm-operand-widths -Wno-initializer-overrides -fno-builtin -Wno-tautological-compare -mno-global-merge -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-unknown-warning-option -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(all)" -D"KBUILD_MODNAME=KBUILD_STR(ext2)" -mcmodel=kernel -c -o fs/ext2_instrumented/all.bc fs/ext2/all.c && cd -
opt -load $(DIR)/build/FSlice.so -constprop -sccp -mergereturn -sink -licm -reg2mem all.bc -o all.inst.bc
llvm-link -o all.inst2.bc $(DIR)/build/libFSlice.bc all.inst.bc
clang -mcmodel=kernel -c all.inst2.bc -o all.o
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) ext2instrumented.ko
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
rm -rf *.bc

Programming in Lua on OS X?

What can I use to program Lua script on Mac OS X? I'm looking for something that I can use to compile/interpret Lua script on OS X.
My preferred way:
brew install lua
Thanks, Max!
And if you need to know how to install Homebrew, see Link and:
/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
The Lua source easily compiles with no changes on the mac. It will build lua (the interpreter which can act on a source script, a pre-compiled script or interactively) and luac which can be used to pre-compile source scripts.
From the lua.org website: http://luabinaries.luaforge.net/download.html. The ones you want are the darwin binaries (they say Mac OS X in the description).
My favorite way (from the shell):
sudo port install lua
I LOVE macports!
Here is my terminal session from compiling and installing Lua from source, basically following these directions. I already had Apple's Developer Tools installed, and /usr/local/bin was already in my PATH, so I was able to skip some of the more time-consuming and/or tedious steps in the directions.
$ cd ~/Downloads
$ tar -xf lua-5.1.4.tar
$ cd lua-5.1.4
$ make macosx
cd src && make macosx
make all MYCFLAGS=-DLUA_USE_LINUX MYLIBS="-lreadline"
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lapi.o lapi.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lcode.o lcode.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ldebug.o ldebug.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ldo.o ldo.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ldump.o ldump.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lfunc.o lfunc.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lgc.o lgc.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o llex.o llex.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lmem.o lmem.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lobject.o lobject.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lopcodes.o lopcodes.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lparser.o lparser.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lstate.o lstate.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lstring.o lstring.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ltable.o ltable.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ltm.o ltm.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lundump.o lundump.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lvm.o lvm.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lzio.o lzio.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lauxlib.o lauxlib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lbaselib.o lbaselib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ldblib.o ldblib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o liolib.o liolib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lmathlib.o lmathlib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o loslib.o loslib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o ltablib.o ltablib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lstrlib.o lstrlib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o loadlib.o loadlib.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o linit.o linit.c
ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o
ranlib liblua.a
gcc -O2 -Wall -DLUA_USE_LINUX -c -o lua.o lua.c
gcc -o lua lua.o liblua.a -lm -lreadline
gcc -O2 -Wall -DLUA_USE_LINUX -c -o luac.o luac.c
gcc -O2 -Wall -DLUA_USE_LINUX -c -o print.o print.c
gcc -o luac luac.o print.o liblua.a -lm -lreadline
$ make test
src/lua test/hello.lua
Hello world, from Lua 5.1!
$ sudo make install INSTALL_TOP=/usr/local
Password:
cd src && mkdir -p /usr/local/bin /usr/local/include /usr/local/lib /usr/local/man/man1 /usr/local/share/lua/5.1 /usr/local/lib/lua/5.1
cd src && install -p -m 0755 lua luac /usr/local/bin
cd src && install -p -m 0644 lua.h luaconf.h lualib.h lauxlib.h ../etc/lua.hpp /usr/local/include
cd src && install -p -m 0644 liblua.a /usr/local/lib
cd doc && install -p -m 0644 lua.1 luac.1 /usr/local/man/man1
$ lua
Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio
> print "Hi"
Hi
> = 2 + 3
5
> ^c
$ cd test
$ lua factorial.lua
0! = 1
1! = 1
2! = 2
3! = 6
4! = 24
5! = 120
6! = 720
7! = 5040
8! = 40320
9! = 362880
10! = 3628800
11! = 39916800
12! = 479001600
13! = 6227020800
14! = 87178291200
15! = 1307674368000
16! = 20922789888000
If you don't want to compile your own Lua binaries, you can try ZeroBrane Studio Lua IDE, which comes packaged as a .dmg file for OSX. It's an IDE that allows you to edit and debug your Lua scripts. If you are just starting with Lua, it also includes 50+ examples and demo scripts, as well as integrated instructions on running them, so you won't be facing an empty screen not knowing where to start.
I just recently found Rudix—a maintained collection of precompiled Unix software for Mac.
While I'm sure you've already figured out a way of installing Lua, I came across your question by Googling the same thing. For anyone that's interested, here's the link to a recent Lua 5.1.4 dmg.

Resources