Compiling XNU kernel 2050 - macos

Question

I'm a bit confused on the best way to compile the latest version of the XNU kernel. I've seen lots of instructions for older kernels that came with Mac OS X 10.4 but the newer sources lack a lot of the things that the instructions contain. Just running make on the XNU kernel source brings a lot of errors about not finding ctfconvert, ctfmerge and ctfdump. Does anyone have a good "howto" to build a new kernel?

Answer 1

A new book by Wiley details the complete set of how-to in chapter 9.
Try this:
#
# Getting C++ filter
#
$ curl http://opensource.apple.com/tarballs/cxxfilt/cxxfilt-9.tar.gz > cxx.tar.gz
$ tar xvf cxx.tar.gz
$ cd cxxfilt-9
$ mkdir -p build obj sym
$ make install RC_ARCHS="i386 x86_64" RC_CFLAGS="-arch i386 -arch x86_64 -pipe" \
RC_OS=macos RC_RELEASE=Lion SRCROOT=$PWD OBJROOT=$PWD/obj \ SYMROOT=$PWD/sym DSTROOT=$PWD/build
#
# Getting DTrace – This is required for ctfconvert, a kernel build tool
#
$ curl http://opensource.apple.com/tarballs/dtrace/dtrace-90.tar.gz > dt.tar.gz
$ tar zxvf dt.tar.gz
$ cd dtrace-90
$ mkdir -p obj sym dst
$ xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge \ ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym \ DSTROOT=$PWD/dst
#
# Getting Kext Tools
#
$ wget http://opensource.apple.com/tarballs/Kext_tools/Kext_tools-180.2.1.tar.gz \ > kt.tar.gz
$ tar xvf kt.tar.gz
$ cd Kext_tools-180.2.1
$ mkdir -p obj sym dst
$ xcodebuild install -target Kextsymboltool -target setsegname \ ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym \
DSTROOT=$PWD/dst
#
# Getting Bootstrap commands – newer versions are available, but would # force xcodebuild
#
$ curl http://opensource.apple.com/tarballs/bootstrap_cmds/bootstrap_cmds-72.tar.gz \ > bc.tar.gz
$ tar zxvf bc.tar.gz
$ cd bootstrap_cmds-84
$ mkdir -p obj sym dst
$ make install RC_ARCHS="i386" RC_CFLAGS="-arch i386 -pipe" RC_OS=macos \
RC_RELEASE=Lion SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst
The tar ball versions are now different (e.g. DTrace is 96, not 90) but this should work to satisfy dependencies. Once you have them, you just run the usual make (make ARCH_CONFIGS=" X86_64" KERNEL_CONFIGS="RELEASE"). You might want to add DEBUG, to get the great debug and trace messages which are disabled by default.
This works with XCode 4.4. Just tried it now, actually.

Answer 2

I don't think this will apply to Lion kernels, which will need a later version of XCode, but the way I got around the ctf* errors while building 10.6.8 kernels, was to use XCode 3.2.*
ctf* binaries are created during the "dtrace" compile.
Just run this script from a directory without any spaces, (eg ~/xnu is perfect). The end result should be a working 10.6.8 kernel. The later (and indeed, earlier) kernels are all simpler than 10.6.8.
#!/usr/bin/env bash
# Builds 10.6.8 kernel - most other builds are easier, this one needs a little patching.
# Script assembled by sfinktah
# Invaluable source: slice - http://www.projectosx.com/forum/lofiversion/index.php/t1922.html
# Note, two (got this down to one) patches necessary to build 10.6.8 - source: http://www.insanelymac.com/forum/index.php?showtopic=261736
# This is automatically applied, but I will detail here:
#
# You will have to do this: (Automated now, but just in case)
# Define CPUFAMILY_INTEL_SANDYBRIDGE in ~/xnu-1504.15.3/osfmk/mach/machine.h
# #define CPUFAMILY_INTEL_SANDYBRIDGE 0x5490b78c
#
# Skipped this step, seemed not to be needed (eventually). Leaving notes in, just in case.
# Add line 1 in ~/xnu-1504.15.3/makedefs/MakeInc.def:
# export BUILD_STABS = 1
#
# You should probaby use a local proxy, since this script makes no effort not to redownload
# existing items. Uncomment this line accordingly.
# export http_proxy=192.168.1.6:3128
export PATH="/usr/local/bin:$PATH"
CURL="curl -O "
echo "Download the build tools source(s)"
KEXT=kext_tools-180.2.1
BOOTSTRAP=bootstrap_cmds-79
DTRACE=dtrace-90
CXXFILT=cxxfilt-9
KERNEL=xnu-1504.15.3
CCTOOLS=cctools-806
# DYLD=dyld-132.13
# LD64=ld64-95.2.12
$CURL http://www.opensource.apple.com/tarballs/cxxfilt/$CXXFILT.tar.gz \
&& $CURL http://www.opensource.apple.com/tarballs/dtrace/$DTRACE.tar.gz \
&& $CURL http://www.opensource.apple.com/tarballs/kext_tools/$KEXT.tar.gz \
&& $CURL http://www.opensource.apple.com/tarballs/bootstrap_cmds/$BOOTSTRAP.tar.gz \
&& $CURL http://www.opensource.apple.com/tarballs/cctools/$CCTOOLS.tar.gz &&
# && # $CURL http://www.opensource.apple.com/tarballs/dyld/$DYLD.tar.gz
# && # $CURL http://www.opensource.apple.com/tarballs/ld64/$LD64.tar.gz
echo "Unpack the tools" \
&&
tar zxf $CXXFILT.tar.gz \
&& tar zxf $DTRACE.tar.gz \
&& tar zxf $KEXT.tar.gz \
&& tar zxf $BOOTSTRAP.tar.gz \
&& tar zxf $CCTOOLS.tar.gz &&
# && # tar zxf $LD64.tar.gz
# && # tar zxf $DYLD.tar.gz
# Copy folder cctools-8xx/include/mach-o/ to /usr/include/mach-o/
sudo cp -a $CCTOOLS/include/mach-o /usr/include/ \
&&
echo "Build cxxfilt" \
&&
cd $CXXFILT \
&& mkdir -p obj sym dst \
&& make install RC_ARCHS="i386 x86_64" RC_CFLAGS="-arch i386 -arch x86_64 -pipe" RC_OS=macos RC_RELEASE=SnowLeopard SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
&& sudo ditto $PWD/dst/usr/local /usr/local \
&& cd .. \
&&
echo "Build dtrace" \
&&
cd $DTRACE \
&& mkdir -p obj sym dst \
&& xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
&& sudo ditto $PWD/dst/usr/local /usr/local \
&& cd .. \
&&
echo "Build kext_tools" \
&&
cd $KEXT \
&& mkdir -p obj sym dst \
&& xcodebuild install -target kextsymboltool -target setsegname ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
&& sudo ditto $PWD/dst/usr/local /usr/local \
&& cd .. \
&&
echo "Build bootstrap_cmds" \
&&
cd $BOOTSTRAP \
&& mkdir -p obj sym dst \
&& make install RC_ARCHS="i386" RC_CFLAGS="-arch i386 -pipe" RC_OS=macos RC_RELEASE=SnowLeopard SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst \
&& sudo ditto $PWD/dst/usr/local /usr/local \
&& cd .. \
&&
echo "Download the xnu source" \
&&
$CURL http://www.opensource.apple.com/tarballs/xnu/$KERNEL.tar.gz \
&&
echo "Unpack xnu" \
&&
tar zxf $KERNEL.tar.gz \
&&
echo "Build xnu" \
&&
cd $KERNEL \
&& sed -i -e '1s/.*/#define CPUFAMILY_INTEL_SANDYBRIDGE 0x5490b78c \/*/' osfmk/mach/machine.h \
&& make ARCH_CONFIGS="I386 X86_64" KERNEL_CONFIGS="RELEASE" \
&& file BUILD/obj/RELEASE_*/mach_kernel &&
# && # Removing AppleProfileFamily.kext - http://lists.apple.com/archives/darwin-kernel/2009/Dec/msg00000.html
echo "Complete. Remember to remove /System/Library/Extensions/AppleProfileFamily.kext from target." \
|| echo "Failed"
# vim: set ts=180 sts=0 sw=3 noet: