android kernel build methodology - linux-kernel

We are developing android kernel drivers in a team. The drivers are developed as loadable modules and are compiled separately from the kernel source. However we need to first build android kernel source in order to build driver modules. Each developer pull kernel changes from git and builds the kernel binary. Since kernel binaries are different on each developer's local system, we need to replace both kernel binary and driver modules each time. Is there a better solution to build drivers apart from sharing kernel source on a common system?

One of you should do this locally. Assume your kernel source directory is $kern_dir and you will build the below artifacts in a new staging area (outside the kernel source hierarchy). Call this $stage.
Within the staging area, create a subdirectory hierarchy: $stage/lib/modules/$kernel_version/build ($kernel_version should match the KERNEL_VERSION in your kernel Makefile; the resulting path will be the same directory name as you're currently installing modules into on the target system); call this $build.
# Configure and build the kernel as you currently do.
cp <my-kernel-config> to $kern_dir
cd $kern_dir
make oldconfig # or whatever is needed to set your config
make
# Create the staging area and build directory
mkdir -p $build
# Copy updated kernel config to staging area:
cp .config $build
# Create necessary kernel configuration support (in staging area) to support out-of-tree modules
make O=$build modules_prepare
# Add Module.symvers to build directory
cp Module.symvers $build
You should now have a completely configured build directory that everyone can install on their own system. This is a directory containing all the header files needed to build with that kernel, together with the info describing what configuration options your kernel was built with and the Module.symvers file that describes kernel exported symbols.
Bundle that up ("cd $stage; tar czf kernel-devel.tgz lib/modules/$kernel_version/build") and distribute it to all developers.
Separately commit the "package" containing the built kernel and the associated modules (i.e. all the ones that come with the kernel and that you're not modifying); for this, use whatever mechanism you're using now. Have everyone install this kernel and build directory onto their target (run-time) system. (This should give everyone a common /boot/ and /lib/modules hierarchy.)
On their build system, everyone should install your kernel-devel.tgz from above. Assuming that the build system is running the same kernel as the target systems and you installed the kernel-devel.tgz relative to root (i.e. it's actually in /lib/modules/$kernel_version/build on your build system), you can then use the "standard" module Makefile model to build your modules. Should be something like this:
obj-m := hello.o
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
If your build system is not running the same kernel, then simply change KDIR to wherever you've installed the kernel-devel.tgz tarball.
Now anyone on your team should be able to build modules that install against the same kernel.

Related

Compile gpio-mockup in 5.4

I've checked out kernel 5.4 source and tried to compile gpio-mockup.c by running make -C /lib/modules/$(uname -r)/build M=$(pwd) modules the build succeeds but there is no gpio-mockup.ko. I think this means that I need to provide some configuration value but I don't know how and I don't know where. Regardless, can anyone help me to build the gpio-mockup kernel module?
Because the gpio-mockup module depends on some GPL symbols in the kernel it needs to be built with the kernel. This means you must build a whole new kernel.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux
cd linux
git checkout v5.4
cp /usr/lib/modules/$(uname -r)/build/.config ./
yes '' | make oldconfig
echo "CONFIG_GPIO_MOCKUP=m" >> .config
make -j $(nproc) deb-pkg LOCALVERSION=-gpio-mockup
This will produce Debian packages that can then be used to install a new kernel that will include the gpio-mockup kernel module.
NOTE: This is a vanilla kernel which may or may not have kernel modules your distribution usually provides (like ZFS on Ubuntu). Other steps may be necessary to get a custom kernel like your distribution provides.
You should enable CONFIG_IRQ_SIM=y, since gpio-mockup.ko is depended irq_sim.
please look at this picture

How to do a clean rebuild of Linux kernel modules in Yocto?

I can rm -rf tmp/ sstate-cache/ cache/ and run a whole Yocto build from scratch just fine, but I'd rather not rebuild everything (especially as packages like Qt can take some time).
I've tried:
bitbake -c cleansstate linux-iwg15 kernel-module-imx-gpu-viv cryptodev-module
Note: I've also tried cleanall, but it has the same result:
Either one of the kernel modules end up throwing this error once they get to do_compile:
ERROR: Kernel configuration is invalid.
include/generated/autoconf.h or include/config/auto.conf are missing.
Run 'make oldconfig && make prepare' on kernel src to fix it.
The {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts folder actually contains include/generated/autoconf.h
I tried copying the contents of kernel-build-artifacts to {build_dir}/tmp/work-shared/{MACHINE}/kernel-source but still get the errors.
The linux-iwg15 BitBake recipe is here.
Is there anything else that I should be cleaning before rebuilding the Linux kernel modules?
After a clean build, I did notice that kernel-build-artifacts contains kernel and scripts folders (as well as a symlink of source to ..\kernel-source) that I don't remember being there when attempting to rebuild after running bitbake -c cleansstate.
Is there anything else that I should be cleaning before rebuilding the Linux kernel modules?
Yes. bitbake -c cleansstate make-mod-scripts
Any kernel module recipe will contain inherit module. This references meta/classes/module.bbclass which contains inherit module-base. This references meta/classes/module-base.bbclass which contains:
# We do the dependency this way because the output is not preserved
# in sstate, so we must force do_compile to run (once).
do_configure[depends] += "make-mod-scripts:do_compile"
The make-mod-scripts recipe (at meta/recipes-kernel/make-mod-scripts/make-mod-scripts.bb) adds files to the {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts directory. (This is referenced as STAGING_KERNEL_BUILDDIR which is set in conf/bitbake.conf.)
Unfortunately, the kernel recipe will remove everything in the STAGING_KERNEL_BUILDDIR directory, since that directory is added to the do_shared_workdir[cleandirs] variable in meta/classes/kernel.bbclass. This ends up removing files that make-mod-scripts put there as well.

Using populate_sdk to include kernel headers

How do I include the linux kernel headers as part of the SDK package in Yocto?
I'm using Yocto 1.8 (fido) in an embedded project and want to do out-of-tree kernel module development. Currently, I can build my kernel modules (apart from bitbake) by pointing my $KERNEL_PATH to the poky/build/tmp/work-shared/<machine>/kernel-source/ directory when I run make. I don't want to do this in the long term, however, since others need to easily build the modules without installing and building a full image from bitbake.
I can generate an SDK using bitbake myimage -c populate_sdk. However, this doesn't include the kernel headers (all I ever see is sysroots/<mach>/usr/include/linux). How do I cause the kernel headers to be included in the SDK package? Also, I don't want the kernel headers to show up as part of my target image.
[Edit]
My image recipe is as follows:
EXTRA_IMAGE_FEATURES_append = " eclipse-debug debug-tweaks"
TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake"
IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL} util-linux kernel-modules netbase busybox base-passwd base-files sysvinit initscripts bash gdbserver strace sysfsutils dtc gawk ethtool grep sed wget iptables oprofile net-tools dropbear rsync stress-ng rt-tests i2c-tools"
inherit core-image
The kernel I'm using is linux-altera-ltsi-rt in the meta-altera layer.
From the fido release, the handling of kernel builds has been changed. In previous releases, you could usually just skip down to the usage example below.
In fido or any 1.8+, if you want the kernel src and build system available in the SDK, you should add
TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc"
to your image recipe. That will ensure that the new, kernel-devsrc package is installed into your toolchain.
The procedure below is just to ensure that the rest of the workflow is fully understood (even though it's not strictly part of the original question).
Usage Example
Lets assume a module Makefile as follows:
obj-m += hello-1.o
all:
make -C $(KERNEL_SRC) M=$(PWD) modules
clean:
make -C $(KERNEL_SRC) M=$(PWD) clean
Example taken from The Linux Kernel Module Programming Guide (Note that the actual commands needs to have a tab character for indentation).
Then you'll have to define KERNEL_SRC to be sysroots/<mach>/usr/src/kernel/, either in the Makefile, or from your make call. (Using a variable like KERNEL_SRC will ensure that your module recipe automatically picks the right location when building using bitbake).
To manually build your kernel module:
Source the environment-* file for your SDK.
Go to you modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make However, this will fail, as fixdep can't be found. We'll fix this manually.
cd <sdk-install-path>/sysroots/<mach>/usr/src/kernel
make modules_prepare
If this needs to be run with sudo, be sure to source the environment file in the sudo environment: sudo bash -c "source <sdk-install-path>/environment-setup-<mach> && make modules_prepare"
Go back to your modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make
This should now allow you to build your modules.
If you don't have the kernel source under sysroots/<mach>/usr/src/kernel/, we'll have to look into that.
anders answer is very good, but in recent versions of yocto the way to add kernel-devsrc seems to be
IMAGE_INSTALL += "kernel-devsrc"
which I found here: https://www.mail-archive.com/yocto#yoctoproject.org/msg36448.html
With Yocto Zeus (3.0.x) add this to your image recipe:
TOOLCHAIN_TARGET_TASK += "kernel-devsrc"
EDIT: Same for Gatesgarth (3.2.x) but the make scripts command has a new host dependency to libyaml-dev

How can I extract the environment variables used when building a recipe in Yocto?

I am working on a kernel module for a project using Yocto Linux (version 1.3). I want to use the kernel headers and the compiler and libraries from my Yocto project, but develop the kernel module without needing to run bitbake every time. My initial solution to this was to execute the devshell task and extract the environment variables using something like this:
bitbake mykernel -c devshell
Then in the new xterm window bitbake opened for me:
env | sed 's/\=\(.*\)/\="\1"/' > buildenv #put quotes around r-values in env listing
^D #(I leave the devshell)
Then copy this to my development directory and source it before running make with all of its options
KERNEL_PATH=/mypathto/build/tmp/sysroots/socfpga_cyclone5/usr/src/kernel
source ./buildenv && make -C $KERNEL_PATH V=1 M=`pwd` \
ARCH=arm CROSS-COMPILE=arm-linux-gnueabihf- \
KERNEL_VERSION=3.13.0-00298-g3c7cbb9 \
CC="arm-linux-gnueabihf-gcc -mno-thumb-interwork -marm" \
LD=arm-linux-gnueabihf-ld AR=arm-linux-gnueabihf-ar
Now to my questions:
Am I going about this completely wrong? What is the recommended way to cross-develop kernel modules? I am doing it this way because I don't want to open a bitbake devshell and do my code development in there every time.
This sort of works (I can compile working modules) but the make script gives me an error message saying that the kernel configuration is invalid. I have also tried this with KERNEL_PATH set to the the kernel package git directory (build/tmp/work///git (which contains what appears to be a valid .config file) and I get a similar error.
How can I extract the env without needing to open a devshell? I would like to write a script that extracts it so my coworkers don't have to do it manually. The devshell command opens a completely separate Xterm window, which rather dampens its scriptability...
the sdk installer is what you are looking for:
bitbake your-image -c populate_sdk
then, from your build directory go to tmp/deploy/sdk
and execute the generated shell script.
this script will allow you to generate and install an sdk.
Not only the sdk will allow you to (cross) compile your kernel by providing the needed environment variables and tools, but it will also provide a sysroot + a standalone toolchain to help you easily (and by easily I mean really easily) crosscompile applications with the autotools (as long as you provide Makefile.am and configure.ac)
just source the environment-setup-* file, got to your kernel directory and compile.
Or, for application developpment based on the autotools,
go to the folder containing your project (sources + Makefile.am and configure.ac)
and do:
libtoolize --automake
aclocal
autoconf
automake -a
now your project is ready for compilation:
./configure $CONFIGURE_FLAGS
make
make install DESTDIR=path/to/destination/dir
If you're after a quick hack, instead of Ayman's more complete solution, the scripts run to complete each build stage are available in the directory:
./build/tmp/work/{target_platform}/{package}/{version}/temp/run.do_{build_stage}
These scripts can be run standalone from the ./temp/ directory, and contain all of the environment variables required.

Reliable build of the Linux Kernel for the BeagleBone Black

Is there a reliable build of the Linux kernel for the BeagleBone Black platform anywhere ? I've followed the instructions here, as well as numerous other places, and none of them work. Either the modules don't build (the .ko files just don't get built and can't be found in the case of the link above), or there's missing build targets for the kernel. I'm using the Linaro ARM compiler (arm-linux-gnueabihf-gcc) cross compiling on 32-bit x86 Ubuntu 14.04.
Specifically, I've checked out the source at the BeagleBone Git repository, branches 3.8, 3.11, 3.12, 3.13 and 3.14. I've also checked out a 3rd party git clone here. The problems I've encountered are as follows:
3.8: Kernel will compile (with numerous warnings), but the modules fail to compile due to errors in the patching process from running patch.sh in the branch.
3.11: Kernel will compile (with numerous warnings), but the uImage-dtb.am335x-boneblack Make target will not. Compiling modules fails due to at least one file having an incorrect patch that yields incorrect C syntax.
3.12: Kernel will compile (with numerous warnings), along with the modules. Copying kernel and modules to the board succeeds. Booting the kernel fails, and freezes as soon as the bootup begins right out of the boot command in U-Boot.
3.13, 3.14: The kernel compilation fails with numerous syntax errors.
3.8.13+ from TowerTech: The kernel compilation succeeds (with numerous warnings), the module compilation fails, again due to incorrect patching from patch.sh.
In the cases where I could compile a kernel, I copied it to a fresh, vanilla board in memory using a TFTP boot server from U-Boot, and booted the kernel in memory. However, none of the successfully compiled kernels could work properly because they were missing their modules / firmware / dtbs. What could be causing the incorrect patching ? Surely, the developers who've written the code and patches must have tested it, so there may be something wrong with my workspace, though I can't imagine what at this point. The commands I used were the same as those at link 1.
I also followed the link you have mentioned and I am able to boot the BBB. I am using the mainline kernel from https://www.kernel.org/. The following are the steps I followed
Download the latest kernel. My version is 3.18-rc4
Use "omap2plus_defconfig" which is the default config for beaglebone black
Make commands are
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- omap2plus_defconfig -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- uImage dtbs LOADADDR=0x80008000 -j4.
The image produced is present in the following path.
arch/arm/boot/uImage uImage
arch/arm/boot/dts/am335x-boneblack.dtb
After this compile the modules.
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules -j4
I was using busybox and NFS server as root
Compiler used is arm-linux-gnueabi-gcc. With this I am able to boot successfully and run modules. It has the advantage that we are using the latest kernel available.
I was finally able to boot the BBB with a new kernel 3.14
I am on a beagleboneblack rev C with debian pre-installed.
I basically followed steps here
http://dev.ardupilot.com/wiki/building-for-beaglebone-black-on-linux/
Kernel download: (I didn't use git, limited bandwidth...)
https://github.com/beagleboard/linux/archive/3.14.tar.gz
Build (done on a Ubuntu 14.04) : (I am not sure if some steps are not relevant or redundant)
Install mkimage
sudo apt-get install libssl-dev
wget ftp://ftp.denx.de/pub/u-boot/u-boot-latest.tar.bz2
tar -xjf u-boot-latest.tar.bz2
cd u-boot-2014.10/
make sandbox_defconfig tools-only
sudo install tools/mkimage /usr/local/bin
Download am335x-pm-firmware.bin from http://arago-project.org/git/projects/?p=am33x-cm3.git;a=tree;f=bin;h=75a5de7aa94ff6ccbfb1b3b9dc80bc2fe5b423bf;hb=refs/heads/master and copy to kernel directory
Build the kernel
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- bb.org_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- uImage dtbs LOADADDR=0x80008000 -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- am335x-boneblack.dtb -j4
Install Modules
mkdir -p ../export/rootfs
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- INSTALL_MOD_PATH=../export/rootfs/ modules_install
Now copy modules to /lib/modules/ directory in your beaglebone black (create a new directory for the modules)
cd ../export/rootfs/lib/modules/
rsync -avz 3.14.25/ root#192.168.1.3:/lib/modules/3.14.25/
Next is to copy the zImage and am335x-boneblack.dtb to the BBB
copy zImage from arch/arm/boot to /boot/uboot/ of BBB
copy am335x-boneblack.dtb from arch/arm/boot/dts to /boot/uboot/dtbs
If you replace the existing zImage and am335x-boneblack.dtb with new ones directly and they don't work, you will be in trouble. I stored the new images in a folder in /boot/uboot and manipulated environment variables in uboot to pick the one i wanted.
I know, I could have used tftpserver..
I hope it helps someone. I spent a lot of time on this which should have been very straightforward :(
Yeah, the documentation is really out of date (circa 2011 or early 2012) and very confusing. First, the github/beagleboard/kernel area says it is deprecated and to use beagleboard/linux for the kernel. BUT, it is still be actively used 1) to add patches that have not been accepted to the kernel tree yet and 2) to add kernel configs specifically for the beaglebone devel builds. The bb.org_defconfig in the kernel tree does not turn on some debug stuff.
So.. both github/beagleboard/linux AND github/beagleboard/kernel work for me (3.14) on a BBB. If you want a standard image, use "linux"; if you want a bells-and-whistles image use "kernel".
Also, a more recent cross-compiler is in APT at gcc-arm-linux-gnueabihf. Make sure you have the "hf" (hard floating point unit) version. No need to depend on linaro.
I'll update the docs after I get the project rolling....

Resources