I can rm -rf tmp/ sstate-cache/ cache/ and run a whole Yocto build from scratch just fine, but I'd rather not rebuild everything (especially as packages like Qt can take some time).
I've tried:
bitbake -c cleansstate linux-iwg15 kernel-module-imx-gpu-viv cryptodev-module
Note: I've also tried cleanall, but it has the same result:
Either one of the kernel modules end up throwing this error once they get to do_compile:
ERROR: Kernel configuration is invalid.
include/generated/autoconf.h or include/config/auto.conf are missing.
Run 'make oldconfig && make prepare' on kernel src to fix it.
The {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts folder actually contains include/generated/autoconf.h
I tried copying the contents of kernel-build-artifacts to {build_dir}/tmp/work-shared/{MACHINE}/kernel-source but still get the errors.
The linux-iwg15 BitBake recipe is here.
Is there anything else that I should be cleaning before rebuilding the Linux kernel modules?
After a clean build, I did notice that kernel-build-artifacts contains kernel and scripts folders (as well as a symlink of source to ..\kernel-source) that I don't remember being there when attempting to rebuild after running bitbake -c cleansstate.
Is there anything else that I should be cleaning before rebuilding the Linux kernel modules?
Yes. bitbake -c cleansstate make-mod-scripts
Any kernel module recipe will contain inherit module. This references meta/classes/module.bbclass which contains inherit module-base. This references meta/classes/module-base.bbclass which contains:
# We do the dependency this way because the output is not preserved
# in sstate, so we must force do_compile to run (once).
do_configure[depends] += "make-mod-scripts:do_compile"
The make-mod-scripts recipe (at meta/recipes-kernel/make-mod-scripts/make-mod-scripts.bb) adds files to the {build_dir}/tmp/work-shared/{MACHINE}/kernel-build-artifacts directory. (This is referenced as STAGING_KERNEL_BUILDDIR which is set in conf/bitbake.conf.)
Unfortunately, the kernel recipe will remove everything in the STAGING_KERNEL_BUILDDIR directory, since that directory is added to the do_shared_workdir[cleandirs] variable in meta/classes/kernel.bbclass. This ends up removing files that make-mod-scripts put there as well.
Related
I'm trying to compile a code to run in parallel on a supercomputer. I know that others have compiled this code to run on the same computer, but for some reason I am having trouble even when using the same methodology as them. For now I'm just trying to compile the code to run in serial as that should be easier to troubleshoot.
configure seems to work correctly.
However make install returns the following:
> make install
CDPATH="${ZSH_VERSION+.}:" && cd .. && /bin/sh /home1/username123/code123/config/missing aclocal-1.13 -I ./config -I /home1/username123/code123/build-tools/aclocal -I /usr/local/share/aclocal
aclocal-1.13: error: couldn't open directory '/usr/local/share/aclocal': No such file or directory
Makefile:534: recipe for target '../aclocal.m4' failed
make: *** [../aclocal.m4] Error 1
aclocal is indeed not located at /usr/local/share/aclocal, it is located at /usr/bin/aclocal - but as /usr/bin is in my path, I don't understand why the location is an issue.
As has been made clear in comments on the question, the problem was that the project sources were copied onto the target system in a way that failed to preserve their original timestamps. The Autotools, through make, use file timestamps to determine which files are out of date, and in particular, Autools-generated Makefiles contain rules for rebuilding the build system itself that can be triggered this way.
It is not ordinarily necessary or desirable to rebuild an Autotools project's build system, except in conjunction with actually performing maintenance on it. It is often the case, in fact, that the necessary support for that is not available locally. To avoid the build system thinking that it needs to rebuild itself, it is important to preserve the file timestamps from the distribution archive. For some packages, it also works to pass the --disable-maintainer-mode argument to the configure script, but by no means do all Autotools configure scripts support that.
The archive extraction tools for the typical archive formats in which Autotools-based packages are distributed do, by default, preserve timestamps when unpacking, so the ordinary procedure of
unpack the archive on the target system (e.g. tar xzf foo-1.2.3.tar.gz)
change to the unpacked source directory (e.g. cd foo-1.2.3)
configure; make; make install
normally does the right thing. Substituting something else for (1), however, such as copying the unpacked source tree from somewhere else, may cause the package to think it needs to rebuild the build system. If that works correctly then it's no big deal, but it is not uncommon that it doesn't. That's what happened here, and following the standard procedure described above solved the problem.
I'm using a embedded linux on a SOM module, based on IMX6.
I would like to patch my kernel,it works fine, it has been patched, but i would like to add some features.
I apply the old patches on the sources, and add my code, and create a new patch.
In yocto I add my patch to the recipe and bitbake it, I start the board with the new loaded kernel, and it seems it does not have my modification!
For bitbake my new kernel, I type :
bitbake -c clean linux-toradex-fsl
bitbake linux-toradex-fsl
It compiles without any errors, I copy my uImage in my tftp folder, but my features do not seems to be executed.
So I try to create a simple patch, with
printk(KERN_WARNING"This is a test\n");
I place it after another warning, called and displayed.
And it does not work either.
I check with -c devshell and my modification are applied in yocto, the patch works here.
I also check the uImage file, it has the same size, with or without my patches.
Do you have some ideas for this issue?
Regards,
Pierre-Olivier
Solution
I found the solution after some tries, I just need to run
bitbake -c cleansstate linux-toradex-fsl
bitbake my-image
And the image is bitbaked with the patched kernel.
Bitbake is funny about -clean. It tries to keep a state directory, so -clean just runs the normal "clean" in the makefile, but the makefile can be short-circuited by the shared state, which can include zipped up prebuilt binaries from a previous build. Try this:
bitbake -c cleansstate linux-toradex-fsl
bitbake linux-toradex-fsl
note the extra 's' in cleansstate (clean shared state).
How do I include the linux kernel headers as part of the SDK package in Yocto?
I'm using Yocto 1.8 (fido) in an embedded project and want to do out-of-tree kernel module development. Currently, I can build my kernel modules (apart from bitbake) by pointing my $KERNEL_PATH to the poky/build/tmp/work-shared/<machine>/kernel-source/ directory when I run make. I don't want to do this in the long term, however, since others need to easily build the modules without installing and building a full image from bitbake.
I can generate an SDK using bitbake myimage -c populate_sdk. However, this doesn't include the kernel headers (all I ever see is sysroots/<mach>/usr/include/linux). How do I cause the kernel headers to be included in the SDK package? Also, I don't want the kernel headers to show up as part of my target image.
[Edit]
My image recipe is as follows:
EXTRA_IMAGE_FEATURES_append = " eclipse-debug debug-tweaks"
TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake"
IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL} util-linux kernel-modules netbase busybox base-passwd base-files sysvinit initscripts bash gdbserver strace sysfsutils dtc gawk ethtool grep sed wget iptables oprofile net-tools dropbear rsync stress-ng rt-tests i2c-tools"
inherit core-image
The kernel I'm using is linux-altera-ltsi-rt in the meta-altera layer.
From the fido release, the handling of kernel builds has been changed. In previous releases, you could usually just skip down to the usage example below.
In fido or any 1.8+, if you want the kernel src and build system available in the SDK, you should add
TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc"
to your image recipe. That will ensure that the new, kernel-devsrc package is installed into your toolchain.
The procedure below is just to ensure that the rest of the workflow is fully understood (even though it's not strictly part of the original question).
Usage Example
Lets assume a module Makefile as follows:
obj-m += hello-1.o
all:
make -C $(KERNEL_SRC) M=$(PWD) modules
clean:
make -C $(KERNEL_SRC) M=$(PWD) clean
Example taken from The Linux Kernel Module Programming Guide (Note that the actual commands needs to have a tab character for indentation).
Then you'll have to define KERNEL_SRC to be sysroots/<mach>/usr/src/kernel/, either in the Makefile, or from your make call. (Using a variable like KERNEL_SRC will ensure that your module recipe automatically picks the right location when building using bitbake).
To manually build your kernel module:
Source the environment-* file for your SDK.
Go to you modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make However, this will fail, as fixdep can't be found. We'll fix this manually.
cd <sdk-install-path>/sysroots/<mach>/usr/src/kernel
make modules_prepare
If this needs to be run with sudo, be sure to source the environment file in the sudo environment: sudo bash -c "source <sdk-install-path>/environment-setup-<mach> && make modules_prepare"
Go back to your modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make
This should now allow you to build your modules.
If you don't have the kernel source under sysroots/<mach>/usr/src/kernel/, we'll have to look into that.
anders answer is very good, but in recent versions of yocto the way to add kernel-devsrc seems to be
IMAGE_INSTALL += "kernel-devsrc"
which I found here: https://www.mail-archive.com/yocto#yoctoproject.org/msg36448.html
With Yocto Zeus (3.0.x) add this to your image recipe:
TOOLCHAIN_TARGET_TASK += "kernel-devsrc"
EDIT: Same for Gatesgarth (3.2.x) but the make scripts command has a new host dependency to libyaml-dev
I am working on a kernel module for a project using Yocto Linux (version 1.3). I want to use the kernel headers and the compiler and libraries from my Yocto project, but develop the kernel module without needing to run bitbake every time. My initial solution to this was to execute the devshell task and extract the environment variables using something like this:
bitbake mykernel -c devshell
Then in the new xterm window bitbake opened for me:
env | sed 's/\=\(.*\)/\="\1"/' > buildenv #put quotes around r-values in env listing
^D #(I leave the devshell)
Then copy this to my development directory and source it before running make with all of its options
KERNEL_PATH=/mypathto/build/tmp/sysroots/socfpga_cyclone5/usr/src/kernel
source ./buildenv && make -C $KERNEL_PATH V=1 M=`pwd` \
ARCH=arm CROSS-COMPILE=arm-linux-gnueabihf- \
KERNEL_VERSION=3.13.0-00298-g3c7cbb9 \
CC="arm-linux-gnueabihf-gcc -mno-thumb-interwork -marm" \
LD=arm-linux-gnueabihf-ld AR=arm-linux-gnueabihf-ar
Now to my questions:
Am I going about this completely wrong? What is the recommended way to cross-develop kernel modules? I am doing it this way because I don't want to open a bitbake devshell and do my code development in there every time.
This sort of works (I can compile working modules) but the make script gives me an error message saying that the kernel configuration is invalid. I have also tried this with KERNEL_PATH set to the the kernel package git directory (build/tmp/work///git (which contains what appears to be a valid .config file) and I get a similar error.
How can I extract the env without needing to open a devshell? I would like to write a script that extracts it so my coworkers don't have to do it manually. The devshell command opens a completely separate Xterm window, which rather dampens its scriptability...
the sdk installer is what you are looking for:
bitbake your-image -c populate_sdk
then, from your build directory go to tmp/deploy/sdk
and execute the generated shell script.
this script will allow you to generate and install an sdk.
Not only the sdk will allow you to (cross) compile your kernel by providing the needed environment variables and tools, but it will also provide a sysroot + a standalone toolchain to help you easily (and by easily I mean really easily) crosscompile applications with the autotools (as long as you provide Makefile.am and configure.ac)
just source the environment-setup-* file, got to your kernel directory and compile.
Or, for application developpment based on the autotools,
go to the folder containing your project (sources + Makefile.am and configure.ac)
and do:
libtoolize --automake
aclocal
autoconf
automake -a
now your project is ready for compilation:
./configure $CONFIGURE_FLAGS
make
make install DESTDIR=path/to/destination/dir
If you're after a quick hack, instead of Ayman's more complete solution, the scripts run to complete each build stage are available in the directory:
./build/tmp/work/{target_platform}/{package}/{version}/temp/run.do_{build_stage}
These scripts can be run standalone from the ./temp/ directory, and contain all of the environment variables required.
We are developing android kernel drivers in a team. The drivers are developed as loadable modules and are compiled separately from the kernel source. However we need to first build android kernel source in order to build driver modules. Each developer pull kernel changes from git and builds the kernel binary. Since kernel binaries are different on each developer's local system, we need to replace both kernel binary and driver modules each time. Is there a better solution to build drivers apart from sharing kernel source on a common system?
One of you should do this locally. Assume your kernel source directory is $kern_dir and you will build the below artifacts in a new staging area (outside the kernel source hierarchy). Call this $stage.
Within the staging area, create a subdirectory hierarchy: $stage/lib/modules/$kernel_version/build ($kernel_version should match the KERNEL_VERSION in your kernel Makefile; the resulting path will be the same directory name as you're currently installing modules into on the target system); call this $build.
# Configure and build the kernel as you currently do.
cp <my-kernel-config> to $kern_dir
cd $kern_dir
make oldconfig # or whatever is needed to set your config
make
# Create the staging area and build directory
mkdir -p $build
# Copy updated kernel config to staging area:
cp .config $build
# Create necessary kernel configuration support (in staging area) to support out-of-tree modules
make O=$build modules_prepare
# Add Module.symvers to build directory
cp Module.symvers $build
You should now have a completely configured build directory that everyone can install on their own system. This is a directory containing all the header files needed to build with that kernel, together with the info describing what configuration options your kernel was built with and the Module.symvers file that describes kernel exported symbols.
Bundle that up ("cd $stage; tar czf kernel-devel.tgz lib/modules/$kernel_version/build") and distribute it to all developers.
Separately commit the "package" containing the built kernel and the associated modules (i.e. all the ones that come with the kernel and that you're not modifying); for this, use whatever mechanism you're using now. Have everyone install this kernel and build directory onto their target (run-time) system. (This should give everyone a common /boot/ and /lib/modules hierarchy.)
On their build system, everyone should install your kernel-devel.tgz from above. Assuming that the build system is running the same kernel as the target systems and you installed the kernel-devel.tgz relative to root (i.e. it's actually in /lib/modules/$kernel_version/build on your build system), you can then use the "standard" module Makefile model to build your modules. Should be something like this:
obj-m := hello.o
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
If your build system is not running the same kernel, then simply change KDIR to wherever you've installed the kernel-devel.tgz tarball.
Now anyone on your team should be able to build modules that install against the same kernel.