Building out-of-tree kernel modules, depmod - and, why reboot? - linux-kernel

I've been working with a userland program writeread.c (referred here), utilizing the ftdi_sio Linux driver.
I had worked with the default ftdi_sio that comes with Ubuntu Lucid that I use - and then, as a sanity check, decided to rebuild ftdi_sio from source. I did not want to download the entire kernel source, so I opted for so called "building out-of-tree modules":
Howto: Build Linux Kernel Module Against Installed Kernel w/o Full Kernel Source Tree
Debian Linux Kernel Handbook - Common kernel-related tasks: Building out-of-tree kernel modules
Building a custom kernel - FedoraProject: Building Only Kernel Modules (Out Of Tree Modules)
... which means you can only have installed linux-headers-* packages (instead of having entire linux-source-* installed) - and the respective kernel module source - in order to build a kernel module.
So, I basically got the following files in a folder:
ftdi_sio.c
ftdi_sio.h
ftdi_sio_ids.h
Makefile (included below)
and ran make - and a ftdi_sio.ko kernel objects compiles, which also can be insmod-ed without a problem.
The problem is now here - as long as I work with low speeds (115200 bps), there is no observable difference between: loading the 'vanilla' driver (using sudo modprobe ftdi_sio); and loading the newly built driver (using sudo modprobe usbserial; sudo insmod /path/to/ftdi_sio.ko) - in respect to the performance of the userland program (writeread.c).
However, if I try the same at 2000000 bps, then the 'vanilla' driver works fine - while the build driver causes a segfault of the userland program (writeread.c). In particular, this segfault happened at free()-ing memory resources - because the driver would end up reading more bytes from the loopback connection, than it itself would write! However, that in itself is not that relevant for this discussion, because I finally found a way to get the built ftdi_sio to work (Edit: just noticed that a similar procedure is also given in External Linux kernel module dependencies R#2238581 - Stack Overflow):
$ sudo modprobe -r ftdi_sio # remove 'vanilla' driver from memory
...
# build the driver
$ cd /path/to/ftdi_sio_2.6.32/
$ make clean && make
# check vanilla version
$ ls -la /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/ftdi_sio.ko
-rw-r--r-- 1 root root 102396 2010-10-17 01:47 /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/ftdi_sio.ko
# (re)move the 'vanilla' kernel object
$ sudo mv /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/ftdi_sio.ko ~/Desktop/ftdi_sio_orig.ko
# symlink our built version, as if it is the 'vanilla' one
$ sudo ln -s /path/to/ftdi_sio_2.6.32/ftdi_sio.ko /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/
# check
$ ls -la /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/ftdi_sio.ko
lrwxrwxrwx 1 root root 57 2010-10-26 12:49 /lib/modules/2.6.32-25-generic/kernel/drivers/usb/serial/ftdi_sio.ko -> /path/to/ftdi_sio_2.6.32/ftdi_sio.ko
# call depmod
$ sudo depmod
# !! plug in USB at this point, built driver autoloads;
# however, running `writeread.c` will segfault!
# !! REBOOT at this point
# after reboot:
# !! plug in USB at this point, built driver autoloads;
# and `writeread.c` runs without segfault
So, my question is: can anyone clarify to me, why - even after running depmod - do I still have to reboot, to have the built driver behave properly? In other words: is there any other command I could call (maybe after depmod), so that the driver works properly when it's loaded, without rebooting?
The reason I'm asking, is that I suspect that upon each recompilation of the kernel module, the symbol table will change - so to ensure that it works properly, I'd have to run depmod and reboot after each time the driver is built - which I think would be kind of tedious :) So, it would be nice to be able to rebuild the kernel module, and use it in the same session - without having to reboot.
Well, thanks in advance for any answers,
Cheers!!
Makefile:
CONFIG_MODULE_FORCE_UNLOAD=y
# flags passed to gcc for compilation - -v:verbose, -H:show include files
# KBUILD_CFLAGS += -v
# for debugging make itself, use --debug=i in make command for targets
# debug build:
EXTRA_CFLAGS=-g -O0
KVERSION = $(shell uname -r)
obj-m += ftdi_sio.o
all:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

In short, no, I can't see anything wrong with the sequence you're doing, or why it would cause that behaviour.
The fact that the kernel allows the newly built module to be inserted basically means that you've built it properly and it has also inserted correctly, so there are no version mismatches or anything.
I'll guess that when you run the test at 2000000 bps, you first tested the vanilla driver and then the new driver? So my guess is the driver is not cleaning up something correctly, so when you run the test again (with the new driver) you see a segfault. But like I said, that's a guess.

Related

Modifications not taken into account when recompiling linux kernel sources

I want to use the writeprotect mode of the userfaultfd feature, but it is implemented only from Linux 5.11.
I have ubuntu 20 with kernel 5.4 so I installed version 5.11 of Linux from sources, but when I reboot on the installed version, and try to use the writeprotect mode in a c user program the macros (e.g., UFFDIO_WRITEPROTECT) still appear not to be defined. And I verified that userfaultfd is not a module (doing lsmod).
In the other hand, if I make a modification for example in the scheduler just for a test (let's say in kernel/sched/core.c:context_switch) the modification are taken into account because this is in-core, but modifications for user space are not, I'm confused.
The userfaultfd.c file is in the fs/ directory of the kernel.
So please is it something that I'm missing in the compilation process (make menuconfig, make, make modules_install, make) ?
I was wrongly interpreting the error I had.
The modifications are indeed taken into account when I boot on the newly compiled kernel.
But while searching an understanding of what happens when booting on a compiled kernel, I got this:
When booting on a kernel compiled from sources, the libraries in /usr/include are not modified on the disk
If you want to overwrite them by the one modified, you should do this:
make deb-pkg //from the kernel sources directory
cd ..
sudo dpkg -i *.deb
This will install the new kernel and overwrite the headers

Compile gpio-mockup in 5.4

I've checked out kernel 5.4 source and tried to compile gpio-mockup.c by running make -C /lib/modules/$(uname -r)/build M=$(pwd) modules the build succeeds but there is no gpio-mockup.ko. I think this means that I need to provide some configuration value but I don't know how and I don't know where. Regardless, can anyone help me to build the gpio-mockup kernel module?
Because the gpio-mockup module depends on some GPL symbols in the kernel it needs to be built with the kernel. This means you must build a whole new kernel.
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux
cd linux
git checkout v5.4
cp /usr/lib/modules/$(uname -r)/build/.config ./
yes '' | make oldconfig
echo "CONFIG_GPIO_MOCKUP=m" >> .config
make -j $(nproc) deb-pkg LOCALVERSION=-gpio-mockup
This will produce Debian packages that can then be used to install a new kernel that will include the gpio-mockup kernel module.
NOTE: This is a vanilla kernel which may or may not have kernel modules your distribution usually provides (like ZFS on Ubuntu). Other steps may be necessary to get a custom kernel like your distribution provides.
You should enable CONFIG_IRQ_SIM=y, since gpio-mockup.ko is depended irq_sim.
please look at this picture

Using populate_sdk to include kernel headers

How do I include the linux kernel headers as part of the SDK package in Yocto?
I'm using Yocto 1.8 (fido) in an embedded project and want to do out-of-tree kernel module development. Currently, I can build my kernel modules (apart from bitbake) by pointing my $KERNEL_PATH to the poky/build/tmp/work-shared/<machine>/kernel-source/ directory when I run make. I don't want to do this in the long term, however, since others need to easily build the modules without installing and building a full image from bitbake.
I can generate an SDK using bitbake myimage -c populate_sdk. However, this doesn't include the kernel headers (all I ever see is sysroots/<mach>/usr/include/linux). How do I cause the kernel headers to be included in the SDK package? Also, I don't want the kernel headers to show up as part of my target image.
[Edit]
My image recipe is as follows:
EXTRA_IMAGE_FEATURES_append = " eclipse-debug debug-tweaks"
TOOLCHAIN_HOST_TASK_append = " nativesdk-cmake"
IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL} util-linux kernel-modules netbase busybox base-passwd base-files sysvinit initscripts bash gdbserver strace sysfsutils dtc gawk ethtool grep sed wget iptables oprofile net-tools dropbear rsync stress-ng rt-tests i2c-tools"
inherit core-image
The kernel I'm using is linux-altera-ltsi-rt in the meta-altera layer.
From the fido release, the handling of kernel builds has been changed. In previous releases, you could usually just skip down to the usage example below.
In fido or any 1.8+, if you want the kernel src and build system available in the SDK, you should add
TOOLCHAIN_TARGET_TASK_append = " kernel-devsrc"
to your image recipe. That will ensure that the new, kernel-devsrc package is installed into your toolchain.
The procedure below is just to ensure that the rest of the workflow is fully understood (even though it's not strictly part of the original question).
Usage Example
Lets assume a module Makefile as follows:
obj-m += hello-1.o
all:
make -C $(KERNEL_SRC) M=$(PWD) modules
clean:
make -C $(KERNEL_SRC) M=$(PWD) clean
Example taken from The Linux Kernel Module Programming Guide (Note that the actual commands needs to have a tab character for indentation).
Then you'll have to define KERNEL_SRC to be sysroots/<mach>/usr/src/kernel/, either in the Makefile, or from your make call. (Using a variable like KERNEL_SRC will ensure that your module recipe automatically picks the right location when building using bitbake).
To manually build your kernel module:
Source the environment-* file for your SDK.
Go to you modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make However, this will fail, as fixdep can't be found. We'll fix this manually.
cd <sdk-install-path>/sysroots/<mach>/usr/src/kernel
make modules_prepare
If this needs to be run with sudo, be sure to source the environment file in the sudo environment: sudo bash -c "source <sdk-install-path>/environment-setup-<mach> && make modules_prepare"
Go back to your modules directory.
KERNEL_SRC=<sdk-install-path>/sysroots/<mach>/usr/src/kernel LDFLAGS="" make
This should now allow you to build your modules.
If you don't have the kernel source under sysroots/<mach>/usr/src/kernel/, we'll have to look into that.
anders answer is very good, but in recent versions of yocto the way to add kernel-devsrc seems to be
IMAGE_INSTALL += "kernel-devsrc"
which I found here: https://www.mail-archive.com/yocto#yoctoproject.org/msg36448.html
With Yocto Zeus (3.0.x) add this to your image recipe:
TOOLCHAIN_TARGET_TASK += "kernel-devsrc"
EDIT: Same for Gatesgarth (3.2.x) but the make scripts command has a new host dependency to libyaml-dev

Reliable build of the Linux Kernel for the BeagleBone Black

Is there a reliable build of the Linux kernel for the BeagleBone Black platform anywhere ? I've followed the instructions here, as well as numerous other places, and none of them work. Either the modules don't build (the .ko files just don't get built and can't be found in the case of the link above), or there's missing build targets for the kernel. I'm using the Linaro ARM compiler (arm-linux-gnueabihf-gcc) cross compiling on 32-bit x86 Ubuntu 14.04.
Specifically, I've checked out the source at the BeagleBone Git repository, branches 3.8, 3.11, 3.12, 3.13 and 3.14. I've also checked out a 3rd party git clone here. The problems I've encountered are as follows:
3.8: Kernel will compile (with numerous warnings), but the modules fail to compile due to errors in the patching process from running patch.sh in the branch.
3.11: Kernel will compile (with numerous warnings), but the uImage-dtb.am335x-boneblack Make target will not. Compiling modules fails due to at least one file having an incorrect patch that yields incorrect C syntax.
3.12: Kernel will compile (with numerous warnings), along with the modules. Copying kernel and modules to the board succeeds. Booting the kernel fails, and freezes as soon as the bootup begins right out of the boot command in U-Boot.
3.13, 3.14: The kernel compilation fails with numerous syntax errors.
3.8.13+ from TowerTech: The kernel compilation succeeds (with numerous warnings), the module compilation fails, again due to incorrect patching from patch.sh.
In the cases where I could compile a kernel, I copied it to a fresh, vanilla board in memory using a TFTP boot server from U-Boot, and booted the kernel in memory. However, none of the successfully compiled kernels could work properly because they were missing their modules / firmware / dtbs. What could be causing the incorrect patching ? Surely, the developers who've written the code and patches must have tested it, so there may be something wrong with my workspace, though I can't imagine what at this point. The commands I used were the same as those at link 1.
I also followed the link you have mentioned and I am able to boot the BBB. I am using the mainline kernel from https://www.kernel.org/. The following are the steps I followed
Download the latest kernel. My version is 3.18-rc4
Use "omap2plus_defconfig" which is the default config for beaglebone black
Make commands are
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- omap2plus_defconfig -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- uImage dtbs LOADADDR=0x80008000 -j4.
The image produced is present in the following path.
arch/arm/boot/uImage uImage
arch/arm/boot/dts/am335x-boneblack.dtb
After this compile the modules.
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules -j4
I was using busybox and NFS server as root
Compiler used is arm-linux-gnueabi-gcc. With this I am able to boot successfully and run modules. It has the advantage that we are using the latest kernel available.
I was finally able to boot the BBB with a new kernel 3.14
I am on a beagleboneblack rev C with debian pre-installed.
I basically followed steps here
http://dev.ardupilot.com/wiki/building-for-beaglebone-black-on-linux/
Kernel download: (I didn't use git, limited bandwidth...)
https://github.com/beagleboard/linux/archive/3.14.tar.gz
Build (done on a Ubuntu 14.04) : (I am not sure if some steps are not relevant or redundant)
Install mkimage
sudo apt-get install libssl-dev
wget ftp://ftp.denx.de/pub/u-boot/u-boot-latest.tar.bz2
tar -xjf u-boot-latest.tar.bz2
cd u-boot-2014.10/
make sandbox_defconfig tools-only
sudo install tools/mkimage /usr/local/bin
Download am335x-pm-firmware.bin from http://arago-project.org/git/projects/?p=am33x-cm3.git;a=tree;f=bin;h=75a5de7aa94ff6ccbfb1b3b9dc80bc2fe5b423bf;hb=refs/heads/master and copy to kernel directory
Build the kernel
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- bb.org_defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- uImage dtbs LOADADDR=0x80008000 -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- modules -j4
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- am335x-boneblack.dtb -j4
Install Modules
mkdir -p ../export/rootfs
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- INSTALL_MOD_PATH=../export/rootfs/ modules_install
Now copy modules to /lib/modules/ directory in your beaglebone black (create a new directory for the modules)
cd ../export/rootfs/lib/modules/
rsync -avz 3.14.25/ root#192.168.1.3:/lib/modules/3.14.25/
Next is to copy the zImage and am335x-boneblack.dtb to the BBB
copy zImage from arch/arm/boot to /boot/uboot/ of BBB
copy am335x-boneblack.dtb from arch/arm/boot/dts to /boot/uboot/dtbs
If you replace the existing zImage and am335x-boneblack.dtb with new ones directly and they don't work, you will be in trouble. I stored the new images in a folder in /boot/uboot and manipulated environment variables in uboot to pick the one i wanted.
I know, I could have used tftpserver..
I hope it helps someone. I spent a lot of time on this which should have been very straightforward :(
Yeah, the documentation is really out of date (circa 2011 or early 2012) and very confusing. First, the github/beagleboard/kernel area says it is deprecated and to use beagleboard/linux for the kernel. BUT, it is still be actively used 1) to add patches that have not been accepted to the kernel tree yet and 2) to add kernel configs specifically for the beaglebone devel builds. The bb.org_defconfig in the kernel tree does not turn on some debug stuff.
So.. both github/beagleboard/linux AND github/beagleboard/kernel work for me (3.14) on a BBB. If you want a standard image, use "linux"; if you want a bells-and-whistles image use "kernel".
Also, a more recent cross-compiler is in APT at gcc-arm-linux-gnueabihf. Make sure you have the "hf" (hard floating point unit) version. No need to depend on linaro.
I'll update the docs after I get the project rolling....

Linux wifi backports cross compile

I am trying to backport Linux wifi drivers from 3.11 to linux-omap-l1(2.6.39). I set up the cross compiler on my Ubuntu 11.4 (kernel is 2.6.38-11-generic) and verified that the cross-compiled kernel can run on an ARM processor as expected. I also was able to backport the wifi drivers to this Ubuntu by following the "Usage Guide" section. However, I have problem to cross compile the wifi backports. Here are two attempts I did:
Attempt#1
Following the "Cross compiling" section from the same link above, I set ARCH(=arm), CROSS_COMPILER, KLIB_BUILD(=linux-omap-l1 directory) and KLIB(=linux-omap-l1 directory), ran make defconfig-wifi, then ran make -j4. It finished without any error. However when I check the generated ko files, they are apparently not cross compiled because they look like "rt2x00usb.ko: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped"
Attempt#2
Then I cleaned the old make result and passed the above four macros directly to make as arguments and ran it again. This time it reported many errors complaining of missing printk.h, atomic.h and average.h
Please help with either of the cases above.
Well, I here post the rt2800usb wifi solution for other's reference. The following steps cross-compiled successfully:
Menuconfig linux-omap-l1(2.6.39) kernel to disable Networking Support -> Wireless, build it and boot it to the embedded unit to ensure the readiness
Run “make mrproper” from the backports directory on the build pc (Ubuntu 11.4)
Run the following sript from the build pc. Ensure the KLIB pointed directory pre-exists, otherwise create it
set -a
CROSS_COMPILE="/opt/arm-2009q1/bin/arm-none-linux-gnueabi-"
ARCH=arm
KLIB_BUILD="/home/xxx/linux-omap-l1"
KLIB="/home/xxx/linux-omap-l1/updates"
set +a
make defconfig-wifi
make oldconfig # menuconfig worked here too
make
make install
Copy the compiled binaries and modules into the unit(10.11.2.3)
scp -r /home/xxx/linux-omap-l1/updates/lib/modules/2.6.39.4-00537-g6c21e4a root#10.11.2.3:/lib/modules/2.6.39.4/.
Log into the embedded unit and run “modprobe -v -f rt2800usb”
Download and install rt2800usb firmware to the /lib/firmware if necessary
lsmod to verify the driver can be loaded
insert the usb dongle (the udev rules may need to be created or modified)
use iw to further examine its properties
backports is unable to pickup my kernel config.....mac80211 is set to module in my kernel config, when I search for mac80211 in backport's menu config, it shows disabled. Tried the above....
set -a
CROSS_COMPILE="/opt/freescale/usr/local/gcc-4.4.4-glibc-2.11.1-multilib-1.0/arm-fsl-linux-gnueabi/bin/arm-none-linux-gnueabi-"
ARCH=arm
KLIB_BUILD="/home/xxx/rpm/BUILD/linux"
KLIB="/home/xxx/rootfs/lib/modules/2.6.35.3-998-ga1cd8a7/updates"
set +a
make defconfig-wifi
make menuconfig
I found this to be sucessfull...
make defconfig-wifi CROSS_COMPILE=/home/xxx/Downloads/gcc-linaro-arm-linux-gnueabi-2012.04-20120426_linux/bin/arm-linux-gnueabi- ARCH=arm KLIB_BUILD=/home/xxx/Downloads/linux-3.10/ KLIB=/home/xxx/Downloads/rootfs/
make menuconfig CROSS_COMPILE=/home/xxx/Downloads/gcc-linaro-arm-linux-gnueabi-2012.04-20120426_linux/bin/arm-linux-gnueabi- ARCH=arm KLIB_BUILD=/home/xxx/Downloads/linux-3.10/ KLIB=/home/xxx/Downloads/rootfs/
make CROSS_COMPILE=/home/xxx/Downloads/gcc-linaro-arm-linux-gnueabi-2012.04-20120426_linux/bin/arm-linux-gnueabi- ARCH=arm KLIB_BUILD=/home/xxx/Downloads/linux-3.10/ KLIB=/home/xxx/Downloads/rootfs/

Resources