KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file does not work as expected - linux-kernel

KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file does not work as I expected.
First lesson I learned: defconfig != .config
Bare-Kernel-Build (without yocto): A new defconfig file below arch/arm/config/xy_defconfig created by make savedefconfig is not equal to the .config file. I assume the linux kernel expand some symbols and create the final .config.
Yocto-Build: Here comes the issue: Yocto can not correctly handle the defconfig file below arch/arm/config/xy_defconfig. When building the linux-kernel with yocto.. the kernel is half size and not bootable. This results, because yocto does not expand the missing symbols in .config, which a make defconfig would do.
How do you handle the issue?
Update:
Figured out an additional make xy_defconfig do the trick and creates the correct .config file. It is a ugly hack, any better ideas are very welcome.
Added this to my custom linux-xy.bb file.
KBUILD_DEFCONFIG ?= xy_defconfig
do_makedefconfig() {
oe_runmake -C ${B} ARCH=${ARCH} $KBUILD_DEFCONFIG
}
addtask do_makedefconfig after do_configme before do_compile

First copy your defconfig to sources/meta-mylayer/recipes-kernel/linux
and write a .bbappend file for linux recipe
linux-ti_%.bbappend
FILESEXTRAPATHS_prepend : "${THISDIR}:"
SRC_URI += "file://your-defconfig"
2. open sources/meta-mylayer/conf/machine/your-machine.conf add below line,
search the macro if already present or nor using grep -inr "INTREE_DEFCONFIG"
INTREE_DEFCONFIG_pn-linux-ti = "am335x_phytec_defconfig"
if the macro already present in the same file replace defconfig name.

kernel-yocto approach
The KBUILD_DEFCONFIG variable is handled by kernel-yocto class, so you need to inherit it. It is usually done indirectly through include file linux-yocto.inc, so you should have this line you linux recipe:
require recipes-kernel/linux/linux-yocto.inc
Please note that KBUILD_DEFCONFIG_KMACHINE consists of two parts. The first one is variable name (KBUILD_DEFCONFIG) and the second one is kernel machine override (KMACHINE). So you need to change the override to fit your machine. That's why the KMACHINE part is written in italics in the documentation [1]. There is an example for RPi from the documentation:
KBUILD_DEFCONFIG_raspberrypi2 = "bcm2709_defconfig"
The KMACHINE is set in linux-yocto.inc to the MACHINE variable by default.
handle in-tree defconfig by hand
We are using in-tree defconfig without the kernel-yocto class. linux.inc from meta-oe layer is used. I don't know if this is a best practice. Here is our linux recipe (it is reduced to a bare minimum):
require recipes-kernel/linux/linux.inc
PV = "your_version"
SRC_URI = "your_sources"
do_configure_prepend() {
bbnote "Copying defconfig"
cp ${S}/arch/${ARCH}/configs/YOUR_defconfig ${WORKDIR}/defconfig
}
[1] https://www.yoctoproject.org/docs/2.4/mega-manual/mega-manual.html#var-KBUILD_DEFCONFIG

Related

How to generate a list of source files compiled for a particular board configuration in u-boot sources?

U-boot being a bootloader targeted at different architecture and SoC's, there are several source files, and only some of them makes it to the final executable for a particular board. For example, in the arch/ directory, there is one directory per architecture. If the build is for an ARM architecture SoC, only some of the files in arch/arm/ will be compiled into the executable.
Which of the source files get compiled into the executable depends on the configuration of the build. This configuration is controlled by a file present in configs/ directory. In case of BeagleBone Black, this file is configs/am335x_boneblack_defconfig. This file defines several variables, which are used in Makefiles.
A part of the configs/am335x_defconfig is shown below:
CONFIG_ARM=y
CONFIG_ARCH_OMAP2PLUS=y
CONFIG_TI_COMMON_CMD_OPTIONS=y
CONFIG_AM33XX=y
CONFIG_DISTRO_DEFAULTS=y
The variables defined in these files are later referenced in other Makefiles. For example, the CONFIG_AM33XX variable defined above is referenced in /arch/arm/mach-omap2/am33xx/Makefile as shown below:
obj-$(CONFIG_AM33XX) += clock_am33xx.o
obj-$(CONFIG_TI814X) += clock_ti814x.o
obj-$(CONFIG_AM43XX) += clock_am43xx.o
ifneq ($(CONFIG_AM43XX)$(CONFIG_AM33XX),)
obj-y += clock.o
endif
I guess while executing the command
$make am335x_boneblack_defconfig
the value of the variable CONFIG_AM33XX gets replaced in some transient copy of the Makefile, so that the content of the above makefile gets replaced by
obj-y += clock_am33xx.o
Figuring out which source files are included in the final executable just by searching for the config variables in all the Makefiles is a tedious task.
I need a way to create a list of source files which gets compiled for a particular config file automatically. Is there a way to do it?
Just do a clean build and search for all object files:
make mrproper
make foo_defconfig
make -j6
find . -name '*.o'
You probably want to exclude directories tools/ and scripts/.

What exactly does Linux kernel's `make defconfig` do?

I can use the following command to create a Linux kernel .config file based on a specified architecture default for a custom ARM-based board:
ARCH=arm make defconfig KBUILD_DEFCONFIG=var_som_mx6_android_defconfig
I thought that this command more or less copies ./arch/arm/configs/var_som_mx6_android_defconfig to ./.config. However the resulting .config file isn't exactly a copy:
$ diff --unified arch/arm/configs/var_som_mx6_android_defconfig .config
--- arch/arm/configs/var_som_mx6_android_defconfig 2017-01-20 12:10:51.891515984 -0800
+++ .config 2017-01-26 15:31:29.000000000 -0800
## -407,6 +407,7 ##
CONFIG_ARM_ERRATA_751472=y
CONFIG_ARM_ERRATA_794072=y
CONFIG_ARM_ERRATA_761320=y
+CONFIG_ARM_ERRATA_845369=y
# CONFIG_ARM_ERRATA_753970 is not set
CONFIG_ARM_ERRATA_754322=y
# CONFIG_ARM_ERRATA_754327 is not set
## -2683,7 +2684,6 ##
CONFIG_AUTOFS4_FS=y
CONFIG_FUSE_FS=y
# CONFIG_CUSE is not set
-CONFIG_AUFS_FS=y
#
# Caches
## -2759,6 +2759,21 ##
# CONFIG_PSTORE is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
+CONFIG_AUFS_FS=y
+CONFIG_AUFS_BRANCH_MAX_127=y
+# CONFIG_AUFS_BRANCH_MAX_511 is not set
+# CONFIG_AUFS_BRANCH_MAX_1023 is not set
+# CONFIG_AUFS_BRANCH_MAX_32767 is not set
+CONFIG_AUFS_SBILIST=y
+# CONFIG_AUFS_HNOTIFY is not set
+# CONFIG_AUFS_RDU is not set
+# CONFIG_AUFS_PROC_MAP is not set
+# CONFIG_AUFS_SP_IATTR is not set
+# CONFIG_AUFS_SHWH is not set
+# CONFIG_AUFS_BR_RAMFS is not set
+# CONFIG_AUFS_BR_FUSE is not set
+CONFIG_AUFS_BDEV_LOOP=y
+# CONFIG_AUFS_DEBUG is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
I don't understand where the extra lines are coming from, and I have always found the internal workings of the kernel configuration, makefiles, and build scripts to be difficult to understand. Can anyone explain where those lines in the .config might be coming from?
Motivation
The .config file is not simply copied from your defconfig file. The motivation for storing defconfig in such a format is next: in defconfig we can specify only options with non-default values (i.e. options we changed for our board). This way we can keep it small and clear. Every new kernel version brings a bunch of new options, and this way we don't need to update our defconfig file each time the kernel releases. Also, it should be mentioned that kernel build system keeps very specific order of options in defconfig file, so it's better to avoid modifying it by hand. Instead you should use make savedefconfig rule.
Simplified explanation
When .config file is being generated, kernel build system goes through all Kconfig files (from all subdirs), checking all options in those Kconfig files:
if option is mentioned in defconfig, build system puts that option into .config with value chosen in defconfig
if option isn't mentioned in defconfig, build system puts that option into .config using its default value, specified in corresponding Kconfig
Check scripts/kconfig/Makefile and scripts/kconfig/conf.c files to see how it's actually done.
More precise and detailed explanation
From "Kbuild: the Linux Kernel Build System" by Javier Martinez:
Defining Configuration Symbols: Kconfig Files
Configuration symbols are defined in files known as Kconfig files. Each Kconfig file can describe an arbitrary number of symbols and can also include (source) other Kconfig files. Compilation targets that construct configuration menus of kernel compile options, such as make menuconfig, read these files to build the tree-like structure. Every directory in the kernel has one Kconfig that includes the Kconfig files of its subdirectories. On top of the kernel source code directory, there is a Kconfig file that is the root of the options tree. The menuconfig (scripts/kconfig/mconf), gconfig (scripts/kconfig/gconf) and other compile targets invoke programs that start at this root Kconfig and recursively read the Kconfig files located in each subdirectory to build their menus. Which subdirectory to visit also is defined in each Kconfig file and also depends on the config symbol values chosen by the user.
Storing Symbol Values: .config File
All config symbol values are saved in a special file called .config. Every time you want to change a kernel compile configuration, you execute a make target, such as menuconfig or xconfig. These read the Kconfig files to create the menus and update the config symbols' values using the values defined in the .config file. Additionally, these tools update the .config file with the new options you chose and also can generate one if it didn't exist before.
Because the .config file is plain text, you also can change it without needing any specialized tool. It is very convenient for saving and restoring previous kernel compilation configurations as well.
Useful commands
You can use simpler syntax for make defconfig, like:
$ make ARCH=arm your_board_defconfig
See the full list of available defconfigs with:
$ make ARCH=arm help | grep defconfig
If you need to do reverse action (i.e. create a neat small defconfig from extensive .config), you can use savedefconfig rule:
$ make ARCH=arm savedefconfig
Also, as 0andriy mentioned, you can use diffconfig script to see changes from one .config to another one:
$ scripts/diffconfig .config_old .config_new
It also generates include/generated/autoconf.h.
This header file is included by C source files. On the other hand, .config is for the Makefile system.
The build system generates two files, and keeps them consistent.

how to run make for a local subdirectory

I am trying to port an existing code into a larger project. The larger project has a main Makefile with Makefiles in each sub-directory. I am sure the path below tells you all about how it is setup. I want to port my code to
/WORKING_DIRECTORY/Drivers/Char/example
And here is the content:
sansari#ubuntu:~/WORKING_DIRECTORY/drivers/char/examples$ ls
hello1.c Makefile
My question is 1- Should modify this local Makefile or the main one? I am setting up to modify this one, but I am not sure.
2- My other question is if I modify this local file, can I just run make from here and validate my configuration instead of running make for the entire project? I know that make only updates the files that are changed; however I feel better when I clean the build environment before each make. I have run into situations, which this alone fixed my issue.
Just as background, I did try to include the make file of my target project, the one I am trying to import here with -f command. What I did was: make -f Makefile -f ../mytarget/core/Makefile
But I ran into some issues with make not doing some of the normal things it does in the primary project. For instance, there was an include statement with a relative path to a header file, which make gave me an error about not seeing it. So now I am abandoning that strategy for the time being.
#Ahmad Masoud - Hey man, thanks. Here is the Makefile. Hey man, the link is exactly what I needed. I think it will address my other questions also. You see, I cross compiled this code, and when I flashed my phone, I get the following for
uname -r: 1|root#hltespr:/lib/modules # uname -r
uname -r
3.4.0-g7e6fbf7-dirty
And I have been wondering what "dirty" means and where it comes from. If you know please tell me. The link you sent, states that perhaps make would insert the Linux kernel version there? I ask this since, modprobe does not work when I try to load my module. Instead insmod works, and I can validate that my module is in. My main issue now is that I don't know how to execute the file to make sure it runs. I only know how to run the file using modprobe, and I can not use it. It gives me the following error:
1|root#hltespr:/ # modprobe /lib/modules/hello1.ko
modprobe /lib/modules/hello1.ko
modprobe: can't change directory to '3.4.0-g7e6fbf7-dirty': No such file or directory
Update as of 06/20/15 -I put in include /home/sansari/mytree2/tbt/makefile in my module's make file. I get the following error: makefile:3: *** missing separator. Stop.
#Ahmad - This is an update as of 062415. Thanks for the info. My goal is to get make to look into this external directory, collect all the source files and build them for me. What would you suggest? I am stuck because as it stands, I know make looks into my examples directory, but no other changes I make to that local make file in the examples directory shows up in make. For instance I tried adding ($warning ....) and #echo messages, but even they do not show up.
Update on 070215- Thanks for the previous comments and support. I feel I really should reopen this thread since I did not explain the goal in detail, and now I feel I can describe it better, and hopefully the resolution will help other. I issue the command:
TARGET=msm8974 PLATFORM=msm8974 make drivers/char/examples
But I get a message stating: Nothing to be done, while I have added a number of tasks to do. Below is my make file, and I'll elaborate on what I have added right after:
lib_tbt := ../../../m/shahin/tbt
lib_daemon := ../../../m/shahin/daemon
lib_lib := ../../../m/shahin/lib
lib_tasks := ../../../m/shahin/tasks
lib_tbt_driver := ../../../m/shahin/tbt_driver
lib_tbt_make := ../../../m/shahin/tbt/make
lib_tbt_msm_common := ../../../m/shahin/tbt/platform/msm8974/common
lib_tbt_msm8974 := ../../../m/shahin/tbt/platform/msm8974
lib_asm_generic = ../../../m/shahin/tbt/platform/msm8974/include/asm-generic
$(warning This is what is in lib_asm_generic $(lib_asm_generic))
#include $(lib_tbt_make)/macros.mk
.PHONY: all $(lib_tbt)
$(lib_tbt) $(lib_daemon) $(lib_lib) $(lib_tasks) $(lib_tbt_driver) $(lib_tbt_make) $(lib_tbt_msm_common) $(lib_tbt_msm8974) $(lib_asmgeneric) :
$(MAKE) --directory=$#
$(lib_*): $(MAKE) --directory=$#
obj-$(CONFIG_EXAMPLES) += hello1.o
_
Initially I only had the obj-$(CONFIG_EXAMPLES) += hello1.ostatement in my make file. I then proceeded to add the directory variables at the top of my makefile and added the $(lib_*): $(MAKE) --directory=$# directing make to compile what is in the directory. I believe that is what it does. Please let me know if I am mistaken. And although this same make file proceeds to create object files when I put it a different directory within my project, it won't do so when it is in a device driver directory. And I do not understand why. The other directory is the /external directory and it is at the top of the tree. But that should not matter right. What I have done was to first make sure I can compile a hello program in my device driver directory called /examples. I now want to add more source code to this section. I believe the correct term is module? I also want to know if I should copy of the source files to the /examples directory or referencing them via the path is ok. That is should I move the source code directory under /examples directory or not?
It is a LOT simpler than that if you are using a kernel that uses Kbuild.
Highly recommend reading
http://www.tldp.org/LDP/lkmpg/2.6/html/x181.html
Situation A - Your source is a sub-tree of the kernel source
You would NOT modify the top-level Makefile, just ensure that ~/WORKING_DIRECTORY/drivers/char/examples/Makefile and ~/WORKING_DIRECTORY/drivers/char/examples/KBuild are set up correctly/normally. THEN at the top-level of the kernel build directory (assuming you have a separate build directory) you would type:
foo#bar:~/build-dir$ make drivers/char/examples
The kernel top-level makefile then builds just that sub-tree. You can try it out on any part of the kernel, for example:
foo#bar:~/build-dir$ make fs
NOTE: build-dir can be the same as the kernel source directory
Situation B - You are building an external module
Then use the normal module KBuild / Makefile process.
P.S.
If you post your makefile / Kbuild then I may be able to help with the actual build processing.

understanding data assigned to macros in a makefile

I have searched through this forum but am not able to find an answer to this question, still if I have missed it please excuse me and direct me to the same.
I am trying to understand makefiles and came across the makefile for the tcpreplay utility on Linux. There are lot of macros that have been defined with the value starting an ending in a #. What are these values, how are they used? A snippet:
ACLOCAL = #ACLOCAL#
AMTAR = #AMTAR#
AR = #AR#
AUTOCONF = #AUTOCONF#
AUTOGEN = #AUTOGEN#
AUTOHEADER = #AUTOHEADER#
This is a makefile template, likely for software built with a GNU configure script. When configure is run, the #NAME# placeholders are replaced with proper values as determined at runtime. E.g. #AR# will be the name (or path) of the archiver, /usr/bin/ar. You then have a proper Makefile that you can run with a make invokation. If an actual Makefile still contains #NAME# placeholders, there was an error in running configure.
You are very likely not looking at a file named Makefile but one named Makefile.in. The .in suffix indicating that this is input to configure.
You can find all the gory details in the GNU autoconf manual.

Depend on the make file itself

In the event that a Makefile itself is changed, a safe bet would be to consider all targets out of date.
Is there a clever way to add this dependency? Are there any alternatives?
Make sure the object files depend on the makefile:
$(OBJFILES) : Makefile
Where Makefile is the name of the make file.
A safe bet, but a terrible idea. Example: you're using automake and update Makefile.am to add a single source file. The correct response is to compile just the new file and link it in. In your scheme everything would be rebuilt.
Moreover, adding the dependency isn't going to do anything unless you touch the file, something like:
$(SRCS): Makefile
touch $#
This will then trip up editors that use the mtime to detect concurrent modification (emacs is one example).
If you're doing something major, just run make clean all after doing the change.
Since GNU make version 4.3 it is now possible with the use of those two special variable:
.EXTRA_PREREQS
To add new prerequisite to every target
MAKEFILE_LIST
To get the path of the make file
To have every target depend on the current make file:
Put near the top of the file (before any include since it would affect the MAKEFILE_LIST) the following line:
.EXTRA_PREREQS:= $(abspath $(lastword $(MAKEFILE_LIST)))
To have every target depend on the current make file and also the make files which were included
Put the following line at the end of your file:
.EXTRA_PREREQS+=$(foreach mk, ${MAKEFILE_LIST},$(abspath ${mk}))

Resources