For example in a videa about u-boot, https://www.youtube.com/watch?v=INWghYZH3hI, near time 43:01, I see the lecturer gives u-boot the kernel address and fdt address but not the initramfs address. (bootz 0x80000000 - 0x80800000) but linux boots to the login prompt and he can log in.
How come this is possible? I understand after the kernel boots it starts init process in the initramfs.(I forgot there were a precedence). Without initramfs, how is it possible to run login process or shell?
(it's related to programming so I ask it here. If requested I can move it to unix stackexchange. Is there a method of moving a question to somerewhere else automatically? guess not..)
this is what I learned from the comments with combination with my previous knowledge.
you can embed the initramfs.tar.gz file in the kernel binary image using CONFIG_INITRAMFS_SOURCE=nitramfs.cpio.gz in the configuration. (menuconfig). The initramfs image is placed at the end of the kernel image maybe (I remember).
But this was not the case in the youtube video I mentioned in my question. Near 40:29 in the video, the kernel boot command is shown to have "root=/dev/mmcblk0p1 rootfstype=ext4 rootwait console=tty0e,115200". So it's telling the kernel to use SD card 0's partition 1 as the root system after boot, instead of using initramfs.(when you want to use initramfs, you specify root=/dev/ram and pass the initramfs location. in qemu you use -initrd initramfs.cpio.gz option, or in real machine this information is passed through the device tree to the kernel, in the chosen node's initrd-start and initrd-end address.).
Related
I'm creating a bootable Linux system on a PicoZed board (ARM CortexA9 core), and I've run into a "limitation", which I don't think really is a limitation (I get the feeling it's another problem masquerading as a limitation).
I boot by starting the system in JTAG boot mode; after powering on the board I use the xmd debugger to place u-boot into the system's RAM and then I run it.
Next I place the kernel (uImage), the gzip'd initramfs image and the device tree into memory. Finally I tell u-boot to boot the system using bootm, and using three arguments to point out the memory locations of all the images.
All of this works, and I manage to boot up Linux + userland. However, I need to grow the initramfs, and this is where I run into problems.
The working image is 16MiB exactly. I tried to make it 24MiB (completely regenerated from scratch each time I try to boot), but just after the kernel has loaded and it tries to find init the kernel reports file system faults and fails. There shouldn't be any overlaps, but just in case I tried moving things around a little, but the same problem occurred.
After searching for some tips, I saw someone on a forum say that the image needs to be placed at a 16MiB alignment (which I don't think is true, but I tried it none the less, but it didn't get a working system). Another post claimed that the images must be aligned with their sizes (which I again don't think is true, but I tried that as well, but with no change). Yet another post claimed that this happens if the initramfs image crosses the __init end boundary, and that placing the initramfs image firmly inside will allow the memory to be reclaimed after the image has been uncompressed by the kernel, and placing it beyond the __init section will work but then that memory is forever lost after boot. I know way too little about Linux in order to know if any of this is in any way true/accurate, and I have no idea where "__init"'s - if such a thing exists - end-boundary is, but if the issue is that I was crossing it I tried moving the initramfs image way beyond anywhere were I was previously using it, but that didn't change anything.
I also tried the original location (which works with a 16MiB image) and create a 16MiB + 1K sized image, but this didn't work either. (Checking that it didn't overlay any of the over images, obviously).
This originally led me to think there's a 16MiB initramfs size limit lurking somewhere. But on searching for it, it got me thinking that doesn't make sense -- as far as I can gather the bootm command in u-boot shoud set up the tag list for the system (which includes the location and size of the initramfs), and I haven't come across any note about a 16MiB limit in relation to the tagged lists for the initramfs so far.
I have found a page which claims that the initramfs size is in practical terms limited to roughly half the size of physical ram in the system, and the PicoZed board has 1G RAM, so we're in orders of magnitude away from what "should" be a limitation.
To clarify: The 16MiB image is the size of the raw image; compressed it's just under 6MiB. However, if I populate it so it's not compressed as tightly it doesn't make any difference -- the problem doesn't appear to be related to the size of the compressed image, only the uncompressed image.
Main question:
Where is this apparent 16MiB initramfs size limit coming from?
Side-question:
Is there such a thing as an "kernel __init section" which is reclaimed by the kernel after it has uncompressed/loaded images? If so, how do I see/configure the location/size of it?
Size constraints of initramfs on ARM?
I have not encountered a 16 MiB size constraint as you allege.
At one time I thought there was a size limit too, but that turned out to be a memory footprint issue during boot. Once that was sorted out I've been using large initramfs of 30MiB (e.g. with glibc, gtstreamer and qt5 libraries).
Where is this apparent 16MiB initramfs size limit coming from?
There isn't one. A ramfs is only constrained by available RAM.
There is a definition for the "Default RAM disk size", but this would not affect the size of an initramfs.
Your method of booting with the U-Boot bootm command is suspect, i.e. you're passing the memory address of the initramfs as the second argument.
The U-Boot documentation clearly describes the second argument as "the address of an initrd image" (emphasis added).
There is no mention of initramfs as an argument.
Linux kernel documentation states that the initramfs archive can be "linked into the linux kernel image". There's a kernel menuconfig entry for specifying the path to the initramfs cpio file. The make script will append this cpio file to the kernel image so that there is a single image file for booting.
Or (like an initrd) "a separate file" can be passed to the kernel at boot to populate the initramfs.
U-Boot passes the location (and length) of this archive either as an ATAG entry or as a reserved memory region in the Device Tree blob.
The kernel expects either a cpio archive for the initramfs or a tar archive for an initrd.
You neglect to mention (besides its compression) what kind of archive this "initramfs" or "separate file" actually is .
So it's not clear if you're booting the kernel with an initrd (tar archive) or an initramfs (cpio archive).
Your repeated reference to the initramfs as an "image" file rather than a cpio archive suggests that you really are using an initrd.
An initrd would definitely have a size constraint.
Is there such a thing as an "kernel __init section" which is reclaimed by the kernel after it has uncompressed/loaded images?
Yes, there is an __init section of memory, which is released after kernel initialization is complete.
If so, how do I see/configure the location/size of it?
Usually routines and data that have no use after initialization can be declared with the __init macro. See this.
The location and size of this memory section would be under the control of the linker script, rather than explicit user control. The kernel System.map file should have the info for review.
I think you need to pass ramdisk_size in uboot bootargs command
ramdisk_size needs to be set if the ram-disk uncompress file
size is bigger than default setting. It should be more than
ramdisk uncompress file size.
So I'm developing for an embedded Linux system and we had some trouble with an external watchdog chip which needed to be fed very early in the boot process.
More specifically, from what we could work out it would this external watchdog would cause a reset while the kernel was decompressing its image in the pre-boot environment. Not enough down time before it starts needing to be fed, which should probably have been sorted in hardware as it is external, but an internal software solution is wanted.
The solution from one of our developers was to put in some extra code into...
int zlib_inflate(z_streamp strm, int flush) in the lib/zlib_inflate/inflate.c kernel code
This new code periodically toggles the watchdog pin during the decompression.
Now besides the fact that I feel like this is a little bit of a dirty hack. It does work, and it has raised an interesting point in my mind. Because this lib is used after boot as well. So is there a nice way for a bit of code detecting whether you're in the pre-boot environment? So it could only preform this toggling pre-boot and not when the lib is used later.
As an aside, I'm also interested in any ideas to avoid the hack in the first place.
So is there a nice way for a bit of code detecting whether you're in the pre-boot environment?
You're asking an XY question.
The solution to the X problem can be cleanly solved if you are using U-Boot.
(BTW instead of "pre-boot", i.e. before boot, you probably mean "boot", i.e. before the kernel is started.)
If you're using U-Boot in the boot sequence, then you do not have to hack any boot or kernel code. Apparently you are booting a self-extracting compressed kernel in a zImage (or a zImage within a uImage) file. The hack-free solution is described by U-Boot's author/maintainer, Wolfgang Denk:
It is much better to use normal (uncompressed) kernel image, compress it
using just gzip, and use this as poayload for mkimage. This way
U-Boot does the uncompresiong instead of including yet another
uncompressor with each kernel image.
So instead of make uImage, do a simple make.
Compress the Image file, and then encapsulate it with the U-Boot wrapper using mkimage (and specify the compression algorithm that was applied so that U-Boot can use its built-in decompressor) to produce your uImage file.
When U-Boot loads this uImage file, the wrapper will indicate that it's a compressed file.
U-Boot will execute its internal decompressor library, which (in recent versions) is already watchdog aware.
Quick and dirty solution off the top of my head:
Make a global static variable in the file that's initialized to 1, and as long as it's 1, consider that "pre-boot".
Add a *_initcall (choose whichever fits your needs. I'm not sure when the kernel is decompressed) to set it to 0.
See include/linux/init.h in the kernel tree for initcall levels.
See #sawdust answer for an answer on how to achieve the watchdog feeding without having to hack the kernel code.
However this does not fully address the original question of how to detect that code is being compiled in the "pre-boot environment", as it is called within kernel source.
Files within the kernel such as ...
include/linux/decompress/mm.h
lib/decompress_inflate.c
And to a lesser extent (it isn't commented as clearly)...
lib/decompress_unlzo.c
Seem to check the STATIC definition to set "pre-boot environment" differences. Such as in this excerpt from include/linux/decompress/mm.h ...
#ifdef STATIC
/* Code active when included from pre-boot environment: */
...
#else /* STATIC */
/* Code active when compiled standalone for use when loading ramdisk: */
...
#endif /* STATIC */
Another idea can be disabling watchdog from bootloader and enabling it from user space once system has booted completely.
I am learning the linux kernel booting process and trying to install linux on my beagleboard xM. I came across two approach both using the SD card.
1. Have the MLO, initrd, uboot.bin and uImage in one partition.
2. Have the MLO, uboot.bin and uImage in one partition and a pre-built rootfs (Angstrom) in 2nd partition.
In the first approach how is the initrd transformed into a complete file system on the 2nd partition. And what happens internally when uboot extracts the kernel from uImage and paste it on the 2nd partition. which directories would get modified of the init rootfs. How the init process of the kernel is called.
In general, the booting sequence in the Beagleboard-XM is something like this :
Firmware -> MLO -> uboot.bin -> uImage -> initrd(optional) -> rootfs
The initrd itself is a simple RAM based file system. It is generally used to mount the actual file system. Some of the modules which are required to mount the rootfs are bundled up into the initrd. Once the rootfs is mounted, the initramfs is cleaned up. If the features that are required to mount the rootfs is built into the kernel (uImage) itself, then there is no need of initrd. As I said earlier, initrd is a simple RAM based file system. That means if you use this as your filesystem, anything you write onto it will not be preserved. A few embedded applications actually use this as their rootfs.
Now answering to your question,
How is the initrd transformed into a complete file system on the 2nd partition?
Initially u-boot loads the kernel(uImage), initrd onto RAM and pass the appropriate the command line arguments to the kernel. Once the u-boot transfers the control over to the kernel, it first initializes the drivers and other modules and finally looks for initrd. The initrd is generally put as a cpio image. The kernel then uncompresses it and looks for 'init'. The init processes few things and then mounts the rootfs. There is no such thing as 2nd partition when it comes to initrd. It resides on the RAM.
What happens internally when uboot extracts the kernel from uImage and paste it on the 2nd partition?
Again, there is no such things as 2nd partition when it comes to uImage. uImage is a format which is understood by u-boot. The u-boot first loads the uImage onto the RAM and before actually passing the control over to the kernel, it extracts the contents from uImage, moves the binary to a predefined location in the RAM and then passes the control over to the kernel. Similar to initrd, the kernel also resides on the RAM itself.
Which directories would get modified of the init rootfs?
No directories gets modifies. But yes, a few of them are mounted, like procfs, sysfs, etc.
How the init process of the kernel is called?
Please refer to the kernel_init() function under init/main.c to see how the kernel executes init (refer http://lxr.free-electrons.com/source/init/main.c#L871).
I am attaching a simple code snippet from it :
if (!try_to_run_init_process("/sbin/init") ||
!try_to_run_init_process("/etc/init") ||
!try_to_run_init_process("/bin/init") ||
!try_to_run_init_process("/bin/sh"))
return 0;
panic("No working init found. Try passing init= option to kernel. "
"See Linux Documentation/init.txt for guidance.");
Therefore, the kernel searches in these default paths for the init binary. If it is not found, the kernel panics!
Hope this helps.
While in u-boot of my ARM based board (DM368) I mark some kernel partition block manually as bad. U-boot says that it was marked and, for example, while writing/reading kernel image I see it skipping this bad block.
But when I try to write the same partition from within Linux (loaded via NFS) I see that Linux nandwrite command USES this bad block! I checked this in several ways - Linux ignores bad block mark for 100%. But everywhere in the internet it is said that BBT is one for both u-boot and Linux.
So, where is the catch?
OK, the answer is found.
For some unclear reason Texas Instruments, manufacturer of the board DM365EVM which I use for development, provides the kernel with different BBT structure. They defined BBT offset as 2, while all the world, including the provided u-boot, defines this offset as 8.
I wish them a good health for many years.
From my understanding, after a PC/embedded system booted up, the OS will occupy the entire RAM region, the RAM will look like this:
Which means, while I'm running a program I write, all the variables, dynamic memory allocated in the stacks, heaps and etc, will remain inside the region. If I run firefox, paint, gedit, etc, they will also be running in this region. (Is this understanding correct?)
However, I would like to shrink the OS region. Below is an illustration of how I want to divide the RAM:
The reason that I want to do this is because, I want to store some data receive externally through the driver into the Custom Region at fixed physical location, then I will be able to access it directly from the user space without using copy_to_user().
I think it is possible to do that by configuring u-boot, but I have no experience in u-boot, can anyone give me some directions where to begin with, such as: do I need to modify the source of u-boot, or changing the environment variables of u-boot will be sufficient?
Or is there any alternative method of doing this?
Any help is much appreciated. Thanks!
p/s: I'm using TI ARM processor, and booting up from an SD card, I'm not sure if it matters.
The platform is ARM. min_addr and max_addr will not work on these platform since these are for Intel-only implementations.
For the ARM platform try to look at "mem=size#start" kernel parameter. Read up on Documentation/kernel-parameters.txt and arch/arm/kernel/setup.c. This option is available on most new Linux code base (ie. 2.6.XX).
You need to set the following parameters:
max_addr=some_max_physical
min_addr=some_min_physical
to be passed to the kernel through uboot in the 'bootargs' u-boot environment variable.
I found myself trying to do the opposite recently - in other words get Linux to use the additional memory in my system - although I'm using Barebox rather than u-boot on a OMAP4 platform.
I found (a bit to my surprise) that once the Barebox MLO first stage boot-loader was aware of the extra RAM, the kernel then detected and used it as well without any bootargs. Since the memory size is not passed anywhere on the boot-line, I can only assume the kernel inspects the memory mappings set up by the boot-loader to determine RAM size. This suggests that modifying your u-boot to not map all of the RAM is the way to go.
On the subject of boot-args, there was a time when you it was recommended that you mapped out a chunk of RAM (used by the frame buffer?) on OMAP4 systems, using the boot-line. It's still unclear whether this is still necessary.