I have 3 MB of SPI flash on my board and I am able to pack the bzImage, busybox initrd and coreboot ROM into this SPI flash. I am also able to boot to the shell , all using the ROM kernel-as-bootloader.
I have bigger kernel on the USB device. I am also able to detect the USB and mount it. But the problem is busybox does not seem to have kexec utility. I cannot use any other initrd package because my cpio (uncompressed) size should not go beyond 1.4 MB due to memory constraints. U-root has kexec support but the minimal image size easily reaches 3MB or at least, I couldn't find a way to built it lesser than that size.
Hence, is there a way to add kexec support to busybox (compile static binary and copy to initrd? ) or any other initrd package that can suffice the need within 1.4MB size?
EDIT
This post suggests that there may be kexec support available in busybox, but I couldn't find any trace of it. In fact the request to add kexec-tools to busybox was done over a decade ago. But when I did grep in the busybox, I saw no traces of it.
The kexec binary from kexec-tools takes about 300KB (x86_64 with -Os). Even if it would be added to busybox, it wouldn't get much smaller than that because it does need to do fairly complicated things that are not done anywhere else in busybox.
If you don't have even 300KB left, then you should probably remove configuration options from busybox itself to save space. With uClibc, you can also remove some options you don't need, like wchar and threading. Make sure you use static linking, it saves a little bit of space.
The following Buildroot defconfig generates an initramfs of exactly 1.4MB.
BR2_x86_64=y
BR2_x86_corei7=y
BR2_STATIC_LIBS=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/x86-64-core-i7/tarballs/x86-64-core-i7--uclibc--stable-2018.11-1.tar.bz2"
BR2_TOOLCHAIN_EXTERNAL_GCC_7=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
BR2_PACKAGE_KEXEC=y
BR2_PACKAGE_KEXEC_ZLIB=y
BR2_TARGET_ROOTFS_CPIO=y
# BR2_TARGET_ROOTFS_TAR is not set
We could build a version of kexec for busybox which results in a 220K static linked binary like so:
Use musl
Strip the binary with strip (Maybe it could reduce the size even more?)
Disable unused features via ./configure --without-<FEATURE>
KEXEC_VERSION=2.0.24
KEXEC=kexec-tools-$KEXEC_VERSION
curl -LO https://mirrors.edge.kernel.org/pub/linux/utils/kernel/kexec/$KEXEC.tar.xz
tar xf $KEXEC.tar.xz
cd $KEXEC
CC=musl-gcc LDFLAGS=-static ./configure \
--without-lzma \
--without-xen \
--sbindir=/bin
make
strip build/sbin/kexec
file buils/sbin/kexec # -> [...] statically linked, stripped
du -h build/sbin/kexec # -> 220K [...]
Still not great, but yeah.. Let us know if you could shrink it even more :-)
Related
I'm developing for a STM32F429 with CLion and trying to create a merged .elf file from three .elf files.
The layout of the merged .elf file should look like this:
Bootloader.elf (maximum 256K, but can vary in size)
FirmwareImage.elf (always 384K, fills up allotted space for 1st firmware image)
FirmwareImage.elf (always 384K, fills up allotted space for 2nd firmware image)
This makes 1M exactly in total.
I've tried to converting the .elf files to .bin (with arm-none-eabi-objcopy) and creating a 1M .bin file with a custom tool I wrote. I can confirm the layout of the .bin this creates is correct.
I then convert the generated .bin file back to .elf with:
arm-none-eabi-objcopy -I binary -O elf32-little --change-section-address .data=0x08000000 in.bin out.elf
The data section address change is to reflect the start of flash memory in the STM32F429.
When I try to flash this file using openocd, it fails with:
Error: invalid ELF file, no program headers
Is there a way of skipping the .elfs -> .bin -> .elf conversion and go directly from .elfs -> .elf? Or otherwise, insert the program headers? I doubt these are actually used during flashing.
I know I could tell openocd to use the generated .bin file directly by specifing the flash address. But CLion won't let openocd flash .bin files from its arm-embedded plugin, only .elf files.
Edit:
At the moment I'm using these linker scripts, adapted from ST's official one. There is quite a bit going on in them.
I might just be able to use a single linker script to achieve this. I could have the following code:
extern "C" __attribute__((__section__(".bootloader"))) void runBootloader()
{
Platform platform;
DFUBootloader bootloader(platform);
}
extern "C" __attribute__((__section__(".image1"))) void runApplication1()
{
Platform platform;
Application application(platform);
}
extern "C" __attribute__((__section__(".image2"))) void runApplication2()
{
Platform platform;
Application application(platform);
}
If these linker places these symbols at the right locations, would the symbols used by them end up in the correct place as well? The firmware image has to be duplicated to fill up both firmware pages, so all the symbols used by the application as well have to be duplicated.
It looks like merging multiple .elf files in the way I described is not possible, or at least not very trivial.
I have found a workaround for my problem trough. Although CLion is incapable getting openocd to flash .bin files (its builtin openocd launch command uses .elf), It's easy to create a custom CMake target that flashes the merged .bin with the following command:
openocd -f board/stm32f429discovery.cfg -c "program merged_image.bin exit 0x08000000"
I have used a makefile to build my code and I have produced an ELF file.
To make it understandable for my attiny85, I usually use avr-objcopy -O ihex -R .eeprom -R .fuse main.elf main_all.hex. I get a hex file containing fuse settings. I flash the hex file with avrdude -p t85 -c avrispmkII -P usb -U flash:w:main_all.hex.
I am using an avrispmkII connected via a working and tested SPI.
This time I got an error.
ERROR: address 0x820003 out of range
I guess because I've played in the code with fuses that this is the problem. According to Contiki compile error, " ERROR: address 0x820003 out of range at line 1740 of...",
I've noticed that you can make avrdude create a hex without fuses.
avr-objcopy -O ihex -R .eeprom -R. Fuse main.elf main_ohne.hex
This has also worked and now lets the attiny85 flash completely normally.
Now the real question.
How do I still get the fuses on the attiny85?
Is there any way to see which fuse I am setting how, before I set the fuses? I ask explicitly before, because I have no experience in flashing with 12V (HV) and this arvmkII synonymous not true (Yes, I should look in the data sheet whether he can).
My main concern is to get the fuses on the attiny. I am a graduate electrical engineer who is programming in the spare time. So I'm fine with overprivileged links and the magic command.
(Rough translation from the German original)
You can set the fuse bytes on the command-line of avrdude. example
There are only 3 fuse bytes on the attiny: low, high, and extended. They can be found on p. 148 of the datasheet.
Just compute the fuse setting as a hex number and include -U switches like
-U efuse:w:0xff:m -U hfuse:w:0x89:m -U lfuse:w:0x2e:m
for the extended, high, and low fuses.
I complied a hello world Go program which generated native executable on my linux machine. But I was surprised to see the size of the simple Hello world Go program, it was 1.9MB !
Why is it that the executable of such a simple program in Go is so huge?
This exact question appears in the official FAQ: Why is my trivial program such a large binary?
Quoting the answer:
The linkers in the gc tool chain (5l, 6l, and 8l) do static linking. All Go binaries therefore include the Go run-time, along with the run-time type information necessary to support dynamic type checks, reflection, and even panic-time stack traces.
A simple C "hello, world" program compiled and linked statically using gcc on Linux is around 750 kB, including an implementation of printf. An equivalent Go program using fmt.Printf is around 1.9 MB, but that includes more powerful run-time support and type information.
So the native executable of your Hello World is 1.9 MB because it contains a runtime which provides garbage collection, reflection and many other features (which your program might not really use, but it's there). And the implementation of the fmt package which you used to print the "Hello World" text (plus its dependencies).
Now try the following: add another fmt.Println("Hello World! Again") line to your program and compile it again. The result will not be 2x 1.9MB, but still just 1.9 MB! Yes, because all the used libraries (fmt and its dependencies) and the runtime are already added to the executable (and so just a few more bytes will be added to print the 2nd text which you just added).
Consider the following program:
package main
import "fmt"
func main() {
fmt.Println("Hello World!")
}
If I build this on my Linux AMD64 machine (Go 1.9), like this:
$ go build
$ ls -la helloworld
-rwxr-xr-x 1 janf group 2029206 Sep 11 16:58 helloworld
I get a a binary that is about 2 Mb in size.
The reason for this (which has been explained in other answers) is that we are using the "fmt" package which is quite large, but the binary has also not been stripped and this means that the symbol table is still there. If we instead instruct the compiler to strip the binary, it will become much smaller:
$ go build -ldflags "-s -w"
$ ls -la helloworld
-rwxr-xr-x 1 janf group 1323616 Sep 11 17:01 helloworld
However, if we rewrite the program to use the builtin function print, instead of fmt.Println, like this:
package main
func main() {
print("Hello World!\n")
}
And then compile it:
$ go build -ldflags "-s -w"
$ ls -la helloworld
-rwxr-xr-x 1 janf group 714176 Sep 11 17:06 helloworld
We end up with an even smaller binary. This is as small as we can get it without resorting to tricks like UPX-packing, so the overhead of the Go-runtime is roughly 700 Kb.
Note that the binary size issue is tracked by issue 6853 in the golang/go project.
For instance, commit a26c01a (for Go 1.4) cut hello world by 70kB:
because we don't write those names into the symbol table.
Considering the compiler, assembler, linker, and runtime for 1.5 will be
entirely in Go, you can expect further optimization.
Update 2016 Go 1.7: this has been optimized: see "Smaller Go 1.7 binaries".
But these day (April 2019), what takes the most place is runtime.pclntab.
See "Why are my Go executable files so large? Size visualization of Go executables using D3" from Raphael ‘kena’ Poss.
It is not too well documented however this comment from the Go source code suggests its purpose:
// A LineTable is a data structure mapping program counters to line numbers.
The purpose of this data structure is to enable the Go runtime system to produce descriptive stack traces upon a crash or upon internal requests via the runtime.GetStack API.
So it seems useful. But why is it so large?
The URL https://golang.org/s/go12symtab hidden in the aforelinked source file redirects to a document that explains what happened between Go 1.0 and 1.2. To paraphrase:
prior to 1.2, the Go linker was emitting a compressed line table, and the program would decompress it upon initialization at run-time.
in Go 1.2, a decision was made to pre-expand the line table in the executable file into its final format suitable for direct use at run-time, without an additional decompression step.
In other words, the Go team decided to make executable files larger to save up on initialization time.
Also, looking at the data structure, it appears that its overall size in compiled binaries is super-linear in the number of functions in the program, in addition to how large each function is.
Can I measure the code size with the help of an fseek() function and store it to a shell variable?
Is it possible to extract the code size, compilation time and execution time using milepost gcc or a GNU Profiler tool? If yes, how to store them into shell variables?
Since my aim is to find the best set of optimization technique upon the basis of the compilation time, execution time and code size, I will be expecting some function that can return these parameters.
MyPgm=/root/Project/Programs/test.c
gcc -Wall -o1 -fauto-inc-dec $MyPgm -o output
time -f "%e" -o Output.log ./output
while read line;
do
echo -e "$line";
Val=$line
done<Output.log
This will store the execution time to the variable Val. Similarly, I want to get the values of code size as well as compilation time.
I will prefer something that I can do to accomplish this, without using an external program!
for code size on linux, you can use size command on terminal.
$size file-name.out
it will give size of different sections. use text section for code size. you can use data and bss if you want to consider global data size as well.
You can use the size(1) command http://www.linuxmanpages.com/man1/size.1.php
Or open the ELF file, walk over section headers and sum the sizes of all the section with type SHT_PROGBITS and the SHF_EXECINSTR flag set.
On non-Linux / non-GNU-utils systems (where you may have neither GNU size nor readelf), the nm program can be used to dump symbol information (including sizes) from object files (libraries / executables). The syntax is slightly system-dependent:
OpenGroup manpage for nm (the "portable subset")
Linux/BSD manpage for nm (GNU version)
Solaris manpage for nm
AIX manpage for nm
nm usage on HP/UX (this says "PA-RISC" but the utility is present / usable on Itanium)
Windows: Doesn't have nm as such, but see: Microsoft equivalent of the nm command
Unfortunately, while the utility is available almost everywhere, its output format is not as portable as could be, so some system-specific scripting is necessary.
If I do --strip-debug or --strip-unneeded, I have the .ko that lists all function names with nm, if I do just strip foo.ko I have a kernel module that refuses to load.
Does anyone know a quick shortcut how to remove all symbols that are not needed for module loading so that people cannot reverse engineer the API:s as easily?
PS: For all you open source bigots missionaries; this is something that general public will never be using in any case so no need to turn the question into a GPL flame war.
With no answer to my previous questions, here are some guesses that could also be some clues, and a step to an answer:
From what I recall, a .ko is nothing but an .o file resulting from the merge of all the .o files generated by your source module, and the addition of a .modinfo section.
At the end of any .ko building Makefile, there is an LD call: from what I recall, ld is called with the -r option, and this is what create that .o file that the Makefile calls a .ko. This resulting file is not to be confused with an archive or object library (.a file), that is just a format archiving / packaging multiple .o files as one: A merged object is the result of a link that produces yet another .o module: But in the resulting module, all sections that could be merged have been, and all public / external pairs that could be resolved have been inside those sections.
So I assume that you end up with your .ko file containing all your "local" extern definitions:
Those that are extern because they
are used to call across the .o
modules in your .ko (but are not
needed anymore since they are not
supposed to be called from outside
the .ko), and
those that the .ko module DO need to
properly communicate with the loader
and kernel.
The former have most likely already been resolved by ld during the merge, but ld has no way to know whether you intend to have them also callable from outside the .ko.
So the extraneous symbols you see are those that are extern for each of your .o files, but are not needed as extern for the resulting .ko.
And what you are looking for is a way to strip only those.
Does this last paragraph properly describe the symbols you want to get rid of?
I think this is exactly what we are
talking about here.
OK, then it looks like one solution is to "manually" remove the extraneous symbols. The "strip" utility seems to allow individually stripping (or keeping) of symbols, so you would have to use one --strip-all and a small bunch of --keep-symbol= . Note that --wildcard might help a bit, too. You can do the opposite, of course, keep all and individually strip, depending on what's the most convenient.
A good start could be to remove all the symbols that you explicitly defined in your module for cross-module linking and don't want to appear - just leaving the obvious useful ones, things like init and exit. And to not touch those that have been generated by / belong to the kernel dev software infrastructure. Then trial and error until you find the right recipe... In fact, I would think that about all your own symbols might be removable, apart from those you explicitly defined yourself as EXPORT_SYMBOL (and init / exit, of course).
Good luck! :)
PS:
In fact, it seems that the required source information exists in all .ko projects to perform the required stripping automatically: Unless I'm missing something, it seems that anything that's not EXPORT_SYMBOL or explicitly inserted by the build software could theoretically be stripped by default at the end of "ld -r" time that ends a .ko build. It's just that I don't think the toolchain (compiler / linker) have provision / directives / options to individually designate "strip or keep" syms for the relocatable link / merge. Otherwise, some modifications in the EXPORT_SYMBOL macro and in a few other places could probably achieve the result you're after, and shave some bytes from most .ko files in any Linux system.
I just built a kernel without realizing the kernel config had debug symbols enabled, so the size of the resulting modules were quite large. This worked for me:
# du -sh /lib/modules/3.1.0/
1.9G /lib/modules/3.1.0/
# find /lib/modules/3.1.0/ -iname "*.ko" -exec strip --strip-debug {} \;
# du -sh /lib/modules/3.1.0/
134M /lib/modules/3.1.0/
Find all files in /lib/modules/3.1.0 named *.ko and execute strip --strip-debug on each of them.
I'm not sure I understand what the problem really is:
When developing a .ko, if I don't explicitly add something like
ccflags-y += -ggdb -O0 -Wall
into my Makefile, I don't get any symbol but for those that I publish or external ref myself. I'm sure I don't get any other symbols for several good reasons:
the resulting .ko file is considerably smaller,
dumping the file and analyzing the ELF shows the tables are not there,
I can't see nor access the symbols in kgdb.
So I'm a little puzzled at your question, actually?... What are those symbols you do see in your .ko (and don't want to)?
How are they declared in your source file?
In which ELF sections do they end up?
And (sorry, dumb question ahead): Did you define static all things that didn't need to be seen outside of their own module?
In addition to filofel's post:
The reason stripping userspace shared libraries keeps them functioning is because their exported symbols are in the .dynsym section which is never stripped. .ko files however do not use dynsym.
people have reported success with
strip --strip-unneeded
strip -g XXX.
My Previous problem like what you happened is sloved by this command in embedded device with Linux Kernel 3.0.8.