Kernel panic on simple loadable kernel module - linux-kernel

I`m trying to create an elementary loadable kernel module to embedded OpenWRT system on the Ralink 3050 SOC (processor MIPS 24KEs). The target system built with MODULES=y and CONFIG_MODULE_UNLOAD=y options. I work under Ubuntu 20 X86 machine.
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("test");
int init_module(void) {
printk(KERN_INFO "Hello, world!\n");
return 0;
}
void cleanup_module(void) {
printk(KERN_INFO "Goodbye, world!\n");
}
The first time, I`m was trying to compile module by the built-in OpenWRT buildroot cross-compiler:
make ARCH=mips CROSS_COMPILE=mipsel-openwrt-linux-musl-
The kernel headers included the OpenWRT buildroot were used.
Result, fail: after insmod load module was marked as [permanent]. I.e. the cleanup_module procedure was not seen by the kernel. I assumed that problem in a file structure. I`m was checking the vermagic signature (to an other system module that is workable): It matched.
$modinfo module.ko -> vermagic: 5.10.100 mod_unload MIPS32_R2 32BIT
Just in case, I had rebuilt the target system, may be it`s stochastic errors? But the result is the same…
Then I decided to build a custom cross-compiler and original kernel headers. I had downloaded and installed the binutils 2.38 and gcc 11.2.0 (same as at the OpenWRT buildroot) and was downloaded and prepared the kernel header with same .config file (kernel 5.10.100 – same as on the target system).
Now the next result:
After compilation make ARCH=mips CROSS_COMPILE=mipsel-unknown-linux-gnu- by custom cross compiler the module loads, but any next callers lsmod, rmmod or insmod finished by kernel panic. Looks like, the module corrupts kernel memory. If the module compiled without the cleanup_module procedure, it loads correct but have status [permanent] and can`t be unloaded.
I tried to change the function declaration: used __exit and static specificator – no result.
root#(none):/# lsmod
[ 456.197482] CPU 0 Unable to handle kernel paging request at virtual address 034881e0, epc == 80068f78, ra == 80068f64
[ 456.218804] Oops[#1]:
[ 456.223350] CPU: 0 PID: 1017 Comm: lsmod Not tainted 5.10.100 #0
[ 456.235314] $ 0 : 00000000 00000001 00000000 00000001
[ 456.245763] $ 4 : 8160200e 80320b64 00000000 00000001
[ 456.256206] $ 8 : 00000020 00000002 00000001 00000000
[ 456.266645] $12 : 7faff400 77eca2b0 77ec0020 7faff43f
[ 456.277090] $16 : 81a440a4 816110d8 034881d0 80320b64
[ 456.287534] $20 : 80380000 81a441d0 816110f0 81611100
[ 456.297977] $24 : 00000000 80101500
[ 456.308420] $28 : 81a46000 81a47d60 00000002 80068f64
[ 456.318864] Hi : 00000000
[ 456.324594] Lo : 0a3d70a4
[ 456.330333] epc : 80068f78 0x80068f78
[ 456.337973] ra : 80068f64 0x80068f64
[ 456.345609] Status: 1100e403 KERNEL EXL IE
[ 456.353965] Cause : 00800008 (ExcCode 02)
[ 456.361946] BadVA : 034881e0
[ 456.367679] PrId : 0001964c (MIPS 24KEc)
[ 456.375658] Modules linked in: module rt2800soc rt2800mmio rt2800lib rt2x00soc rt2x00mmio rt2x00lib mac80211 cfg80211 crc_ccitt compat sha256_generic libsha256 seqiv jitterentropy_rng drbg hmac cmac leds_gpio crc32c_generic
[ 456.415692] Process lsmod (pid: 1017, threadinfo=1acffaea, task=80d5ed61, tls=77ecbdd4)
[ 456.431637] Stack : 81621700 00400cc0 00000000 00000001 ffffffff 800b6700 81847c80 00002000
[ 456.448345] 00000000 81a47e18 00000000 a050f4de 81847c80 816110d8 81a440a4 00000000
[ 456.465053] 81a47f00 81a47e38 81a47e20 801011d8 00000001 00000001 00000000 00000601
[ 456.481760] 816e2904 800c7f98 77ec1000 81a47e8c 80584860 00000000 00000000 00000000
[ 456.498467] 81847c80 81a47f00 00000400 00000000 00000000 00000003 00000002 80101630
[ 456.515174] ...
[ 456.520050] Call Trace:
[ 456.520071] [<800b6700>] 0x800b6700
[ 456.531895] [<801011d8>] 0x801011d8
[ 456.538848] [<800c7f98>] 0x800c7f98
[ 456.545803] [<80101630>] 0x80101630
[ 456.552751] [<800c1bdc>] 0x800c1bdc
[ 456.559699] [<80018628>] 0x80018628
[ 456.566665] [<800df8f0>] 0x800df8f0
[ 456.573627] [<800faae8>] 0x800faae8
[ 456.580579] [<800de2b0>] 0x800de2b0
[ 456.587531] [<800dfbf0>] 0x800dfbf0
[ 456.594479] [<800c7e2c>] 0x800c7e2c
[ 456.601433] [<8000d36c>] 0x8000d36c
[ 456.611364] Сode: 00001025 10000007 26730b64 <8e460010> 24c6000c 0c040199 02202025 8e520000 24020001
UPD
Did you include calls to the module_init() and module_exit() macros to
register the functions as init/exit? Seems like you didn't from your
example. Also, you should add __init and __exit annotations. See this
simple example of mine. – Marco Bonelli
Yes, I tried to use the macro module_init() and module_exit(). It has no result. Anyway, this macro just translates the init and clean function name to the canonical function name: init_module and cleanup_module, like the main function in a user space program. Prefix __init and __exit also has no effect.
Smells like you use a compiler which configuration is not suitable for
the kernel. You could try to debug "Unable to handle kernel paging
request" error. – Tsyvarev
What is a "compiler configuration" that specifies the user or kernel object file type?
If we look (using readelf -a utility) at the difference between my module and any built-in module, we will see that the entry points of the init_module method in the .rel.gnu.linkonce.this_module section are identical, but the entry points of the cleanup_module are different.
Relocation section '.rel.gnu.linkonce.this_module' at offset 0x594 contains 2 entries:
Offset Info Type Sym.Value Sym. Name
000000d0 00001d02 R_MIPS_32 00000000 init_module
00000130 00001c02 R_MIPS_32 00000000 cleanup_module
Relocation section '.rel.gnu.linkonce.this_module' at offset 0x1c8c contains 2 entries:
Offset Info Type Sym.Value Sym. Name
000000d0 00000a02 R_MIPS_32 00000000 _1
00000140 00000902 R_MIPS_32 00000000 _0

Related

Raspberry PI 4 hardware registers access

I'm working on an embedded Linux training with Raspberry PI 4B and in order to create I2C bus driver I need to access hardware registers. The problem is when I try to read any of them I get "Unhandled fault: unknown 3 (0x203) at 0xf08c5000".
Here is the code I'm using to read register:
void i2cHandler_init(void)
{
const u32 rpiPeriphBase = 0xfe000000;
const u32 rpiGpioOffset = 0x200000;
const void *rpiGpioAddress = rpiPeriphBase + rpiGpioOffset;
__iomem u32 *ioMemory;
ioMemory = ioremap(rpiGpioAddress, 0x40 * 8);
pr_info("%s: rpiGpioAddress = 0x%x\r\n", __func__, rpiGpioAddress);
pr_info("%s: ioMemory = 0x%x\r\n", __func__, ioMemory);
pr_info("%s: Register value = 0x%x\r\n", __func__, readl(ioMemory));
if(ioMemory != NULL)
iounmap(ioMemory);
}
Yes, I'm trying to read GPIO GPFSEL0 register here, but changing offset to read I2C Control register changes nothing.
Kernel log output looks like this:
[ 1467.677279] i2cHandler_init: rpiGpioAddress = 0xfe200000
[ 1467.677289] i2cHandler_init: ioMemory = 0xf08c5000
[ 1467.677308] 8<--- cut here ---
[ 1467.677325] Unhandled fault: unknown 3 (0x203) at 0xf08c5000
[ 1467.677340] pgd = f812a8c9
[ 1467.677353] [f08c5000] *pgd=80000000007003, *pmd=2ff5b003, *pte=c00ffffe200713
[ 1467.677385] Internal error: : 203 [#1] SMP ARM
[ 1467.677400] Modules linked in: rpi_driver_I2C(O+) rfcomm cmac algif_hash aes_arm_bs crypto_simd cryptd algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc spidev snd_soc_hdmi_codec 8021q garp stp llc brcmfmac brcmutil v3d cfg80211 gpu_sched rfkill raspberrypi_hwmon i2c_brcmstb vc4 spi_bcm2835 cec bcm2835_codec(C) bcm2835_v4l2(C) bcm2835_isp(C) v4l2_mem2mem bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common vc_sm_cma(C) drm_kms_helper videodev snd_bcm2835(C) snd_soc_core mc snd_compress snd_pcm_dmaengine snd_pcm rpivid_mem snd_timer snd syscopyarea sysfillrect sysimgblt fb_sys_fops uio_pdrv_genirq uio nvmem_rmem i2c_dev drm drm_panel_orientation_quirks backlight fuse ip_tables x_tables ipv6 [last unloaded: rpi_driver_I2C]
[ 1467.677744] CPU: 0 PID: 2059 Comm: insmod Tainted: G C O 5.15.32-v7l+ #1538
[ 1467.677764] Hardware name: BCM2711
[ 1467.677775] PC is at i2cHandler_init+0xc4/0x178 [rpi_driver_I2C]
[ 1467.677808] LR is at irq_work_queue+0x14/0x2c
[ 1467.677833] pc : [<bf3fe464>] lr : [<c0358054>] psr: 60000013
[ 1467.677846] sp : c3e57d68 ip : 00000000 fp : c3e57d7c
[ 1467.677859] r10: c1205048 r9 : bf400380 r8 : 00000000
[ 1467.677871] r7 : 00000002 r6 : bf3fe5a8 r5 : c1205048 r4 : f08c5000
[ 1467.677884] r3 : 3391a86e r2 : 3391a86e r1 : 00000027 r0 : 00000029
[ 1467.677898] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 1467.677914] Control: 30c5383d Table: 041e1e40 DAC: fffffffd
[ 1467.677926] Register r0 information: non-paged memory
[ 1467.677943] Register r1 information: non-paged memory
[ 1467.677958] Register r2 information: non-paged memory
[ 1467.677973] Register r3 information: non-paged memory
[ 1467.677988] Register r4 information: 0-page vmalloc region starting at 0xf08c5000 allocated at i2cHandler_init+0x80/0x178 [rpi_driver_I2C]
[ 1467.678024] Register r5 information: non-slab/vmalloc memory
[ 1467.678041] Register r6 information: 4-page vmalloc region starting at 0xbf3fe000 allocated at load_module+0xb94/0x2840
[ 1467.678069] Register r7 information: non-paged memory
[ 1467.678084] Register r8 information: NULL pointer
[ 1467.678099] Register r9 information: 4-page vmalloc region starting at 0xbf3fe000 allocated at load_module+0xb94/0x2840
[ 1467.678124] Register r10 information: non-slab/vmalloc memory
[ 1467.678140] Register r11 information: non-slab/vmalloc memory
[ 1467.678155] Register r12 information: NULL pointer
[ 1467.678170] Process insmod (pid: 2059, stack limit = 0x91bbcd6e)
[ 1467.678185] Stack: (0xc3e57d68 to 0xc3e58000)
[ 1467.678199] 7d60: bf400380 c1205048 c3e57d8c c3e57d80 bf3fe5fc bf3fe3ac
[ 1467.678217] 7d80: c3e57d9c c3e57d90 bf3fe5cc bf3fe5e4 c3e57e14 c3e57da0 c02021c4 bf3fe5b4
[ 1467.678234] 7da0: c0439ca0 c0bd5bf4 c1401180 00000000 c3e57dd4 c3e57dc0 c0bd5bf4 c029f244
[ 1467.678251] 7dc0: c1401180 c043ade4 c3e57e14 c3e57dd8 c043ade4 c03f4518 c041c20c c043997c
[ 1467.678269] 7de0: 00000008 c02d056c f097d000 3391a86e 00000002 bf400380 00000002 c3c36d80
[ 1467.678286] 7e00: 00000002 c4ad01c8 c3e57e3c c3e57e18 c02d058c c0202180 c3e57e3c c3e57e28
[ 1467.678303] 7e20: c041c394 c3e57f30 00000002 c4ad0180 c3e57f14 c3e57e40 c02d2dac c02d0544
[ 1467.678320] 7e40: bf40038c 00007fff bf400380 c02cf240 c1205048 c0e3b448 c0e3b460 c0e3b3f0
[ 1467.678337] 7e60: bf0c609a c0e3b4b0 c0c03df0 c3e57f30 bf400594 c4dc6418 bf4003c8 c4ad0188
[ 1467.678354] 7e80: c0e3afcc 00000001 00000000 c0efea84 c0ee67d0 00000000 00000000 00000000
[ 1467.678371] 7ea0: 00000000 00000000 6e72656b 00006c65 00000000 00000000 00000000 00000000
[ 1467.678388] 7ec0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1467.678405] 7ee0: 00000000 3391a86e c3e57f2c c1205048 00000000 0002de04 00000003 c0200244
[ 1467.678423] 7f00: c3e56000 0000017b c3e57fa4 c3e57f18 c02d33b0 c02d0900 c3e57f2c 7fffffff
[ 1467.678439] 7f20: 00000000 00000002 c3e57f24 f097d000 f097db9e f097e5c0 f097d000 000033ac
[ 1467.678456] 7f40: f097ff9c f097fea8 f097f3a0 00003000 000036e0 00002018 00003aab 00000000
[ 1467.678474] 7f60: 00000000 00000000 00002008 00000017 00000018 00000010 0000000d 0000000a
[ 1467.678491] 7f80: 00000000 3391a86e 00000000 00000002 f9957800 0000017b 00000000 c3e57fa8
[ 1467.678508] 7fa0: c0200040 c02d32f4 00000000 00000002 00000003 0002de04 00000000 b6f68074
[ 1467.678535] 7fc0: 00000000 00000002 f9957800 0000017b 00e9acc8 00000002 bee367d4 00000000
[ 1467.678554] 7fe0: bee36600 bee365f0 00023bc0 b6c459e0 60000010 00000003 00000000 00000000
[ 1467.678568] Backtrace:
[ 1467.678582] [<bf3fe3a0>] (i2cHandler_init [rpi_driver_I2C]) from [<bf3fe5fc>] (mainCycle_init+0x24/0x38 [rpi_driver_I2C])
[ 1467.678632] r5:c1205048 r4:bf400380
[ 1467.678643] [<bf3fe5d8>] (mainCycle_init [rpi_driver_I2C]) from [<bf3fe5cc>] (init_module+0x24/0x30 [rpi_driver_I2C])
[ 1467.678687] [<bf3fe5a8>] (init_module [rpi_driver_I2C]) from [<c02021c4>] (do_one_initcall+0x50/0x244)
[ 1467.678726] [<c0202174>] (do_one_initcall) from [<c02d058c>] (do_init_module+0x54/0x23c)
[ 1467.678760] r8:c4ad01c8 r7:00000002 r6:c3c36d80 r5:00000002 r4:bf400380
[ 1467.678776] [<c02d0538>] (do_init_module) from [<c02d2dac>] (load_module+0x24b8/0x2840)
[ 1467.678807] r6:c4ad0180 r5:00000002 r4:c3e57f30
[ 1467.678824] [<c02d08f4>] (load_module) from [<c02d33b0>] (sys_finit_module+0xc8/0xfc)
[ 1467.678857] r10:0000017b r9:c3e56000 r8:c0200244 r7:00000003 r6:0002de04 r5:00000000
[ 1467.678876] r4:c1205048
[ 1467.678892] [<c02d32e8>] (sys_finit_module) from [<c0200040>] (ret_fast_syscall+0x0/0x1c)
[ 1467.678925] Exception stack(0xc3e57fa8 to 0xc3e57ff0)
[ 1467.678943] 7fa0: 00000000 00000002 00000003 0002de04 00000000 b6f68074
[ 1467.678969] 7fc0: 00000000 00000002 f9957800 0000017b 00e9acc8 00000002 bee367d4 00000000
[ 1467.678986] 7fe0: bee36600 bee365f0 00023bc0 b6c459e0
[ 1467.679007] r7:0000017b r6:f9957800 r5:00000002 r4:00000000
[ 1467.679030] Code: e1a02004 e30f0360 e34b0f3f eb5f37d6 (e5942000)
[ 1467.679052] ---[ end trace b887a2abb18bba14 ]---
Also I'm sure peripherals base address is correct because output of "sudo cat /proc/iomem" looks like this:
00000000-3b3fffff : System RAM
00008000-00ffffff : Kernel code
01200000-013ef203 : Kernel data
40000000-7fffffff : System RAM
fd500000-fd50930f : fd500000.pcie pcie#7d500000
fd580000-fd58ffff : fd580000.ethernet ethernet#7d580000
fd580e14-fd580e1c : unimac-mdio.-19
fe004000-fe00401f : fe004000.txp txp#7e004000
fe007000-fe007aff : fe007000.dma dma#7e007000
fe007b00-fe007eff : fe007b00.dma dma#7e007b00
fe00a000-fe00a023 : fe100000.watchdog watchdog#7e100000
fe00b840-fe00b87b : fe00b840.mailbox mailbox#7e00b840
fe00b880-fe00b8bf : fe00b880.mailbox mailbox#7e00b880
fe100000-fe100113 : fe100000.watchdog watchdog#7e100000
fe101000-fe102fff : fe101000.cprman cprman#7e101000
fe104000-fe104027 : fe104000.rng rng#7e104000
**fe200000-fe2000b3 : fe200000.gpio gpio#7e200000**
fe201000-fe2011ff : serial#7e201000
fe201000-fe2011ff : fe201000.serial serial#7e201000
fe204000-fe2041ff : fe204000.spi spi#7e204000
fe206000-fe2060ff : fe206000.pixelvalve pixelvalve#7e206000
fe207000-fe2070ff : fe207000.pixelvalve pixelvalve#7e207000
fe20a000-fe20a0ff : fe20a000.pixelvalve pixelvalve#7e20a000
fe215000-fe215007 : fe215000.aux aux#7e215000
fe216000-fe2160ff : fe216000.pixelvalve pixelvalve#7e216000
fe300000-fe3000ff : fe300000.mmcnr mmcnr#7e300000
fe340000-fe3400ff : fe340000.mmc mmc#7e340000
fe400000-fe407fff : fe400000.hvs hvs#7e400000
fec00000-fec03fff : fec00000.v3d hub
fec04000-fec07fff : fec00000.v3d core0
fec11000-fec1101f : fe100000.watchdog watchdog#7e100000
fec12000-fec120ff : fec12000.pixelvalve pixelvalve#7ec12000
fef00000-fef0000f : fef00000.clock clock#7ef00000
fef00b00-fef00dff : fef04500.i2c auto-i2c
fef04500-fef045ff : fef04500.i2c bsc
fef05b00-fef05dff : fef09500.i2c auto-i2c
fef09500-fef095ff : fef09500.i2c bsc
600000000-63fffffff : pcie#7d500000
600000000-6000fffff : PCI Bus 0000:01
600000000-600000fff : 0000:01:00.0
600000000-600000fff : xhci-hcd
Raspbeerry PI works under the Raspbian OS 32-bit version. Kernel version is 5.15.32-v7l+.
I tried different base addresses found from different sources:
0x20000000 - ioremap returns 0.
0x3e000000 - ioremap works, no errors, but values there make no sense and writing with this base address leads to no reaction at all.
0x7e000000 - ioremap returns 0.
0x7e000000 without ioremap - error "Unable to handle kernel paging request at virtual address ...".
According to different forums topics and examples my piece of code should just work, but it is not, so I'm out of ideas right now. Does anyone have any thoughts? Is this my error or something else is wrong?
Problem is solved. Nothing seems wrong with my code, 64-bit OS helped.
I didn't actually install 64-bit system, I added "arm_64bit=1" to my /boot/config.txt and updated all packages, because this was just for testing purposes, but this worked.

ARM + gcc: global destructors not called after main() returns, but constructors are

I am trying to write a simple "Hello, World!" firmware for Cortex-M0 CPU. The goal is to correctly initialize and shutdown C++ runtime so that global constructors are called before main() and global destructors are called after main().
So far I managed to get exactly half of it working - global ctors run correctly, but global destructors don't. I use gcc on Windows with the Newlib, which is supposed to initialize the runtime.
struct Test {
Test() // This is invoked correctly.
{}
~Test() // This is not invoked at all.
{}
};
Test test;
int main()
{
return 0;
}
extern void __libc_init_array();
extern void __libc_fini_array();
void reset_handler() // This is the entry point.
{
__libc_init_array(); // Test::Test().
asm volatile( "bl main" ); // Invoke 'main'.
__libc_fini_array(); // Expect Test::~Test() but nothing happens.
}
I did quite a bit of research already, and it seems that for ARM compiler should generate a section called .init_array, for global constructors, and .fini_array, for global destructors, where it put pointer to the function to be invoked. Then, the linker will merge the sections from all units together, and __libc_init_array will "walk" through that section and call corresponding functions.
This is the relevant part of the linker script:
/* Initialization functions which run before main(),
such as global constructors. */
.init_array : ALIGN( 4 ) {
/* preinit data */
PROVIDE_HIDDEN ( __preinit_array_start = . );
KEEP( *( .preinit_array ) )
PROVIDE_HIDDEN( __preinit_array_end = . );
. = ALIGN(4);
/* init data */
PROVIDE_HIDDEN ( __init_array_start = . );
KEEP( *( SORT( .init_array.* ) ) )
KEEP(*(.init_array))
PROVIDE_HIDDEN ( __init_array_end = . );
} > flash
/* Finalization functions which run after main(),
such as global destructors. */
.fini_array : ALIGN( 4 ) {
/* finit data */
PROVIDE_HIDDEN (__fini_array_start = .);
KEEP ( *( .fini_array.* ) )
KEEP ( *( .fini_array ) )
PROVIDE_HIDDEN (__fini_array_end = .);
} > flash
What worries me, however, is that when I dump the object file, main.o, not even an executable, I can only see the .init_array section, but no .fini_array:
Sections:
Idx Name Size VMA LMA File off Algn
0 .group 00000008 00000000 00000000 00000034 2**2
CONTENTS, READONLY, GROUP, LINK_ONCE_DISCARD
1 .group 00000008 00000000 00000000 0000003c 2**2
CONTENTS, READONLY, GROUP, LINK_ONCE_DISCARD
2 .text 0000022c 00000000 00000000 00000044 2**2
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
3 .data 00000000 00000000 00000000 00000270 2**0
CONTENTS, ALLOC, LOAD, DATA
4 .bss 00000001 00000000 00000000 00000270 2**2
ALLOC
5 .text._ZN4TestC2Ev 00000012 00000000 00000000 00000270 2**1 << Test::Test()
CONTENTS, ALLOC, LOAD, READONLY, CODE
6 .text._ZN4TestD2Ev 00000012 00000000 00000000 00000282 2**1 << Test::~Test()
CONTENTS, ALLOC, LOAD, READONLY, CODE
7 .init_array 00000004 00000000 00000000 00000294 2**2 << WHERE IS .fini_array???
CONTENTS, ALLOC, LOAD, RELOC, DATA
8 .debug_info 000012d9 00000000 00000000 00000298 2**0
CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
9 .debug_abbrev 000003a1 00000000 00000000 00001571 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
10 .debug_aranges 00000030 00000000 00000000 00001912 2**0
CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
11 .debug_ranges 00000020 00000000 00000000 00001942 2**0
CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
12 .debug_line 000002d2 00000000 00000000 00001962 2**0
CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
13 .debug_str 000009e4 00000000 00000000 00001c34 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
14 .comment 0000004d 00000000 00000000 00002618 2**0
CONTENTS, READONLY
15 .debug_frame 0000054c 00000000 00000000 00002668 2**2
CONTENTS, RELOC, READONLY, DEBUGGING, OCTETS
16 .ARM.attributes 0000002c 00000000 00000000 00002bb4 2**0
CONTENTS, READONLY
This is how I invoke the compiler:
arm-none-eabi-c++.exe -o main.elf --verbose -mcpu=cortex-m0 -mthumb --specs=nano.specs --entry reset_handler -T./../src/firmware.ld -ggdb -O0 -Wall -Wextra -Wpedantic -Werror ../src/main.cpp
And this is the log:
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 7a1ab17ae8404f635d46188cccacd8be
COLLECT_GCC_OPTIONS='-o' 'main.elf' '-v' '-mcpu=cortex-m0' '-mthumb' '-specs=nano.specs' '-e' 'reset_handler' '-T' './../src/firmware.ld' '-ggdb' '-O0' '-Wall' '-Wextra' '-Wpedantic' '-Werror' '-mfloat-abi=soft' '-march=armv6s-m'
c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/bin/as.exe -v -march=armv6s-m -mfloat-abi=soft -meabi=5
GNU assembler version 2.34.0 (arm-none-eabi) using BFD version (GNU Arm Embedded Toolchain 9-2020-q2-update) 2.34.0.20200428
COMPILER_PATH=c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/;c:/id/gcc/bin/../lib/gcc/;c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/bin/
LIBRARY_PATH=c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp/;c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/lib/thumb/v6-m/nofp/;c:/id/gcc/bin/../arm-none-eabi/lib/thumb/v6-m/nofp/;c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/;c:/id/gcc/bin/../lib/gcc/;c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/lib/;c:/id/gcc/bin/../arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-o' 'main.elf' '-v' '-mcpu=cortex-m0' '-mthumb' '-specs=nano.specs' '-e' 'reset_handler' '-T' './../src/firmware.ld' '-ggdb' '-O0' '-Wall' '-Wextra' '-Wpedantic' '-Werror' '-mfloat-abi=soft' '-march=armv6s-m'
c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/collect2.exe -plugin c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/liblto_plugin-0.dll -plugin-opt=c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/lto-wrapper.exe -plugin-opt=-fresolution=C:\Users\MAXID~1\AppData\Local\Temp\ccpOTkFN.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lg_nano -plugin-opt=-pass-through=-lc_nano -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lc_nano --sysroot=c:\id\gcc\bin\../arm-none-eabi -X -o main.elf -e reset_handler c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp/crti.o c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp/crtbegin.o c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/lib/thumb/v6-m/nofp/crt0.o -Lc:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp -Lc:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/lib/thumb/v6-m/nofp -Lc:/id/gcc/bin/../arm-none-eabi/lib/thumb/v6-m/nofp -Lc:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1 -Lc:/id/gcc/bin/../lib/gcc -Lc:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/lib -Lc:/id/gcc/bin/../arm-none-eabi/lib C:\Users\MAXID~1\AppData\Local\Temp\cc1lU5Gg.o -lstdc++_nano -lm --start-group -lgcc -lg_nano -lc_nano --end-group --start-group -lgcc -lc_nano --end-group c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp/crtend.o c:/id/gcc/bin/../lib/gcc/arm-none-eabi/9.3.1/thumb/v6-m/nofp/crtn.o -T ./../src/firmware.ld
COLLECT_GCC_OPTIONS='-o' 'main.elf' '-v' '-mcpu=cortex-m0' '-mthumb' '-specs=nano.specs' '-e' 'reset_handler' '-T' './../src/firmware.ld' '-ggdb' '-O0' '-Wall' '-Wextra' '-Wpedantic' '-Werror' '-mfloat-abi=soft' '-march=armv6s-m'
To be fair, I have zero idea of what could be wrong - I tried the default linker script, .fini_array did not appear.
The only idea which comes to mind is that my compiler was built with some kind of a flag which disabled the .fini_array section???
Any ideas will be much appreciated!
UPDATE: after further investigation, it seems that the destructors have more to do with __cxa_atexit... Will investigate further.
UPDATE2: Thanks to this wonderful discussion here https://forum.osdev.org/viewtopic.php?f=13&t=36728 I added -fno-use-cxa-atexit, and the .fini_array section appeared!
Ok, so for those who are interested, there are two ways for gcc to generate calls to global destructors. One way is via __cxa_atexit. The compiler will inject a piece of code into the global constructor, which uses __cxa_atexit to register the destructor of that object. This way, when exit is called, the runtime "knows" which destructors to invoke. This complexity is needed to deal with loading/unloading of shared libraries.
However, it is also possible to pass a -fno-cxa-atexit flag, and the compiler will put the calls to global destructors into .fini_array. The Newlib then uses this section to invoke the destructors.
More can be found here: https://forum.osdev.org/viewtopic.php?f=13&t=36728

RAM section is part of the binary firmware

I am trying to use a custom RAM section to be able to pass information across reboot. This section will not be erased at boot and so the variables placed in this section will be kept across reboots (if there is no alimentation loss of course).
I use GNU toolchain and a Cortex-M0 (STM32) MCU
So I added in the linker script a new memory area before RAM :
RAM_PERSIST (xrw) : ORIGIN = 0x20000000, LENGTH = 0x0040
RAM (xrw) : ORIGIN = 0x20000040, LENGTH = 0x0FD0
Then a section to go in there :
.pds :
{
KEEP(*(.pds))
} >RAM_PERSIST
Finally in the C code, I declare some data in this section :
data_t __attribute((section(".pds")) data;
I does compile but I could not upload the generated binary on my target. Using objdump I discovered that my firmware got a new section ".sec2" beginning at 0x20000000 :
> (...)/arm-none-eabi-objdump -s ./obj/firmware.hex | tail
8006d20 f8bc08bc 9e467047 f8b5c046 f8bc08bc .....FpG...F....
8006d30 9e467047 e9000008 c1000008 00127a00 .FpG..........z.
8006d40 19000000 e0930400 409c0000 400d0300 ........#...#...
8006d50 c0c62d00 30750000 ffffffff 01000000 ..-.0u..........
8006d60 04000000 ....
Contents of section .sec2:
20000000 00000000 00000000 00000000 00000000 ................
20000010 00000000 00000000 00000000 00000000 ................
20000020 00000000 00000000 00000000 00000000 ................
20000030 00000000 00000000 00000000 00000000 ................
So I think I have to tell the linker this section is not in the flash so must not be part of the firmware.
Am I right ? If so, how to do that ?
Thanks by advance.
MEMORY
{
rom : ORIGIN = 0x00000000, LENGTH = 0x40000
ram : ORIGIN = 0x20000000, LENGTH = 0x4000
}
SECTIONS
{
.text : { *(.text*) } > rom
.rodata : { *(.rodata*) } > rom
.bss : { *(.bss*) } > ram
}
I had more control/success when I stopped using xrw, etc in the memory definition and instead went with control over .text, .bss, .data, etc. and if you then further want a specific object somewhere you add that. etc...
I did achieve what I wanted by adding NOLOAD attibute to my custom section :
.pds (NOLOAD): { KEEP(*(.pds)) } >RAM
Here is the NOLOAD description (gcc documentation) :
(NOLOAD)
The (NOLOAD) directive will mark a section to not be loaded at run time. The linker will process the section normally, but will mark
it so that a program loader will not load it into memory. For example,
in the script sample below, the ROM section is addressed at memory
location 0 and does not need to be loaded when the program is run.
The contents of the ROM section will appear in the linker output file
as usual.
SECTIONS {
ROM 0 (NOLOAD) : { ... }
...
}
I found a similar post which helped me, I add a link here for reference : GCC (NOLOAD) directive loads memory into section anyway

system call hacking in linux for arm

I am trying to write a kernel module that will replace system calls for linux 4.9. All the solutions on the Internet are specific to x86 but I am working on Beaglebone Black that has an arm cortex A8.
This is what I have done so far.
static unsigned long *sys_call_table; // this is a global
The module when insmoded appears as a device in /dev which the user can open and give an ioctl command. In ioctl I use
sys_call_table=(void*)kallsyms_lookup_name("sys_call_table");
which obtains the same address as given in System.map file. But the moment I try to change system call using
*(sys_call_table + __NR_open) = (unsigned long)custom_open;
It gives errors. they are
[ 155.354417] Unable to handle kernel paging request at virtual address c01079f8
[ 155.361959] pgd = de6c0000
[ 155.364780] [c01079f8] *pgd=8000041e(bad)
[ 155.368981] Internal error: Oops: 80d [#1] SMP ARM
[ 155.373980] Modules linked in: intercept(O) [last unloaded: intercept]
[ 155.380821] CPU: 0 PID: 120 Comm: test Tainted: G O 4.9.39 #1
[ 155.387991] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 155.394342] task: de6b2380 task.stack: de664000
[ 155.399089] PC is at my_ioctl+0x64/0x94 [intercept]
[ 155.404180] LR is at my_ioctl+0x58/0x94 [intercept]
[ 155.409269] pc : [<bf0040dc>] lr : [<bf0040d0>] psr: 60000013
[ 155.409269] sp : de665f08 ip : 00000001 fp : bedb0c54
[ 155.421258] r10: 00000000 r9 : 00000003 r8 : 00000003
[ 155.426711] r7 : c02b354c r6 : de6293c0 r5 : de6dc2f0 r4 : bf004580
[ 155.433522] r3 : c01079e4 r2 : bf004000 r1 : ffffe000 r0 : bf004294
[ 155.440334] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 155.447773] Control: 10c5387d Table: 9e6c0019 DAC: 00000051
[ 155.453772] Process test (pid: 120, stack limit = 0xde664218)
[ 155.459771] Stack: (0xde665f08 to 0xde666000)
[ 155.464321] 5f00: bedb0dac c02b2aec 00000000 de6b2670 de665f7c c07ddc98
[ 155.472862] 5f20: 60000013 c0c0512c c0cbfe80 c0192d10 c0c8311c de611000 c029ddd8 c0cbf624
[ 155.481401] 5f40: 2ae98e92 00000024 2b36fb89 00000024 c07de574 df947010 de6293c8 de664000
[ 155.489938] 5f60: 00000000 00000000 de6293c0 de6293c0 00000005 bedb0dac 00000003 00000000
[ 155.498469] 5f80: bedb0c54 c02b354c 00000000 00000000 0001036c 00000036 c01079e4 de664000
[ 155.507005] 5fa0: 00000000 c0107840 00000000 00000000 00000003 00000005 bedb0dac 00010494
[ 155.515541] 5fc0: 00000000 00000000 0001036c 00000036 00000000 00000000 b6f12000 bedb0c54
[ 155.524075] 5fe0: b6e74d90 bedb0c44 000104bc b6e74d9c 60000010 00000003 00000000 00000000
[ 155.532633] [<bf0040dc>] (my_ioctl [intercept]) from [<c02b2aec>] (do_vfs_ioctl+0x90/0xa84)
[ 155.541357] [<c02b2aec>] (do_vfs_ioctl) from [<c02b354c>] (SyS_ioctl+0x6c/0x7c)
[ 155.548992] [<c02b354c>] (SyS_ioctl) from [<c0107840>] (ret_fast_syscall+0x0/0x1c)
[ 155.556898] Code: eb48ce6e e5943004 e59f2028 e59f0028 (e5832014)
[ 155.563276] ---[ end trace 0529de7e48dd6bb4 ]---
[ 155.571707] MyDevice closed
Segmentation fault
Please give me a solution specific to arm.

How can I see the full backtrace using kgdb to debug an ARM Linux module?

I worked my way through all of the free Linux training materials created by Free Electrons. In the last lab, we learn to use kgdb to remotely debug a simple crash in a loadable module. The crash is caused by a null pointer dereference in a memzero function call.
I am using Linux kernel 4.9 and a BeagleBone Black as the target, all according to the recommendations for the labs, and I've had no problems up to this point. My host is Ubuntu xenial and I am using standard packages for the ARM toolchain (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) and gdb (7.11.1-0ubuntu1~16.04) debugger.
gdb is able to read the symbol tables from vmlinux and from the module with the bug in it, which is called drvbroken.ko. The module has a bug in its init function, so it crashes immediately when I insmod it.
gdb output:
(gdb) backtrace
#0 __memzero () at arch/arm/lib/memzero.S:69
#1 0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) list 69
64 ldmeqfd sp!, {pc} # 1/2 quick exit
65 /*
66 * No need to correct the count; we're only testing bits from now on
67 */
68 tst r1, #32 # 1
69 stmneia r0!, {r2, r3, ip, lr} # 4
70 stmneia r0!, {r2, r3, ip, lr} # 4
71 tst r1, #16 # 1 16 bytes or more?
72 stmneia r0!, {r2, r3, ip, lr} # 4
73 ldr lr, [sp], #4 # 1
The result is the same whether I build the kernel with CONFIG_ARM_UNWIND (the default) or disable that and use CONFIG_FRAME_POINTER (the old method recommended by the lab notes).
I tried the same procedure in kdb, and here I see a very long backtrace that includes the calling functions. The caller of memzero is cdev_init.
kdb output:
Entering kdb (current=0xde616240, pid 106) on processor 0 Oops: (null)
due to oops # 0xc04c2be0
CPU: 0 PID: 106 Comm: insmod Tainted: G O 4.9.0-dirty #1
Hardware name: Generic AM33XX (Flattened Device Tree)
task: de616240 task.stack: de676000
PC is at __memzero+0x40/0x7c
LR is at 0x0
pc : [<c04c2be0>] lr : [<00000000>] psr: 00000013
sp : de677da4 ip : 00000000 fp : de677dbc
r10: bf000240 r9 : 219a3868 r8 : 00000000
r7 : de65c7c0 r6 : de6420c0 r5 : bf0000b4 r4 : 00000000
r3 : 00000000 r2 : 00000000 r1 : fffffffc r0 : 00000000
Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
Control: 10c5387d Table: 9e69c019 DAC: 00000051
CPU: 0 PID: 106 Comm: insmod Tainted: G O 4.9.0-dirty #1
Hardware name: Generic AM33XX (Flattened Device Tree)
Backtrace:
... pruned function calls related to kdb itself ...
[<c08326cc>] (do_page_fault) from [<c010138c>] (do_DataAbort+0x3c/0xbc)
r10:bf000240 r9:de676000 r8:de677d50 r7:00000000 r6:c08326cc r5:00000817
r4:c0d0bb2c
[<c0101350>] (do_DataAbort) from [<c0831d04>] (__dabt_svc+0x64/0xa0)
Exception stack(0xde677d50 to 0xde677d98)
7d40: 00000000 fffffffc 00000000 00000000
7d60: 00000000 bf0000b4 de6420c0 de65c7c0 00000000 219a3868 bf000240 de677dbc
7d80: 00000000 de677da4 00000000 c04c2be0 00000013 ffffffff
r8:00000000 r7:de677d84 r6:ffffffff r5:00000013 r4:c04c2be0
[<c02bf44c>] (cdev_init) from [<bf002048>] (init_module+0x48/0xb4 [drvbroken])
r5:bf002000 r4:bf000480
[<bf002000>] (init_module [drvbroken]) from [<c01018d4>] (do_one_initcall+0x44/0x180)
r5:bf002000 r4:ffffe000
[<c0101890>] (do_one_initcall) from [<c024fa2c>] (do_init_module+0x64/0x1d8)
r8:00000001 r7:de65c7c0 r6:de6420c0 r5:c0dbfa84 r4:bf000240
[<c024f9c8>] (do_init_module) from [<c01e10e8>] (load_module+0x1d6c/0x23d8)
r6:c0d0512c r5:c0dbfa84 r4:c0d4c70f
[<c01df37c>] (load_module) from [<c01e18ac>] (SyS_init_module+0x158/0x17c)
r10:00000051 r9:de676000 r8:e0a95100 r7:00000000 r6:000ac118 r5:00004100
It is pretty easy to figure out where to look for the bug with this information, but alas, it is not possible to get a line number or list the source directly from kdb. This is much easier in gdb, assuming that I can get a full backtrace.

Resources