I don't know about kernel, I don't know how to troubleshoot.
When the problem happened, my system kernel crashed, the following is the vmcore-dmesg.txt log
##dmesg logs
[ 378.442884] SPDMD-LUN:[ERROR]lun_del:2218 device nvfile-mgmtd-0 is busy :2
[ 424.511211] XFS (nvfile-storage-0): Mounting V5 Filesystem
[ 424.513045] XFS (nvfile-storage-2): Mounting V5 Filesystem
[ 424.538953] XFS (nvfile-storage-2): Starting recovery (logdev: internal)
[ 424.546536] XFS (nvfile-storage-2): Ending recovery (logdev: internal)
[ 425.512217] XFS (nvfile-storage-0): Starting recovery (logdev: internal)
[ 425.518929] XFS (nvfile-storage-1): Mounting V5 Filesystem
[ 425.520813] XFS (nvfile-storage-3): Mounting V5 Filesystem
[ 426.288987] XFS (nvfile-storage-3): Starting recovery (logdev: internal)
[ 426.475465] XFS (nvfile-storage-1): Starting recovery (logdev: internal)
[ 427.496551] XFS (nvfile-storage-0): Ending recovery (logdev: internal)
[ 428.320932] XFS (nvfile-storage-3): Ending recovery (logdev: internal)
[ 428.977479] XFS (nvfile-storage-1): Ending recovery (logdev: internal)
[ 445.927118] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[ 445.927685] PGD 0 P4D 0
[ 445.928216] Oops: 0000 [#1] SMP NOPTI
[ 445.928723] CPU: 4 PID: 591 Comm: kworker/u193:1 Kdump: loaded Tainted: G OE --------- - - 4.18.0-305.el8.x86_64 #1
[ 445.929735] Hardware name: Lenovo ThinkSystem SR860 V2/7Z60CTO1WW, BIOS M5E118K-1.52 08/06/2021
[ 445.930305] Workqueue: xfs-cil/nvfile-storage- xlog_cil_push_work [xfs]
[ 445.930910] RIP: 0010:blk_queue_split+0x1c6/0x660
[ 445.933048] Code: c0 45 31 f6 89 44 24 44 89 7c 24 40 31 ff 85 db 0f 84 56 04 00 00 8b 44 24 44 48 89 c1 48 89 44 24 30 48 c1 e1 04 49 03 4d 78 <8b> 41 08 8b 71 0c 48 8b 11 44 29 f8 39 d8 48 89 54 24 48 0f 47 c3
[ 445.936323] RSP: 0018:ffffbcbf9a4bfaf8 EFLAGS: 00010246
[ 445.936711] RAX: 0000000000000000 RBX: 0000000000003000 RCX: 0000000000000000
[ 445.937028] RDX: 0000000000000018 RSI: 0000000000000018 RDI: 0000000000000000
[ 445.937343] RBP: ffffbcbf9a4bfb90 R08: 0000000000000000 R09: ffff940c1965d700
[ 445.937659] R10: 0000000000000000 R11: 0000000000000000 R12: ffff94ca6f590000
[ 445.937979] R13: ffff940bda0ddc80 R14: 0000000000000000 R15: 0000000000000000
[ 445.938304] FS: 0000000000000000(0000) GS:ffff940effd00000(0000) knlGS:0000000000000000
Im learning to develop and backport drivers and Im new to this.
I am wondering how to debug this issue. I found that there is some usage of NULL pointer but I dont know how to find it. This debug trace doesn't shows me where it is in the driver code. And I dont know how to start.
I succeeded to backport mcp251xfd driver to 4.9 linux and it works, I succeeded to read data with candump but when I do reboot this happens: ( and sometimes happens on bringing up the interface)
[ 35.383520] Unable to handle kernel NULL pointer dereference at virtual address 000000c8
[ 35.392667] pgd = ffffff80094a1000
[ 35.396502] [000000c8] *pgd=000000007f7fe003, *pud=000000007f7fe003, *pmd=0000000000000000
[ 35.405833] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[ 35.412105] Modules linked in: vin_v4l2 ov8858_r2a_4lane vin_io videobuf2_dma_contig gt9xxnew_ts sprdbt_tty sprdwl_ng uwe5622_bsp_sdio sunxi_gmac
[ 35.426863] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.9.170 #2
[ 35.432909] NOHZ: local_softirq_pending 08
[ 35.433034] NOHZ: local_softirq_pending 08
[ 35.433125] NOHZ: local_softirq_pending 08
[ 35.447419] Hardware name: sun50iw10 (DT)
[ 35.451931] task: ffffffc03e1cc880 task.stack: ffffffc03e1e0000
[ 35.458608] PC is at test_and_set_bit+0x18/0x40
[ 35.463713] LR is at __netif_schedule+0x28/0x80
[ 35.468811] pc : [<ffffff80083fddb8>] lr : [<ffffff80087e5588>] pstate: 40400145
[ 35.477129] sp : ffffffc03f733d70
[ 35.480856] x29: ffffffc03f733d70 x28: 000000000002536] 3cf0 3b36d4a0 ffffffc0 3b2f6c80 ffffffc0 00000003 00000000 ffffffff 00000000
[ 35.591815] 3d10 0000000a 00000000 08e8f018 ffffff80 00000001 00000000 3f733d70 ffffffc0
[ 35.601089] 3d30 087e5588 ffffff80 3f733d70 ffffffc0 083fddb8 ffffff80 40400145 00000000
[ 35.610363] 3d50 3b36d000 ffffffc0 00000001 00000000 ffffffff 0000007f 3b36d4a0 ffffffc0
[ 35.619637] 3d70 3f733da0 ffffffc0 087e5610 ffffff80 3b2f6c00 ffffffc0 00000001 00000000
[ 35.628911] 3d90 3b2f6c00 ffffffc0 087e55fc ffffff80 3f733dc0 ffffffc0 088162f0 ffffff80
[ 35.638185] 3db0 3b36d000 ffffffc0 00000001 00000000 3f733e10 ffffffc0 08114920 ffffff80
[ 35.647459] 3dd0 00000101 00000000 3b36d4b8 ffffffc0 088161f4 ffffff80 088161f4 ffffff80
[ 35.656742]
[ 35.656742] X6: 0xffffffc03f733e68:
[ 35.662328] 3e68 08e93000 ffffff80 0920d000 ffffff80 09206000 ffffff80 09210000 ffffff80
[ 35.671603] 3e88 ffffffff 00000000 00000101 00000000 08114c1c ffffff80 3f733f30 ffffffc0
[ 35.680876] 3ea8 08081438 ffffff80 09205188 ffffff80 00000001 00000000 00000002 00000000
[ 35.690149] 3ec8 09412e80 ffffff80 00000282 00000000 08e8f018 ffffff80 00000000 00000000
[ 35.699422] 3ee8 0841cb98 ffffff80 09205188 ffffff80 00000001 00000000 00000002 00000000
[ 35.708694] 3f08 09412e80 ffffff80 00000282 00000000 08081328 ffffff80 3f733f30 ffffffc0
[ 35.717968] 3f28 000409aa 00000000 3f733fc0 ffffffc0 080aa3a8 ffffff80 09412e80 ffffff80
[ 35.727242] 3f48 00000005 00000000 08cf4000 ffffff80 00000000 00000000 08e98000 ffffff80
[ 35.736535]
[ 35.736535] X21: 0xffffffc03b2f6b80:
[ 35.742218] 6b80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.751490] 6ba0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.760762] 6bc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.770034] 6be0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000 00000000 00000000 00000000
[ 35.831355] d440 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.840625] d460 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.849897] d480 00000000 00000000 00000000 00000000 00000000 00000000 00000064 00000000
[ 35.859169] d4a0 00310030 0000012c 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.868441] d4c0 3f734ff0 ffffffc0 fffecaff 00000000 088161f4 ffffff80 3b36d000 ffffffc0
[ 35.877714] d4e0 18000003 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.886987] d500 08ea3fdc ffffff80 00000000 00000000 00000000 00000000 3b36d518 ffffffc0
[ 35.896264]
[ 35.896264] X23: 0xffffffc03b2f6c00:
[ 35.901947] 6c00 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.911219] 6c20 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.920491] 6c40 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.929763] 6c60 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.939038] 6c80 00010001 ffffffff 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.948310] 6ca0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.957583] 6cc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.966856] 6ce0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 35.976137]
[ 35.976137] X29: 0xffffffc03f733cf0:
[ 35.981821] 3cf0 3b36d4a0 ffffffc0 3b2f6c80 ffffffc0 00000003 00000000 ffffffff 00000000
[ 35.991093] 3d10 0000000a 00000000 08e8f018 ffffff80 00000001 00000000 3f733d70 ffffffc0
[ 36.000367] 3d30 087e5588 ffffff80 3f733d70 ffffffc0 083fddb8 ffffff80 40400145 00000000
[ 36.009640] 3d50 3b36d000 ffffffc0 00000001 00000000 ffffffff 0000007f 3b36d4a0 ffffffc0
[ 36.018914] 3d70 3f733da0 ffffffc0 087e5610 ffffff80 3b2f6c00 ffffffc0 00000001 00000000
[ 36.028187] 3d90 3b2f6c00 ffffffc0 087e55fc ffffff80 3f733dc0 ffffffc0 088162f0 ffffff80
[ 36.037464] 3db0 3b36d000 ffffffc0 00000001 00000000 3f733e10 ffffffc0 08114920 ffffff80
[ 36.046737] 3dd0 00000101 00000000 3b36d4b8 ffffffc0 088161f4 ffffff80 088161f4 ffffff80
[ 36.056010]
[ 36.057689] Process swapper/3 (pid: 0, stack limit = 0xffffffc03e1e0000)
[ 36.065231] Stack: (0xffffffc03f733d70 to 0xffffffc03e1e4000)
[ 36.071694] Call trace:
[ 36.074449] Exception stack(0xffffffc03f733b80 to 0xffffffc03f733cb0)
[ 36.081705] 3b80: 0000000000000000 0000007fffffffff 00000000414a3000 ffffff80083fddb8
[ 36.090520] 3ba0: 0000000040400145 ffffffc03e21dda0 0000000000000000 ffffffc03f73b780
[ 36.099336] 3bc0: ffffffc03f733d30 0000000000000000 ffffffc03f733c00 0000000000000000
[ 36.108151] 3be0: 0000000000000000 0000000000000000 0000000000000000 0000000000000400
[ 36.116966] 3c00: 0000000000000018 0000000100000000 0000000000000001 0000000000000000
[ 36.125781] 3c20: 0000000000000000 ffffffc03e21df00 0000000000000006 00000000000409aa
[ 36.134597] 3c40: 0000000000000000 00000000000000c8 0000000000000001 0000000000000000
[ 36.143413] 3c60: 0000000000000001 000000000000000f ffffffc03f733ee8 0800000000000000
[ 36.152227] 3c80: 0000000000000001 0000000000000000 0000000000000004 000000000000000f
[ 36.161037] 3ca0: 0000000000000000 000000000000004c
[ 36.166536] [<ffffff80083fddb8>] test_and_set_bit+0x18/0x40
[ 36.172813] [<ffffff80087e5610>] netif_schedule_queue+0x30/0x40
[ 36.179482] [<ffffff80088162f0>] dev_watchdog+0xfc/0x278
[ 36.185467] [<ffffff8008114920>] call_timer_fn+0xa4/0x1c0
[ 36.191546] [<ffffff8008114b8c>] expire_timers+0x124/0x170
[ 36.197723] [<ffffff8008114c90>] run_timer_softirq+0xb8/0x188
[ 36.204192] [<ffffff8008081438>] __do_softirq+0x178/0x338
[ 36.210272] [<ffffff80080aa3a8>] irq_exit+0x90/0xd0
[ 36.215767] [<ffffff80000012 0000000000000000 000000000000002d
[ 36.270918] 3ea0: 000000000000001f 0000000000000000 ffffff800821c950 0000007faa340698
[ 36.279734] 3ec0: 0000007faa3caa70 ffffff800920df48 0000000000000003 0000000000000008
[ 36.288549] 3ee0: ffffff800920dfdc ffffffc03e1cc880 0000000000000000 0000000000000000
[ 36.297363] 3f00: 0000000000000000 0000000000000000 0000000000000000 ffffffc03e1e3f60
[ 36.306179] 3f20: ffffff800808577c ffffffc03e1e3f60 ffffff8008085780 0000000060400145
[ 36.314995] 3f40: ffffffc03e1cc880 ffffff800810c27c ffffffffffffffff ffffff8008085700
[ 36.323808] [<ffffff8008082b68>] el1_irq+0xe8/0x18c
[ 36.329302] [<ffffff8008085780>] arch_cpu_idle+0x98/0x178
[ 36.335385] [<ffffff80089fed18>] default_idle_call+0x28/0x30
[ 36.341759] [<ffffff80080f2394>] cpu_startup_entry+0xc4/0x104
[ 36.348231] [<ffffff8008091034>] secondary_start_kernel+0x1d0/0x1dc
[ 36.355285] [<0000000040a011b4>] 0x40a011b4
[ 36.360002] Code: d2800022 8b400c21 f9800031 9ac32044 (c85f7c22)
[ 36.366860] ---[ end trace a8e12e41d725d1cb ]---
[ 36.372057] Kernel panic - not syncing: Fatal exception in interrupt
[ 36.379202] SMP: stopping secondary CPUs
[ 36.383621] Kernel Offset: disabled
[ 36.387563] Memory Limit: none
[ 36.391016] Rebooting in 5 seconds..
Firstly I wasnt able to bring up the interface with similar problem and I readed that this is buffer overflow or double free. From the previous dump I saw in which function the problem happens and I commented this line -> kfree(ram); # 1034 line # this file --> https://github.com/minima34/mcp251xfd_4.9/blob/master/mcp251xfd/mcp251xfd-core.c
This allowed me to get my driver working but now I have this nasty issue, that blocked me and freaked me. This time I can't see where the problem happens and its confusing.
I think that the problem is that I commented kfree() in 1034 line but I dont know how to do it then.
I'm trying to boot a Debian Wheezy Image, Ker 3.8 on my BeagleCore (a smaller version of BeagleBone) with TI AM335x Cortex-A8 processor.
I took the Debian Image from beagleboard site.
When I try to boot, on a serial interface for debug, I get this messages:
U-Boot SPL 2016.01-00001-g4eb802e (Jan 13 2016 - 11:14:31)
Trying to boot from MMC
bad magic
U-Boot 2016.01-00001-g4eb802e (Jan 13 2016 - 11:14:31 -0600), Build: jenkins-github_Bootloader-Builder-313
Watchdog enabled
I2C: ready
DRAM: 512 MiB
Reset Source: Power-on reset has occurred.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
Using default environment
Net: <ethaddr> not set. Validating first E-fuse MAC
Could not get PHY for cpsw: addr 0
cpsw, usb_ether
Press SPACE to abort autoboot in 2 seconds
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
gpio: pin 56 (gpio 56) value is 0
gpio: pin 55 (gpio 55) value is 0
gpio: pin 54 (gpio 54) value is 0
gpio: pin 53 (gpio 53) value is 1
switch to partitions #0, OK
mmc0 is current device
gpio: pin 54 (gpio 54) value is 1
Checking for: /uEnv.txt ...
Checking for: /boot.scr ...
Checking for: /boot/boot.scr ...
Checking for: /boot/uEnv.txt ...
gpio: pin 55 (gpio 55) value is 1
2181 bytes read in 16 ms (132.8 KiB/s)
Loaded environment from /boot/uEnv.txt
Checking if uname_r is set in /boot/uEnv.txt...
gpio: pin 56 (gpio 56) value is 1
Running uname_boot ...
loading /boot/vmlinuz-3.8.13-bone79 ...
5644336 bytes read in 333 ms (16.2 MiB/s)
loading /boot/dtbs/3.8.13-bone79/am335x-boneblack.dtb ...
26118 bytes read in 24 ms (1 MiB/s)
loading /boot/initrd.img-3.8.13-bone79 ...
2905600 bytes read in 179 ms (15.5 MiB/s)
debug: [console=ttyO0,115200n8 capemgr.enable_partno=BB-UART1,BB-UART2,BB-UART4,BB-UART5 capemgr.disable_partno=BB-BONELT-HDMI,BB-BONELT-HDMIN root=UUID=4d8c9d4c-a16d-47ac-a32c-43d0155df072 ro rootfstype=ext4 rootwait coherent_pool=1M quiet init=/lib/systemd/systemd cape_universal=enable] ...
debug: [bootz 0x82000000 0x88080000:2c5600 0x88000000] ...
Kernel image # 0x82000000 [ 0x000000 - 0x562030 ]
## Flattened Device Tree blob at 88000000
Booting using the fdt blob at 0x88000000
Loading Ramdisk to 8fd3a000, end 8ffff600 ... OK
Loading Device Tree to 8fd30000, end 8fd39605 ... OK
Starting kernel ...
Uncompressing Linux... done, booting the kernel.
[ 0.384810] omap2_mbox_probe: platform not supported
[ 0.540541] tps65217-bl tps65217-bl: no platform data provided
[ 0.604330] bone-capemgr bone_capemgr.9: slot #0: No cape found
[ 0.641437] bone-capemgr bone_capemgr.9: slot #1: No cape found
[ 0.678546] bone-capemgr bone_capemgr.9: slot #2: No cape found
[ 0.715656] bone-capemgr bone_capemgr.9: slot #3: No cape found
[ 0.741854] omap_hsmmc mmc.5: of_parse_phandle_with_args of 'reset' failed
[ 0.803809] pinctrl-single 44e10800.pinmux: pin 44e10854 already requested by 44e10800.pinmux; cannot claim for gpio-leds.8
[ 0.815463] pinctrl-single 44e10800.pinmux: pin-21 (gpio-leds.8) status -22
[ 0.822748] pinctrl-single 44e10800.pinmux: could not request pin 21 on device pinctrl-single
[ 0.893233] Unhandled fault: external abort on non-linefetch (0x1008) at 0xe0858c20
[ 0.901225] Internal error: : 1008 [#1] SMP THUMB2
[ 0.906217] Modules linked in:
[ 0.909405] CPU: 0 Not tainted (3.8.13-bone79 #1)
[ 0.914691] PC is at cpts_fifo_read.constprop.1+0x18/0xc4
[ 0.920317] LR is at cpts_systim_read+0x11/0x7c
[ 0.925040] pc : [<c0326468>] lr : [<c0326761>] psr: 000001b3
[ 0.925040] sp : df071db8 ip : 00000000 fp : de231664
[ 0.936993] r10: de231000 r9 : de231758 r8 : c084e0c0
[ 0.942440] r7 : 00000001 r6 : ffffffff r5 : 00000010 r4 : de231670
[ 0.949241] r3 : e0858c00 r2 : 00000001 r1 : de2316d0 r0 : de231670
[ 0.956039] Flags: nzcv IRQs off FIQs on Mode SVC_32 ISA Thumb Segment kernel
[ 0.963925] Control: 50c5387d Table: 80004019 DAC: 00000015
[ 0.969907] Process swapper/0 (pid: 1, stack limit = 0xdf070240)
[ 0.976163] Stack: (0xdf071db8 to 0xdf072000)
[ 0.980699] 1da0: e0858c00 de2316d0
[ 0.989219] 1dc0: de2316bc 35318bf5 00000000 0000001d c052e7a8 c0326761 de2316e8 de2316bc
[ 0.997740] 1de0: 35318bf5 c00611f1 de231670 20000113 de2316e8 c0326927 35318bf5 00000000
[ 1.006259] 1e00: 00000000 00000004 df0d5410 de231000 df0d5400 c0325bab df0d8ac0 de231540
[ 1.014775] 1e20: c0893bb8 0000002b de231540 df0d5400 df0d5410 00000005 00000000 df0d5410
[ 1.023298] 1e40: e0858800 e0858a00 e0858a20 e0858a40 e0858a60 e08588c0 e08588e0 00000008
[ 1.031813] 1e60: 00000001 0000003c 4a102000 4a102000 00002000 00000010 00000001 de231298
[ 1.040338] 1e80: e0858d00 0000000a 00000400 00000002 00000020 00000008 df0d5410 c094362c
[ 1.048868] 1ea0: df0d5410 c08b2c40 00000000 c0829039 00000102 c0846d70 00000000 c02c82b1
[ 1.057381] 1ec0: c02c82a1 c02c7753 00000000 df0d5410 c08b2c40 df0d5444 00000000 c02c78b3
[ 1.065896] 1ee0: c08b2c40 c02c7869 00000000 c02c6887 df049478 df0c6180 c08b2c40 c08a8090
[ 1.074421] 1f00: de23d140 c02c7247 c0753554 c08b2c40 c08b2c40 df070000 c08d4180 00000000
[ 1.082937] 1f20: c0829039 c02c7bb5 00000000 c0833968 df070000 c08d4180 00000000 c0829039
[ 1.091461] 1f40: 00000102 c000867f 00000007 00000007 c088bc98 c0833964 c0833968 00000007
[ 1.099978] 1f60: c0833948 c08d4180 c080d1c9 c0846d70 00000000 c080d6a3 00000007 00000007
[ 1.108503] 1f80: c080d1c9 c0d60fc0 00000000 c04ccfb1 00000000 00000000 00000000 00000000
[ 1.117013] 1fa0: 00000000 c04ccfb7 00000000 c000c8fd 00000000 00000000 00000000 00000000
[ 1.125537] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.134055] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[ 1.142587] [<c0326468>] (cpts_fifo_read.constprop.1+0x18/0xc4) from [<c0326761>] (cpts_systim_read+0x11/0x7c)
[ 1.153018] [<c0326761>] (cpts_systim_read+0x11/0x7c) from [<c00611f1>] (timecounter_init+0x11/0x1c)
[ 1.162545] [<c00611f1>] (timecounter_init+0x11/0x1c) from [<c0326927>] (cpts_register+0xf3/0x1b8)
[ 1.171894] [<c0326927>] (cpts_register+0xf3/0x1b8) from [<c0325bab>] (cpsw_probe+0x823/0x960)
[ 1.180877] [<c0325bab>] (cpsw_probe+0x823/0x960) from [<c02c82b1>] (platform_drv_probe+0x11/0x14)
[ 1.190222] [<c02c82b1>] (platform_drv_probe+0x11/0x14) from [<c02c7753>] (driver_probe_device+0x53/0x168)
[ 1.200282] [<c02c7753>] (driver_probe_device+0x53/0x168) from [<c02c78b3>] (__driver_attach+0x4b/0x4c)
[ 1.210093] [<c02c78b3>] (__driver_attach+0x4b/0x4c) from [<c02c6887>] (bus_for_each_dev+0x27/0x48)
[ 1.219521] [<c02c6887>] (bus_for_each_dev+0x27/0x48) from [<c02c7247>] (bus_add_driver+0xe3/0x168)
[ 1.228949] [<c02c7247>] (bus_add_driver+0xe3/0x168) from [<c02c7bb5>] (driver_register+0x3d/0xc4)
[ 1.238289] [<c02c7bb5>] (driver_register+0x3d/0xc4) from [<c000867f>] (do_one_initcall+0x1f/0xf4)
[ 1.247630] [<c000867f>] (do_one_initcall+0x1f/0xf4) from [<c080d6a3>] (kernel_init_freeable+0xc3/0x158)
[ 1.257516] [<c080d6a3>] (kernel_init_freeable+0xc3/0x158) from [<c04ccfb7>] (kernel_init+0x7/0x98)
[ 1.266951] [<c04ccfb7>] (kernel_init+0x7/0x98) from [<c000c8fd>] (ret_from_fork+0x11/0x34)
[ 1.275659] Code: 2701 f100 09e8 6823 (6a1a) 07d3
[ 1.280655] ---[ end trace b2036333b4d03ad2 ]---
[ 1.285687] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
With a Debian Jessie image, Ker 4.4, board is booting normally.
Any idea how to solve this, is kindly appreciated.
Thank you.
As per my knowledge there was some issue with CTPS driver in kernel V3.13 (forget the exact kernel version). So it was an open issue. May be with newer kernel version they have fixed it. If you have the source code then try by disabling the CTPS driver CONFIG_TI_CPTS=n.
I wrote a kernel module which works as expected. But I want that to be loaded at the beginning of the boot process. So I moved this code to
OpenWRT/build_dir/target-i386_geode_eglibc-2.19/linux-x86_alix2/linux-3.10.49/arch/x86/platform
and my code is here:
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/gpio.h>
#include <linux/delay.h>
#include <linux/cs5535.h>
MODULE_AUTHOR("Ramana");
MODULE_DESCRIPTION("POWER LED DRIVER");
#ifdef MODULE_LICENSE
MODULE_LICENSE("Dual BSD/GPL");
#endif
#define HW_VERSION_GPIO 15
#define LATCH_GPIO 6
#define DATA_GPIO 25
#define CLOCK_GPIO 27
#define HIGH 1
#define LOW 0
static void set_power_led(void)
{
uint8_t i = 0;
/*
* Configure Pins Q8 Q7 Q6....Q0 in shift register
* Set 0 to glow LED
*
* shift_reg:
* indices 0, 1 and 2 are for LED3
* 0 1 1 -> Blue_ON, GREEN_OFF, RED_OFF
*
* indices 3, 4 and 5 are for LED1
*
* indices 6, 7 and 8 are for LED2
* 0 1 1 -> Blue_ON, GREEN_OFF, RED_OFF
*
* The pins Q9 Q10 and Q11 are don't care, so we are not using here
*/
uint8_t shift_reg[9] = {0, 1, 1, 1, 1, 1, 1, 1, 1};
/*
* Clear register before set
*/
for (i = 0; i < 9 ; i ++) {
gpio_set_value(CLOCK_GPIO, LOW);
if (shift_reg[i] == 0) {
gpio_set_value(DATA_GPIO, HIGH);
} else {
gpio_set_value(DATA_GPIO, LOW);
}
gpio_set_value(CLOCK_GPIO, HIGH);
}
gpio_set_value(LATCH_GPIO, HIGH);
msleep(1);
gpio_set_value(LATCH_GPIO, LOW);
}
static int __init power_led_init(void)
{
/*
* If GPIO 15 is high, it is old hardware
*/
printk(KERN_INFO "LED INIT\n");
if (!gpio_is_valid(LATCH_GPIO)) {
printk(KERN_INFO "LEDs: Latch gpio is not valid\n");
return -ENODEV;
}
if (!gpio_is_valid(DATA_GPIO)) {
printk(KERN_INFO "LEDs: Data gpio is not valid\n");
return -ENODEV;
}
if (!gpio_is_valid(CLOCK_GPIO)) {
printk(KERN_INFO "LEDs: Clock gpio is not valid\n");
return -ENODEV;
}
gpio_request(LATCH_GPIO, "sysfs");
gpio_request(DATA_GPIO, "sysfs");
gpio_request(CLOCK_GPIO, "sysfs");
gpio_direction_output(LATCH_GPIO, LOW);
gpio_direction_output(DATA_GPIO, LOW);
gpio_direction_output(CLOCK_GPIO, LOW);
set_power_led();
printk(KERN_INFO "Power LED: registered\n");
return 0;
}
static void __exit power_led_exit(void)
{
uint8_t i;
for (i = 0; i < 9 ; i++) {
gpio_set_value(CLOCK_GPIO, LOW);
gpio_set_value(DATA_GPIO, LOW);
gpio_set_value(CLOCK_GPIO, HIGH);
}
gpio_set_value(LATCH_GPIO, HIGH);
msleep(1);
gpio_set_value(LATCH_GPIO, LOW);
}
module_init(power_led_init);
module_exit(power_led_exit);
With this there is a kernel panic:
[ 0.104709] LED INIT
[ 0.105306] BUG: unable to handle kernel NULL pointer dereference at 0000004c
[ 0.106284] IP: [<c115acc2>] __gpio_set_value+0x12/0x80
[ 0.106284] *pde = 00000000
[ 0.106284] Oops: 0000 [#1]
[ 0.106284] Modules linked in:
[ 0.106284] CPU: 0 PID: 1 Comm: swapper Not tainted 3.10.49 #33
[ 0.106284] task: cf834000 ti: cf840000 task.ti: cf840000
[ 0.106284] EIP: 0060:[<c115acc2>] EFLAGS: 00010286 CPU: 0
[ 0.106284] EIP is at __gpio_set_value+0x12/0x80
[ 0.106284] EAX: c13a1624 EBX: c13a1624 ECX: ffffffea EDX: 00000000
[ 0.106284] ESI: 00000000 EDI: 00000000 EBP: cf841f80 ESP: cf841f24
[ 0.106284] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 0.106284] CR0: 8005003b CR2: 0000004c CR3: 01371000 CR4: 00000090
[ 0.106284] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 0.106284] DR6: ffff0ff0 DR7: 00000400
[ 0.106284] Stack:
[ 0.106284] cf841f3b cf841f44 0000003e c133b05e c12b0d03 006e6967 01010101 01010101
[ 0.106284] 00000000 c133afbb c1000172 cfdff401 00060006 c13019c0 c12ced19 cfdff460
[ 0.106284] 00000000 cfdff460 00000200 c114956a c136c480 00000006 0000003e cf840000
[ 0.106284] Call Trace:
[ 0.106284] [<c133b05e>] ? power_led_init+0xa3/0x112
[ 0.106284] [<c133afbb>] ? alix_init+0xf6/0xf6
[ 0.106284] [<c1000172>] ? do_one_initcall+0xb2/0x150
[ 0.106284] [<c114956a>] ? strcpy+0xa/0x20
[ 0.106284] [<c132da22>] ? kernel_init_freeable+0xd1/0x173
[ 0.106284] [<c132d4aa>] ? do_early_param+0x77/0x77
[ 0.106284] [<c124ad68>] ? kernel_init+0x8/0x170
[ 0.106284] [<c1251322>] ? ret_from_kernel_thread+0x6/0x28
[ 0.106284] [<c1251337>] ? ret_from_kernel_thread+0x1b/0x28
[ 0.106284] [<c124ad60>] ? rest_init+0x60/0x60
[ 0.106284] Code: d6 ab aa aa aa ff d1 5b 5e 5f c3 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 57 f
[ 0.106284] EIP: [<c115acc2>] __gpio_set_value+0x12/0x80 SS:ESP 0068:cf841f24
[ 0.106284] CR2: 000000000000004c
[ 0.106284] ---[ end trace 23021a4cac17faa2 ]---
[ 0.107751] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[ 0.107751]
Why this is crashing here and where can I add this to make this module loaded at the earliest.
Full minicom log
PC Engines ALIX.3 v0.99h
640 KB Base Memory
153603174448128645128089697280113664130048146432162816179200195584211968228352244736261120 KB Extended Memory
01F0 Master 045A InnoDisk Corp. - iCF4000 8GB
Phys C/H/S 16000/16/63 Log C/H/S 1003/255/63 LBA
GRUB loading....
Booting `OpenWrt'
[ 0.000000] Linux version 3.10.49 (savari#Ramana) (gcc version 4.8.3 (OpenWrt/Linaro GCC 4.8-2014.04 unknown) ) #40 Tue Nov 8 13:11:49 IST 2016
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000000fffffff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000fff00000-0x00000000ffffffff] reserved
[ 0.000000] Notice: NX (Execute Disable) protection missing in CPU!
[ 0.000000] DMI not present or invalid.
[ 0.000000] e820: last_pfn = 0x10000 max_arch_pfn = 0x100000
[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[ 0.000000] init_memory_mapping: [mem 0x0fc00000-0x0fffffff]
[ 0.000000] init_memory_mapping: [mem 0x08000000-0x0fbfffff]
[ 0.000000] init_memory_mapping: [mem 0x00100000-0x07ffffff]
[ 0.000000] 256MB LOWMEM available.
[ 0.000000] mapped low ram: 0 - 10000000
[ 0.000000] low ram: 0 - 10000000
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
[ 0.000000] Normal [mem 0x01000000-0x0fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00001000-0x0009ffff]
[ 0.000000] node 0: [mem 0x00100000-0x0fffffff]
[ 0.000000] Using APIC driver default
[ 0.000000] No local APIC present or hardware disabled
[ 0.000000] APIC: disable apic facility
[ 0.000000] APIC: switched to apic NOOP
[ 0.000000] e820: [mem 0x10000000-0xffefffff] available for PCI devices
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 64927
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz block2mtd.block2mtd=/dev/hda2,131072,rootfs,5 root=/dev/mtdblock0 rootfstype=jffs2 rootwait console=tty0 console=ttyS0,38400n8 noinitrd
[ 0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Memory: 255600k/262144k available (2377k kernel code, 6156k reserved, 873k data, 260k init, 0k highmem)
[ 0.000000] virtual kernel memory layout:
[ 0.000000] fixmap : 0xfffa3000 - 0xfffff000 ( 368 kB)
[ 0.000000] vmalloc : 0xd0800000 - 0xfffa1000 ( 759 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xd0000000 ( 256 MB)
[ 0.000000] .init : 0xc132d000 - 0xc136e000 ( 260 kB)
[ 0.000000] .data : 0xc1252630 - 0xc132cd00 ( 873 kB)
[ 0.000000] .text : 0xc1000000 - 0xc1252630 (2377 kB)
[ 0.000000] Checking if this processor honours the WP bit even in supervisor mode...Ok.
[ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS:2304 nr_irqs:256 16
[ 0.000000] console [ttyS0] enabled
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 498.062 MHz processor
[ 0.003005] Calibrating delay loop (skipped), value calculated using timer frequency.. 996.12 BogoMIPS (lpj=498062)
[ 0.005013] pid_max: default: 32768 minimum: 301
[ 0.007653] Mount-cache hash table entries: 512
[ 0.011056] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.011056] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.011056] tlb_flushall_shift: -1
[ 0.012009] CPU: Geode(TM) Integrated Processor by AMD PCS (fam: 05, model: 0a, stepping: 02)
[ 0.018327] Performance Events: no PMU driver, software events only.
[ 0.026472] NET: Registered protocol family 16
[ 0.030862] PCI: PCI BIOS revision 2.10 entry at 0xfced9, last bus=0
[ 0.031012] PCI: Using configuration type 1 for base access
[ 0.051103] bio: create slab <bio-0> at 0
[ 0.057403] SCSI subsystem initialized
[ 0.060534] PCI: Probing PCI hardware
[ 0.062281] PCI host bridge to bus 0000:00
[ 0.063030] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 0.064033] pci_bus 0000:00: root bus resource [mem 0x00000000-0xffffffff]
[ 0.065020] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[ 0.078410] Switching to clocksource pit
[ 0.086307] NET: Registered protocol family 2
[ 0.088599] TCP established hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.089847] TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.090949] TCP: Hash tables configured (established 2048 bind 2048)
[ 0.092136] TCP: reno registered
[ 0.093867] UDP hash table entries: 256 (order: 0, 4096 bytes)
[ 0.095398] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
[ 0.096873] NET: Registered protocol family 1
[ 0.100630] platform rtc_cmos: registered platform RTC device (no PNP device found)
[ 0.104665] alix: system is recognized as "PC Engines ALIX.3 v0.99h"
[ 0.106619] LED INIT
[ 0.107229] BUG: unable to handle kernel NULL pointer dereference at 0000004c
[ 0.108207] IP: [<c115acc2>] __gpio_set_value+0x12/0x80
[ 0.108207] *pde = 00000000
[ 0.108207] Oops: 0000 [#1]
[ 0.108207] Modules linked in:
[ 0.108207] CPU: 0 PID: 1 Comm: swapper Not tainted 3.10.49 #40
[ 0.108207] task: cf834000 ti: cf840000 task.ti: cf840000
[ 0.108207] EIP: 0060:[<c115acc2>] EFLAGS: 00010286 CPU: 0
[ 0.108207] EIP is at __gpio_set_value+0x12/0x80
[ 0.108207] EAX: c13a1624 EBX: c13a1624 ECX: ffffffea EDX: 00000000
[ 0.108207] ESI: 00000000 EDI: 00000000 EBP: cf841f80 ESP: cf841f24
[ 0.108207] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 0.108207] CR0: 8005003b CR2: 0000004c CR3: 01371000 CR4: 00000090
[ 0.108207] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 0.108207] DR6: ffff0ff0 DR7: 00000400
[ 0.108207] Stack:
[ 0.108207] cf841f3b cf841f44 0000003e c133b031 c12b0deb 006e6967 01010101 01010101
[ 0.108207] 00000000 c133afbb c1000172 00000001 00060006 c1301bf0 c12ceeb1 cfdff460
[ 0.108207] 00000000 cfdff460 00000200 00000000 c136c480 00000006 0000003e cf840000
[ 0.108207] Call Trace:
[ 0.108207] [<c133b031>] ? power_led_init+0x76/0xe5
[ 0.108207] [<c133afbb>] ? alix_init+0xf6/0xf6
[ 0.108207] [<c1000172>] ? do_one_initcall+0xb2/0x150
[ 0.108207] [<c132da22>] ? kernel_init_freeable+0xd1/0x173
[ 0.108207] [<c132d4aa>] ? do_early_param+0x77/0x77
[ 0.108207] [<c124b058>] ? kernel_init+0x8/0x170
[ 0.108207] [<c1251622>] ? ret_from_kernel_thread+0x6/0x28
[ 0.108207] [<c1251637>] ? ret_from_kernel_thread+0x1b/0x28
[ 0.108207] [<c124b050>] ? rest_init+0x60/0x60
[ 0.108207] Code: d6 ab aa aa aa ff d1 5b 5e 5f c3 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 57 89 d7 56 53 e8 46 f4 ff ff 85 c0 89 c3 74 5d 8b 30 <f6> 46 4c 01 74 10 ba 8a 07 00 00 b8 5c da 2c c1 e8 b9 bb ec ff
[ 0.108207] EIP: [<c115acc2>] __gpio_set_value+0x12/0x80 SS:ESP 0068:cf841f24
[ 0.108207] CR2: 000000000000004c
[ 0.108207] ---[ end trace 7b3836317c1bee78 ]---
[ 0.108879] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[ 0.108879]
I think you have put this in kernel source directory. You should place this in drivers/gpio/ in kernel source. Then add the entries in Kconfig and Makefile of drivers/gpio/ directory. In Kconfig, you can specify the dependency. Selecting it with make menuconfig, you will be able to compile this module as a part of kernel, and it will be loaded at boot-time when kernel is loaded. You can decrease the time further in this case using early_initcall() instead of module_init().
If you are not putting this at kernel source (i.e. not building the driver as part of kernel), then you should call this using "insmod my_module.ko" in a shell script, and put it in init.d and call it at your desired runlevel.
I am getting following Oops message while testing on a device which is running on Linux Kernel 3.4.5 and ARM processor.I am unable to trace the issue.
If you look at call stack , you would see it starting from ret_fast_syscall() , now actually sysrq for Crash has been triggered inside the code but i am not getting from where.How can i find that. I have Lauterbach installed but no idea from where to find which part of kernel code has actually triggered this SysRq.
[50728.239318] C0 [ sh] SysRq : Trigger a crash
[50728.239501] C0 [ sh] **************** READ GIC status
[50728.239654] C0 [ sh] Unable to handle kernel NULL pointer dereference at virtual address 00000000
[50728.239776] C0 [ sh] pgd = ebc9c000
[50728.239929] C0 [ sh] [00000000] *pgd=00000000
[50728.240081] C0 [ sh] Internal error: Oops: 805 [#1] PREEMPT SMP ARM
[50728.240234] C0 [ sh] kona_fb: die notifier invoked
[50728.240356] C0 [ sh] Modules linked in: bcmdhd dm_crypt(O) moc_crypto(PO) moc_platform_mod(O) texfat(PO)
[50728.241088] C0 [ sh] CPU: 0 Tainted: P W O (3.4.5-g4192471-dirty #116)
[50728.241271] C0 [ sh] PC is at sysrq_handle_crash+0x14/0x20
[50728.241424] C0 [ sh] LR is at __handle_sysrq+0xa0/0x14c
[50728.241516] C0 [ sh] pc : [<c026ad48>] lr : [<c026b210>] psr: 60000093
[50728.241516] C0 [ sh] sp : ebd0ff20 ip : 0000000c fp : 4003a404
[50728.241821] C0 [ sh] r10: 4070002c r9 : edbc2f0c r8 : 00000000
[50728.241912] C0 [ sh] r7 : 60000013 r6 : 00000063 r5 : 00000007 r4 : c0a26b90
[50728.242065] C0 [ sh] r3 : 00000001 r2 : 00000000 r1 : c07ab3a2 r0 : 00000063
[50728.242248] C0 [ sh] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
[50728.242401] C0 [ sh] Control: 10c53c7d Table: adc9c06a DAC: 00000015
[50728.242492] C0 [ sh]
.................. (Register Contents).............
[50728.284851] C0 [ sh] Stack: (0xebd0ff20 to 0xebd10000)
[50728.284942] C0 [ sh] ff20: 00000002 c026b2bc ec2a4ef0 ebd0ff88 00000002 c026b2e0 edbc2ec0 c014c840
[50728.285125] C0 [ sh] ff40: 00000002 ec2a4ef0 4070002c ebd0ff88 00000002 ebd0e000 00000000 c0109d24
[50728.285308] C0 [ sh] ff60: ec2a4ef0 4070002c ec2a4ef0 4070002c 00000000 00000000 00000002 c0109f5c
[50728.285491] C0 [ sh] ff80: 00000001 00000000 00000000 00000000 00000003 00000002 00000001 00000004
[50728.285675] C0 [ sh] ffa0: c000e304 c000e140 00000003 00000002 00000001 4070002c 00000002 ffffffff
[50728.285858] C0 [ sh] ffc0: 00000003 00000002 00000001 00000004 4070002c 00000000 406ffc84 4003a404
[50728.286010] C0 [ sh] ffe0: 40035f38 bea5e710 40020257 40150e44 20000010 00000001 00000000 00000000
[50728.286224] C0 [ sh] [<c026ad48>] (sysrq_handle_crash+0x14/0x20) from [<c026b210>] (__handle_sysrq+0xa0/0x14c)
***[50728.286376] C0 [ sh] [<c026b210>] (__handle_sysrq+0xa0/0x14c) from [<c026b2e0>] (write_sysrq_trigger+0x24/0x34)
[50728.286560] C0 [ sh] [<c026b2e0>] (write_sysrq_trigger+0x24/0x34) from [<c014c840>] (proc_reg_write+0x80/0x94)
[50728.286682] C0 [ sh] [<c014c840>] (proc_reg_write+0x80/0x94) from [<c0109d24>] (vfs_write+0xb0/0x128)
[50728.286865] C0 [ sh] [<c0109d24>] (vfs_write+0xb0/0x128) from [<c0109f5c>] (sys_write+0x38/0x64)
[50728.287048] C0 [ sh] [<c0109f5c>] (sys_write+0x38/0x64) from [<c000e140>] (ret_fast_syscall+0x0/0x48)***
[50728.287200] C0 [ sh] Code: e3a03001 e5823000 f57ff04f e3a02000 (e5c23000)
[50728.287384] C0 [ sh] ---[ end trace 1b75b31a2719ed7e ]---
[50728.287475] C0 [ sh] Kernel panic - not syncing: Fatal exception
[50728.239318] C0 [ sh] SysRq : Trigger a crash
Someone at userland triggered a crash as you would do via echo c > /proc/sysrq-trigger.