I'm trying to set my CPU cache write policy to 'write back' so I need to set CR0.NW = 1.
I wrote a kernel module:
int
init_module (void)
{
printk (KERN_INFO "init_module\n\n\n");
uint64_t cr0;
asm volatile ("mov %%cr0,%%rax\n\t":"=a"(cr0));
printk(KERN_INFO"CR0 ===== %ld\n",cr0);
asm volatile("push %rax\n\t" "push %rbx\n\t");
asm volatile( //disable cache before changing cr0.nw
"mov $1,%rbx\n\t"
"shl $30,%rbx\n\t"
"mov %cr0,%rax\n\t"
"xor %rbx,%rax\n\t"
"mov %rax,%cr0\n\t"
"wbinvd\n\t" //flush
);
asm volatile( //invert bit
"mov $1,%rbx\n\t"
"shl $29,%rbx\n\t"
"mov %cr0,%rax\n\t"
"xor %rbx,%rax\n\t"
"mov %rax,%cr0\n\t"
);
asm volatile( //enable cache
"mov $1,%rbx\n\t"
"shl $30,%rbx\n\t"
"mov %cr0,%rax\n\t"
"xor %rbx,%rax\n\t" //xor : 1 => 0 , 0 => 1
"mov %rax,%cr0\n\t"
"wbinvd\n\t" //flush
);
asm volatile("pop %rbx\n\t" "pop %rax\n\t");
return 0;
}
but it doesn't work. dmesg give me :
[ 1190.301973] general protection fault: 0000 [#1] SMP
[ 1190.301975] Modules linked in: cache(POE+) ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp bridge stp llc ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables bnep rfcomm bluetooth 6lowpan_iphc binfmt_misc nls_iso8859_1 nvidia(POE) cp210x usbserial joydev snd_hda_codec_hdmi eeepc_wmi asus_wmi sparse_keymap snd_hda_codec_realtek snd_hda_codec_generic intel_rapl snd_hda_intel x86_pkg_temp_thermal intel_powerclamp snd_hda_controller coretemp snd_hda_codec kvm_intel snd_hwdep snd_pcm kvm snd_seq_midi snd_seq_midi_event crct10dif_pclmul crc32_pclmul snd_rawmidi ghash_clmulni_intel aesni_intel snd_seq aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd nouveau serio_raw mxm_wmi ttm snd_seq_device snd_timer lpc_ich drm_kms_helper drm snd mei_me mei soundcore i2c_algo_bit shpchp wmi video mac_hid soc_button_array tpm_infineon parport_pc ppdev lp parport uas usb_storage hid_generic usbhid hid ahci r8169 psmouse libahci mii
[ 1190.302013] CPU: 5 PID: 5159 Comm: insmod Tainted: P OE 3.16.0-45-generic #60~14.04.1-Ubuntu
[ 1190.302014] Hardware name: ASUS All Series/Z87-A, BIOS 1007 05/17/2013
[ 1190.302015] task: ffff8807d95765e0 ti: ffff8807d95b0000 task.ti: ffff8807d95b0000
[ 1190.302016] RIP: 0010:[<ffffffffc0fd402a>] [<ffffffffc0fd402a>] init_module+0x2a/0x40 [cache]
[ 1190.302019] RSP: 0018:ffff8807d95b3d30 EFLAGS: 00010206
[ 1190.302019] RAX: 00000000a0050033 RBX: 0000000020000000 RCX: 0000000000000000
[ 1190.302020] RDX: ffff88081ed4ee40 RSI: ffff88081ed4d418 RDI: 0000000000000246
[ 1190.302021] RBP: ffff8807d95b3d40 R08: 0000000000000082 R09: 00000000000012e5
[ 1190.302022] R10: 0000000000000000 R11: ffff8807d95b3a5e R12: ffff8807ef817de0
[ 1190.302022] R13: 0000000000000000 R14: ffffffffc0fd4000 R15: ffffffffc0fd6000
[ 1190.302023] FS: 00007f405d04f740(0000) GS:ffff88081ed40000(0000) knlGS:0000000000000000
[ 1190.302024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1190.302025] CR2: 00007f405d26a248 CR3: 00000007e5d81000 CR4: 00000000001407e0
[ 1190.302026] Stack:
[ 1190.302026] ffffffff81c1a020 000000000000000d ffff8807d95b3db8 ffffffff81002144
[ 1190.302028] 0000000000000001 0000000000000001 0000000000000001 ffff8800dca9b440
[ 1190.302029] 0000000000000001 ffff8807d95b3da0 ffffffff8119d7d2 ffffffffc0fd6018
[ 1190.302030] Call Trace:
[ 1190.302035] [<ffffffff81002144>] do_one_initcall+0xd4/0x210
[ 1190.302037] [<ffffffff8119d7d2>] ? __vunmap+0xb2/0x100
[ 1190.302039] [<ffffffff810edd79>] load_module+0x13d9/0x1b90
[ 1190.302043] [<ffffffff810e9910>] ? store_uevent+0x40/0x40
[ 1190.302044] [<ffffffff810ee6a6>] SyS_finit_module+0x86/0xb0
[ 1190.302048] [<ffffffff8176e34d>] system_call_fastpath+0x1a/0x1f
[ 1190.302048] Code: <0f> 22 c0 5b 58 31 c0 5d c3 66 66 66 66 2e 0f 1f 84 00 00 00 00 00
[ 1190.302055] RIP [<ffffffffc0fd402a>] init_module+0x2a/0x40 [cache]
[ 1190.302056] RSP <ffff8807d95b3d30>
[ 1190.302057] ---[ end trace bf14887f4e905bad ]---
Do you know what is happening? doesn't it mean i can't change CR0.NW ?
My CPU : i7-4770K
According to the information provided by Intel's manual (See Table 11-5),
it is not allowed to set CR0.NW to 1 when CR0.CD is 0.
I assume you tried changing the write-policy (CR.NW=1) and using caches (CR.CD = 0) at the same time, which is invalid.
I found it when I was looking for the same thing as you did...
Related
I'm working on an embedded Linux training with Raspberry PI 4B and in order to create I2C bus driver I need to access hardware registers. The problem is when I try to read any of them I get "Unhandled fault: unknown 3 (0x203) at 0xf08c5000".
Here is the code I'm using to read register:
void i2cHandler_init(void)
{
const u32 rpiPeriphBase = 0xfe000000;
const u32 rpiGpioOffset = 0x200000;
const void *rpiGpioAddress = rpiPeriphBase + rpiGpioOffset;
__iomem u32 *ioMemory;
ioMemory = ioremap(rpiGpioAddress, 0x40 * 8);
pr_info("%s: rpiGpioAddress = 0x%x\r\n", __func__, rpiGpioAddress);
pr_info("%s: ioMemory = 0x%x\r\n", __func__, ioMemory);
pr_info("%s: Register value = 0x%x\r\n", __func__, readl(ioMemory));
if(ioMemory != NULL)
iounmap(ioMemory);
}
Yes, I'm trying to read GPIO GPFSEL0 register here, but changing offset to read I2C Control register changes nothing.
Kernel log output looks like this:
[ 1467.677279] i2cHandler_init: rpiGpioAddress = 0xfe200000
[ 1467.677289] i2cHandler_init: ioMemory = 0xf08c5000
[ 1467.677308] 8<--- cut here ---
[ 1467.677325] Unhandled fault: unknown 3 (0x203) at 0xf08c5000
[ 1467.677340] pgd = f812a8c9
[ 1467.677353] [f08c5000] *pgd=80000000007003, *pmd=2ff5b003, *pte=c00ffffe200713
[ 1467.677385] Internal error: : 203 [#1] SMP ARM
[ 1467.677400] Modules linked in: rpi_driver_I2C(O+) rfcomm cmac algif_hash aes_arm_bs crypto_simd cryptd algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc spidev snd_soc_hdmi_codec 8021q garp stp llc brcmfmac brcmutil v3d cfg80211 gpu_sched rfkill raspberrypi_hwmon i2c_brcmstb vc4 spi_bcm2835 cec bcm2835_codec(C) bcm2835_v4l2(C) bcm2835_isp(C) v4l2_mem2mem bcm2835_mmal_vchiq(C) videobuf2_dma_contig videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common vc_sm_cma(C) drm_kms_helper videodev snd_bcm2835(C) snd_soc_core mc snd_compress snd_pcm_dmaengine snd_pcm rpivid_mem snd_timer snd syscopyarea sysfillrect sysimgblt fb_sys_fops uio_pdrv_genirq uio nvmem_rmem i2c_dev drm drm_panel_orientation_quirks backlight fuse ip_tables x_tables ipv6 [last unloaded: rpi_driver_I2C]
[ 1467.677744] CPU: 0 PID: 2059 Comm: insmod Tainted: G C O 5.15.32-v7l+ #1538
[ 1467.677764] Hardware name: BCM2711
[ 1467.677775] PC is at i2cHandler_init+0xc4/0x178 [rpi_driver_I2C]
[ 1467.677808] LR is at irq_work_queue+0x14/0x2c
[ 1467.677833] pc : [<bf3fe464>] lr : [<c0358054>] psr: 60000013
[ 1467.677846] sp : c3e57d68 ip : 00000000 fp : c3e57d7c
[ 1467.677859] r10: c1205048 r9 : bf400380 r8 : 00000000
[ 1467.677871] r7 : 00000002 r6 : bf3fe5a8 r5 : c1205048 r4 : f08c5000
[ 1467.677884] r3 : 3391a86e r2 : 3391a86e r1 : 00000027 r0 : 00000029
[ 1467.677898] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 1467.677914] Control: 30c5383d Table: 041e1e40 DAC: fffffffd
[ 1467.677926] Register r0 information: non-paged memory
[ 1467.677943] Register r1 information: non-paged memory
[ 1467.677958] Register r2 information: non-paged memory
[ 1467.677973] Register r3 information: non-paged memory
[ 1467.677988] Register r4 information: 0-page vmalloc region starting at 0xf08c5000 allocated at i2cHandler_init+0x80/0x178 [rpi_driver_I2C]
[ 1467.678024] Register r5 information: non-slab/vmalloc memory
[ 1467.678041] Register r6 information: 4-page vmalloc region starting at 0xbf3fe000 allocated at load_module+0xb94/0x2840
[ 1467.678069] Register r7 information: non-paged memory
[ 1467.678084] Register r8 information: NULL pointer
[ 1467.678099] Register r9 information: 4-page vmalloc region starting at 0xbf3fe000 allocated at load_module+0xb94/0x2840
[ 1467.678124] Register r10 information: non-slab/vmalloc memory
[ 1467.678140] Register r11 information: non-slab/vmalloc memory
[ 1467.678155] Register r12 information: NULL pointer
[ 1467.678170] Process insmod (pid: 2059, stack limit = 0x91bbcd6e)
[ 1467.678185] Stack: (0xc3e57d68 to 0xc3e58000)
[ 1467.678199] 7d60: bf400380 c1205048 c3e57d8c c3e57d80 bf3fe5fc bf3fe3ac
[ 1467.678217] 7d80: c3e57d9c c3e57d90 bf3fe5cc bf3fe5e4 c3e57e14 c3e57da0 c02021c4 bf3fe5b4
[ 1467.678234] 7da0: c0439ca0 c0bd5bf4 c1401180 00000000 c3e57dd4 c3e57dc0 c0bd5bf4 c029f244
[ 1467.678251] 7dc0: c1401180 c043ade4 c3e57e14 c3e57dd8 c043ade4 c03f4518 c041c20c c043997c
[ 1467.678269] 7de0: 00000008 c02d056c f097d000 3391a86e 00000002 bf400380 00000002 c3c36d80
[ 1467.678286] 7e00: 00000002 c4ad01c8 c3e57e3c c3e57e18 c02d058c c0202180 c3e57e3c c3e57e28
[ 1467.678303] 7e20: c041c394 c3e57f30 00000002 c4ad0180 c3e57f14 c3e57e40 c02d2dac c02d0544
[ 1467.678320] 7e40: bf40038c 00007fff bf400380 c02cf240 c1205048 c0e3b448 c0e3b460 c0e3b3f0
[ 1467.678337] 7e60: bf0c609a c0e3b4b0 c0c03df0 c3e57f30 bf400594 c4dc6418 bf4003c8 c4ad0188
[ 1467.678354] 7e80: c0e3afcc 00000001 00000000 c0efea84 c0ee67d0 00000000 00000000 00000000
[ 1467.678371] 7ea0: 00000000 00000000 6e72656b 00006c65 00000000 00000000 00000000 00000000
[ 1467.678388] 7ec0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1467.678405] 7ee0: 00000000 3391a86e c3e57f2c c1205048 00000000 0002de04 00000003 c0200244
[ 1467.678423] 7f00: c3e56000 0000017b c3e57fa4 c3e57f18 c02d33b0 c02d0900 c3e57f2c 7fffffff
[ 1467.678439] 7f20: 00000000 00000002 c3e57f24 f097d000 f097db9e f097e5c0 f097d000 000033ac
[ 1467.678456] 7f40: f097ff9c f097fea8 f097f3a0 00003000 000036e0 00002018 00003aab 00000000
[ 1467.678474] 7f60: 00000000 00000000 00002008 00000017 00000018 00000010 0000000d 0000000a
[ 1467.678491] 7f80: 00000000 3391a86e 00000000 00000002 f9957800 0000017b 00000000 c3e57fa8
[ 1467.678508] 7fa0: c0200040 c02d32f4 00000000 00000002 00000003 0002de04 00000000 b6f68074
[ 1467.678535] 7fc0: 00000000 00000002 f9957800 0000017b 00e9acc8 00000002 bee367d4 00000000
[ 1467.678554] 7fe0: bee36600 bee365f0 00023bc0 b6c459e0 60000010 00000003 00000000 00000000
[ 1467.678568] Backtrace:
[ 1467.678582] [<bf3fe3a0>] (i2cHandler_init [rpi_driver_I2C]) from [<bf3fe5fc>] (mainCycle_init+0x24/0x38 [rpi_driver_I2C])
[ 1467.678632] r5:c1205048 r4:bf400380
[ 1467.678643] [<bf3fe5d8>] (mainCycle_init [rpi_driver_I2C]) from [<bf3fe5cc>] (init_module+0x24/0x30 [rpi_driver_I2C])
[ 1467.678687] [<bf3fe5a8>] (init_module [rpi_driver_I2C]) from [<c02021c4>] (do_one_initcall+0x50/0x244)
[ 1467.678726] [<c0202174>] (do_one_initcall) from [<c02d058c>] (do_init_module+0x54/0x23c)
[ 1467.678760] r8:c4ad01c8 r7:00000002 r6:c3c36d80 r5:00000002 r4:bf400380
[ 1467.678776] [<c02d0538>] (do_init_module) from [<c02d2dac>] (load_module+0x24b8/0x2840)
[ 1467.678807] r6:c4ad0180 r5:00000002 r4:c3e57f30
[ 1467.678824] [<c02d08f4>] (load_module) from [<c02d33b0>] (sys_finit_module+0xc8/0xfc)
[ 1467.678857] r10:0000017b r9:c3e56000 r8:c0200244 r7:00000003 r6:0002de04 r5:00000000
[ 1467.678876] r4:c1205048
[ 1467.678892] [<c02d32e8>] (sys_finit_module) from [<c0200040>] (ret_fast_syscall+0x0/0x1c)
[ 1467.678925] Exception stack(0xc3e57fa8 to 0xc3e57ff0)
[ 1467.678943] 7fa0: 00000000 00000002 00000003 0002de04 00000000 b6f68074
[ 1467.678969] 7fc0: 00000000 00000002 f9957800 0000017b 00e9acc8 00000002 bee367d4 00000000
[ 1467.678986] 7fe0: bee36600 bee365f0 00023bc0 b6c459e0
[ 1467.679007] r7:0000017b r6:f9957800 r5:00000002 r4:00000000
[ 1467.679030] Code: e1a02004 e30f0360 e34b0f3f eb5f37d6 (e5942000)
[ 1467.679052] ---[ end trace b887a2abb18bba14 ]---
Also I'm sure peripherals base address is correct because output of "sudo cat /proc/iomem" looks like this:
00000000-3b3fffff : System RAM
00008000-00ffffff : Kernel code
01200000-013ef203 : Kernel data
40000000-7fffffff : System RAM
fd500000-fd50930f : fd500000.pcie pcie#7d500000
fd580000-fd58ffff : fd580000.ethernet ethernet#7d580000
fd580e14-fd580e1c : unimac-mdio.-19
fe004000-fe00401f : fe004000.txp txp#7e004000
fe007000-fe007aff : fe007000.dma dma#7e007000
fe007b00-fe007eff : fe007b00.dma dma#7e007b00
fe00a000-fe00a023 : fe100000.watchdog watchdog#7e100000
fe00b840-fe00b87b : fe00b840.mailbox mailbox#7e00b840
fe00b880-fe00b8bf : fe00b880.mailbox mailbox#7e00b880
fe100000-fe100113 : fe100000.watchdog watchdog#7e100000
fe101000-fe102fff : fe101000.cprman cprman#7e101000
fe104000-fe104027 : fe104000.rng rng#7e104000
**fe200000-fe2000b3 : fe200000.gpio gpio#7e200000**
fe201000-fe2011ff : serial#7e201000
fe201000-fe2011ff : fe201000.serial serial#7e201000
fe204000-fe2041ff : fe204000.spi spi#7e204000
fe206000-fe2060ff : fe206000.pixelvalve pixelvalve#7e206000
fe207000-fe2070ff : fe207000.pixelvalve pixelvalve#7e207000
fe20a000-fe20a0ff : fe20a000.pixelvalve pixelvalve#7e20a000
fe215000-fe215007 : fe215000.aux aux#7e215000
fe216000-fe2160ff : fe216000.pixelvalve pixelvalve#7e216000
fe300000-fe3000ff : fe300000.mmcnr mmcnr#7e300000
fe340000-fe3400ff : fe340000.mmc mmc#7e340000
fe400000-fe407fff : fe400000.hvs hvs#7e400000
fec00000-fec03fff : fec00000.v3d hub
fec04000-fec07fff : fec00000.v3d core0
fec11000-fec1101f : fe100000.watchdog watchdog#7e100000
fec12000-fec120ff : fec12000.pixelvalve pixelvalve#7ec12000
fef00000-fef0000f : fef00000.clock clock#7ef00000
fef00b00-fef00dff : fef04500.i2c auto-i2c
fef04500-fef045ff : fef04500.i2c bsc
fef05b00-fef05dff : fef09500.i2c auto-i2c
fef09500-fef095ff : fef09500.i2c bsc
600000000-63fffffff : pcie#7d500000
600000000-6000fffff : PCI Bus 0000:01
600000000-600000fff : 0000:01:00.0
600000000-600000fff : xhci-hcd
Raspbeerry PI works under the Raspbian OS 32-bit version. Kernel version is 5.15.32-v7l+.
I tried different base addresses found from different sources:
0x20000000 - ioremap returns 0.
0x3e000000 - ioremap works, no errors, but values there make no sense and writing with this base address leads to no reaction at all.
0x7e000000 - ioremap returns 0.
0x7e000000 without ioremap - error "Unable to handle kernel paging request at virtual address ...".
According to different forums topics and examples my piece of code should just work, but it is not, so I'm out of ideas right now. Does anyone have any thoughts? Is this my error or something else is wrong?
Problem is solved. Nothing seems wrong with my code, 64-bit OS helped.
I didn't actually install 64-bit system, I added "arm_64bit=1" to my /boot/config.txt and updated all packages, because this was just for testing purposes, but this worked.
I am trying to write a kernel module that will replace system calls for linux 4.9. All the solutions on the Internet are specific to x86 but I am working on Beaglebone Black that has an arm cortex A8.
This is what I have done so far.
static unsigned long *sys_call_table; // this is a global
The module when insmoded appears as a device in /dev which the user can open and give an ioctl command. In ioctl I use
sys_call_table=(void*)kallsyms_lookup_name("sys_call_table");
which obtains the same address as given in System.map file. But the moment I try to change system call using
*(sys_call_table + __NR_open) = (unsigned long)custom_open;
It gives errors. they are
[ 155.354417] Unable to handle kernel paging request at virtual address c01079f8
[ 155.361959] pgd = de6c0000
[ 155.364780] [c01079f8] *pgd=8000041e(bad)
[ 155.368981] Internal error: Oops: 80d [#1] SMP ARM
[ 155.373980] Modules linked in: intercept(O) [last unloaded: intercept]
[ 155.380821] CPU: 0 PID: 120 Comm: test Tainted: G O 4.9.39 #1
[ 155.387991] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 155.394342] task: de6b2380 task.stack: de664000
[ 155.399089] PC is at my_ioctl+0x64/0x94 [intercept]
[ 155.404180] LR is at my_ioctl+0x58/0x94 [intercept]
[ 155.409269] pc : [<bf0040dc>] lr : [<bf0040d0>] psr: 60000013
[ 155.409269] sp : de665f08 ip : 00000001 fp : bedb0c54
[ 155.421258] r10: 00000000 r9 : 00000003 r8 : 00000003
[ 155.426711] r7 : c02b354c r6 : de6293c0 r5 : de6dc2f0 r4 : bf004580
[ 155.433522] r3 : c01079e4 r2 : bf004000 r1 : ffffe000 r0 : bf004294
[ 155.440334] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 155.447773] Control: 10c5387d Table: 9e6c0019 DAC: 00000051
[ 155.453772] Process test (pid: 120, stack limit = 0xde664218)
[ 155.459771] Stack: (0xde665f08 to 0xde666000)
[ 155.464321] 5f00: bedb0dac c02b2aec 00000000 de6b2670 de665f7c c07ddc98
[ 155.472862] 5f20: 60000013 c0c0512c c0cbfe80 c0192d10 c0c8311c de611000 c029ddd8 c0cbf624
[ 155.481401] 5f40: 2ae98e92 00000024 2b36fb89 00000024 c07de574 df947010 de6293c8 de664000
[ 155.489938] 5f60: 00000000 00000000 de6293c0 de6293c0 00000005 bedb0dac 00000003 00000000
[ 155.498469] 5f80: bedb0c54 c02b354c 00000000 00000000 0001036c 00000036 c01079e4 de664000
[ 155.507005] 5fa0: 00000000 c0107840 00000000 00000000 00000003 00000005 bedb0dac 00010494
[ 155.515541] 5fc0: 00000000 00000000 0001036c 00000036 00000000 00000000 b6f12000 bedb0c54
[ 155.524075] 5fe0: b6e74d90 bedb0c44 000104bc b6e74d9c 60000010 00000003 00000000 00000000
[ 155.532633] [<bf0040dc>] (my_ioctl [intercept]) from [<c02b2aec>] (do_vfs_ioctl+0x90/0xa84)
[ 155.541357] [<c02b2aec>] (do_vfs_ioctl) from [<c02b354c>] (SyS_ioctl+0x6c/0x7c)
[ 155.548992] [<c02b354c>] (SyS_ioctl) from [<c0107840>] (ret_fast_syscall+0x0/0x1c)
[ 155.556898] Code: eb48ce6e e5943004 e59f2028 e59f0028 (e5832014)
[ 155.563276] ---[ end trace 0529de7e48dd6bb4 ]---
[ 155.571707] MyDevice closed
Segmentation fault
Please give me a solution specific to arm.
I am trying to port kernel 4.12 to my board, but I'm facing a very weird issue.
Some context: I am using u-boot as bootloader, booting from a zImage with device tree support. The board has a Sitara AM3352.
What happens is that sometimes the kernel will start printing stuff, but most often it does nothing.
Situation 1:
U-Boot# bootz $kloadaddr - $fdtaddr
## Current stack ends at 0x83ee8bf8 Kernel image # 0x80007fc0 [ 0x00000000 - 0x002d9430 ]
## Flattened Device Tree blob at 88000000
Booting using the fdt blob at 0x88000000
using: FDT
using: FDT
Loading Device Tree to 83edd000, end 83ee7337 ... OK
## Transferring control to Linux (at address 80007fc0)...
Starting kernel ...
Situation 2:
U-Boot# bootz $kloadaddr - $fdtaddr
## Current stack ends at 0x83ee8bf8 Kernel image # 0x80007fc0 [ 0x00000000 - 0x002d9430 ]
## Flattened Device Tree blob at 88000000
Booting using the fdt blob at 0x88000000
using: FDT
using: FDT
Loading Device Tree to 83edd000, end 83ee7337 ... OK
## Transferring control to Linux (at address 80007fc0)...
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.12.0R01-Beta (guilherme#epoch) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0) ) #4 Thu Aug 3 14:30:16 BRT 2017
[ 0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Test Board
[ 0.000000] earlycon: omapserial0 at MMIO 0x44e09000 (options '')
[ 0.000000] bootconsole [omapserial0] enabled
[ 0.000000] Memory policy: Data cache writeback
[ 0.000000] CPU: All CPU(s) started in SVC mode.
[ 0.000000] AM335X ES2.1 (neon)
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16256
[ 0.000000] Kernel command line: console=ttyO0,115200n8 root=/dev/mtdblock10 rw rootfstype=jffs2 noinitrd earlycon
[ 0.000000] PID hash table entries: 256 (order: -2, 1024 bytes)
[ 0.000000] Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.000000] Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.000000] Memory: 56952K/65536K available (4096K kernel code, 179K rwdata, 1484K rodata, 1024K init, 438K bss, 8584K reserved, 0K cma-reserved, 0K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xc4800000 - 0xff800000 ( 944 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xc4000000 ( 64 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0008000 - 0xc0500000 (5088 kB)
[ 0.000000] .init : 0xc0700000 - 0xc0800000 (1024 kB)
[ 0.000000] .data : 0xc0800000 - 0xc082cd90 ( 180 kB)
[ 0.000000] .bss : 0xc0831794 - 0xc089f2c8 ( 439 kB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] IRQ: Found an INTC at 0xfa200000 (revision 5.0) with 128 interrupts
[ 0.000000] OMAP clockevent source: timer2 at 24000000 Hz
[ 0.000020] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
[ 0.008003] clocksource: timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
[ 0.017467] OMAP clocksource: timer1 at 24000000 Hz
[ 0.022622] clocksource_probe: no matching clocksources found
[ 0.028583] Calibrating delay loop... 599.04 BogoMIPS (lpj=1198080)
[ 0.059960] pid_max: default: 32768 minimum: 301
[ 0.064987] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.071729] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.079579] CPU: Testing write buffer coherency: ok
[ 0.085216] Setting up static identity map for 0x80100000 - 0x80100060
[ 0.095211] devtmpfs: initialized
[ 0.107961] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
[ 0.116265] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.126579] futex hash table entries: 256 (order: -1, 3072 bytes)
[ 0.132888] pinctrl core: initialized pinctrl subsystem
[ 0.139759] NET: Registered protocol family 16
[ 0.145433] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.165036] omap_hwmod: debugss: _wait_target_disable failed
[ 0.171060] Unhandled fault: external abort on non-linefetch (0x1028) at 0xf9e3e078
[ 0.178849] pgd = c0004000
[ 0.181618] [f9e3e078] *pgd=44e11452(bad)
[ 0.185740] Internal error: : 1028 [#1] ARM
[ 0.190019] Modules linked in:
[ 0.193157] CPU: 0 PID: 1 Comm: swapper Not tainted 4.12.0R01-Beta #4
[ 0.199760] Hardware name: Generic AM33XX (Flattened Device Tree)
[ 0.206019] task: c3830000 task.stack: c382e000
[ 0.210684] PC is at _update_sysc_cache+0x2c/0x94
[ 0.215494] LR is at _enable+0x240/0x2a8
[ 0.219517] pc : [<c0120b24>] lr : [<c0121314>] psr: 40000013
[ 0.219517] sp : c382fe98 ip : c0124774 fp : 00000000
[ 0.231311] r10: c0721838 r9 : 00000045 r8 : c08317c0
[ 0.236673] r7 : c080abf0 r6 : 00000000 r5 : c0831c68 r4 : c080a5a4
[ 0.243377] r3 : f9e3e078 r2 : c080b084 r1 : f9e3e000 r0 : c080a5a4
[ 0.250083] Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 0.257409] Control: 10c5387d Table: 80004019 DAC: 00000051
[ 0.263307] Process swapper (pid: 1, stack limit = 0xc382e210)
[ 0.269296] Stack: (0xc382fe98 to 0xc3830000)
[ 0.273768] fe80: 00000000 c0121314
[ 0.282173] fea0: 00000000 a0000013 c08092a8 c0803048 c080a5a4 c080a5d4 c080a5e8 c0709328
[ 0.290574] fec0: c382fed4 d3053b52 c080ae5c d3053b52 c080a5a4 c0807808 c0709240 00000000
[ 0.298975] fee0: c08317c0 c0121718 0000100c c0803048 00000002 c07099c8 00000000 c0709a6c
[ 0.307377] ff00: 00000000 c0101794 00000154 c3ffed09 c0671b34 c0147200 000017c0 c0228d30
[ 0.315777] ff20: c0803048 c05edd2c c07005dc 00000000 00000002 00000002 00000000 c0803048
[ 0.324179] ff40: 00000000 c3ffed09 00000000 d3053b52 00000000 d3053b52 00000002 c072c0f8
[ 0.332580] ff60: 00000002 c0721820 c08317c0 c08317c0 00000045 c0700dc8 00000002 00000002
[ 0.340981] ff80: 00000000 c07005dc 00000000 c04a7cc0 00000000 00000000 00000000 00000000
[ 0.349382] ffa0: 00000000 c04a7cc8 00000000 c010dd98 00000000 00000000 00000000 00000000
[ 0.357783] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 0.366184] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff fffffffb
[ 0.374609] [<c0120b24>] (_update_sysc_cache) from [<c0121314>] (_enable+0x240/0x2a8)
[ 0.382637] [<c0121314>] (_enable) from [<c0709328>] (_setup+0xe8/0x4e8)
[ 0.389516] [<c0709328>] (_setup) from [<c0121718>] (omap_hwmod_for_each+0x38/0x60)
[ 0.397378] [<c0121718>] (omap_hwmod_for_each) from [<c0709a6c>] (__omap_hwmod_setup_all+0xa4/0xac)
[ 0.406676] [<c0709a6c>] (__omap_hwmod_setup_all) from [<c0101794>] (do_one_initcall+0x50/0x18c)
[ 0.415708] [<c0101794>] (do_one_initcall) from [<c0700dc8>] (kernel_init_freeable+0x140/0x1dc)
[ 0.424650] [<c0700dc8>] (kernel_init_freeable) from [<c04a7cc8>] (kernel_init+0x8/0x10c)
[ 0.433052] [<c04a7cc8>] (kernel_init) from [<c010dd98>] (ret_from_fork+0x14/0x3c)
[ 0.440815] Code: e3110c01 e5901054 e6f13073 1a00000c (e5933000)
[ 0.447075] ---[ end trace 396cebd12b8145f9 ]---
[ 0.451867] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 0.451867]
[ 0.461190] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 0.461190]
[ 1.373583] random: fast init done
This is my .dts file:
/dts-v1/;
#include "am33xx.dtsi"
#include <dt-bindings/interrupt-controller/irq.h>
/ {
model = "Test Board";
compatible = "guilherme,test_board", "ti,am33xx";
memory#80000000 {
device_type = "memory";
reg = <0x80000000 0x04000000>; /* 64 MB */
};
chosen {
stdout-path = &uart0;
bootargs ="console=ttyO0,115200n8 root=/dev/mtdblock10 rw rootfstype=jffs2 noinitrd earlycon";
};
};
While I was expecting to run into a kernel panic, what bugs me is this printing inconsistency. Also I can't see a way to pinpoint if this issue is due to some weird u-boot config, or something related to the kernel itself.
So, found out the problem! As I was working with a previously configured u-Boot, I tried to keep the u-Boot modifications to a minimum. Among these previously existing configurations, was a setenv bootargs = .... This bootargs definition overrides the definition in the .dtb file, causing conflicts. Once I removed this variable definition, the printing issue was resolved.
The reason the system was printing seemingly at random is because the bootargs definition was dependant on the state of a pin the is used to boot from images in a SDCard.
I had a kernel crash on a Redhat 6.
I want to retreive which file was openend
reading the dump I see
crash> bt
(...)
#8 [ffff8805050d7dc0] page_fault at ffffffff814ef845
[exception RIP: configfs_readdir+244]
RIP: ffffffffa0422444 RSP: ffff8805050d7e78 RFLAGS: 00010282
RAX: 0000000000000000 RBX: ffff880814a738d0 RCX: ffff880814a738c8
RDX: 0000000000000006 RSI: ffff880814a73830 RDI: ffffffffa04486d0
RBP: ffff8805050d7ed8 R8: ffff880814a738d0 R9: 0000000000000004
R10: 00000000000000a8 R11: 0000000000000246 R12: ffff8804cf782b00
R13: ffffffffa04486d0 R14: ffff8802087039d0 R15: ffff8802087039d8
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#9 [ffff8805050d7ee0] vfs_readdir at ffffffff8118a670
(...)
crash > bt -f
(...)
#9 [ffff8805050d7ee0] vfs_readdir at ffffffff8118a670
ffff8805050d7ee8: ffff8805050d7f28 ffffffff81176652
ffff8805050d7ef8: 0000000000001000 00000000009ca848
ffff8805050d7f08: ffff8804cf782b00 0000000000008000
ffff8805050d7f18: 0000000000000000 00000000009ca820
ffff8805050d7f28: ffff8805050d7f78 ffffffff8118a7f9
I look at vfs_readdir
crash> whatis vfs_readdir
int vfs_readdir(struct file *, filldir_t, void *);
Then searching for the file (address ffff8805050d7f28 picked from top of stack at #9 )
crash> struct file ffff8805050d7f28
struct file {
f_u = {
fu_list = {
next = 0xffff8805050d7f78,
prev = 0xffffffff8118a7f9
},
fu_rcuhead = {
next = 0xffff8805050d7f78,
func = 0xffffffff8118a7f9 <sys_getdents+137>
}
},
f_path = {
mnt = 0x9ca878,
dentry = 0x9ca860
},
f_op = 0xffffffea00007fd0,
f_lock = {
raw_lock = {
slock = 578155200
}
},
At this point, I can go no further, how can I get string from f_path struct ?
I want to convert
f_path = {
mnt = 0x9ca878,
dentry = 0x9ca860
},
to strings
reading the struct, file owner is -1, effective uid didn't map to a known user.
from crash, files command show only 2 regular file
EDIT: as requested
crash> p ffff8805050d7f28
No symbol "ffff8805050d7f28" in current context.
p: gdb request failed: p ffff8805050d7f28
crash> p *(struct file) ffff8805050d7f28
No symbol "ffff8805050d7f28" in current context.
p: gdb request failed: p *(struct file) ffff8805050d7f28
I have a kernel module for splitting incoming rtp packets and merging rtp outgoing packets. The program crashes once in 2/3 days. If would be very convenient for me if its possible to find the exact line where the module crashes.
I have given the crash dump below. Is it possible to find the exact line in the code from crash dump?
PID: 1256 TASK: ffff88020fc71700 CPU: 0 COMMAND: "rtpproxy"
#0 [ffff880212faf2f0] machine_kexec at ffffffff8103bb7a
#1 [ffff880212faf360] crash_kexec at ffffffff810bb968
#2 [ffff880212faf430] oops_end at ffffffff8169fad8
#3 [ffff880212faf460] die at ffffffff81017808
#4 [ffff880212faf490] do_general_protection at ffffffff8169f5d2
#5 [ffff880212faf4c0] general_protection at ffffffff8169eef5
[exception RIP: pkt_queue+388]
RIP: ffffffffa00f3fa0 RSP: ffff880212faf578 RFLAGS: 00010292
RAX: ffff8802110ae400 RBX: ffff880213a53f38 RCX: 00015d910000a20f
RDX: 497d74565cede60c RSI: 000000006df1ed57 RDI: 00000000e46e0cfc
RBP: ffff880212faf728 R8: ffff880211a8b000 R9: ffff880212fafa60
R10: ffff880212fafbc8 R11: 0000000000000293 R12: 00000000134ab2b4
R13: 000000008386615c R14: 00000000000000e3 R15: 00000000000000e3
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#6 [ffff880212faf730] obsf_tg at ffffffffa00f34a0 [xt_OBSF]
#7 [ffff880212faf890] ipt_do_table at ffffffffa00e41a5 [ip_tables]
#8 [ffff880212faf970] ipt_mangle_out at ffffffffa00dd129 [iptable_mangle]
#9 [ffff880212faf9c0] iptable_mangle_hook at ffffffffa00dd1eb [iptable_mangle]
#10 [ffff880212faf9d0] nf_iterate at ffffffff815aded5
#11 [ffff880212fafa20] nf_hook_slow at ffffffff815adf85
#12 [ffff880212fafaa0] __ip_local_out at ffffffff815babb2
#13 [ffff880212fafac0] ip_local_out at ffffffff815babd6
#14 [ffff880212fafae0] ip_send_skb at ffffffff815bbefb
#15 [ffff880212fafb00] udp_send_skb at ffffffff815df1d1
#16 [ffff880212fafb50] udp_sendmsg at ffffffff815e0286
#17 [ffff880212fafc90] inet_sendmsg at ffffffff815eabc4
#18 [ffff880212fafcd0] sock_sendmsg at ffffffff8156a437
#19 [ffff880212fafe50] sys_sendto at ffffffff8156d91d
#20 [ffff880212faff80] system_call_fastpath at ffffffff816a7029
RIP: 00007f17363b83a3 RSP: 00007ffff2965f90 RFLAGS: 00010213
RAX: 000000000000002c RBX: ffffffff816a7029 RCX: 00007ffff29ff99b
RDX: 0000000000000020 RSI: 00007f1737da4378 RDI: 0000000000000006
RBP: 0000000000000001 R8: 00007f1737da67a0 R9: 0000000000000010
R10: 0000000000000000 R11: 0000000000000293 R12: 00007f1737da4378
R13: 0000000000000001 R14: 00007f1737da42a0 R15: 0000000000000000
ORIG_RAX: 000000000000002c CS: 0033 SS: 002b
[157707.736203] general protection fault: 0000 [#1] SMP
[157707.736955] CPU 0
[157707.736973] Modules linked in:
[157707.737654] arc4 xt_tcpudp xt_OBSF(O) iptable_mangle ip_tables x_tables ghash_clmulni_intel aesni_intel cryptd aes_x86_64 joydev hid_generic microcode ext2 usbhid psmouse hid serio_raw i2c_piix4 virtio_balloon lp parport mac_hid floppy
[157707.740018]
[157707.740102] Pid: 1256, comm: rtpproxy Tainted: G O 3.5.0-23-generic #35~precise1-Ubuntu Bochs Bochs
[157707.740102] RIP: 0010:[<ffffffffa00f3fa0>] [<ffffffffa00f3fa0>] pkt_queue+0x184/0x48a [xt_OBSF]
[157707.740102] RSP: 0018:ffff880212faf578 EFLAGS: 00010292
[157707.740102] RAX: ffff8802110ae400 RBX: ffff880213a53f38 RCX: 00015d910000a20f
[157707.740102] RDX: 497d74565cede60c RSI: 000000006df1ed57 RDI: 00000000e46e0cfc
[157707.740102] RBP: ffff880212faf728 R08: ffff880211a8b000 R09: ffff880212fafa60
[157707.740102] R10: ffff880212fafbc8 R11: 0000000000000293 R12: 00000000134ab2b4
[157707.740102] R13: 000000008386615c R14: 00000000000000e3 R15: 00000000000000e3
[157707.740102] FS: 00007f1736ad9700(0000) GS:ffff88021fc00000(0000) knlGS:0000000000000000
[157707.740102] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[157707.740102] CR2: 00007fd8a39f8000 CR3: 0000000211ad7000 CR4: 00000000000407f0
[157707.740102] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[157707.740102] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[157707.740102] Process rtpproxy (pid: 1256, threadinfo ffff880212fae000, task ffff88020fc71700)
[157707.740102] Stack:
[157707.740102] ffff880212faf5a8 0000000000015d91 134ab2b400000008 000008f58386615c
[157707.740102] 00015d910000a20f a080527800000014 3a78560000d1fa00 564812de1a006045
[157707.740102] ffff880212faf618 ffffffff81872e20 0000000000000000 ffff880210ca9000
[157707.740102] Call Trace:
[157707.740102] [<ffffffff8169e7de>] ? _raw_spin_lock+0xe/0x20
[157707.740102] [<ffffffff815a0958>] ? sch_direct_xmit+0x88/0x1c0
[157707.740102] [<ffffffff81090833>] ? update_cpu_power+0x63/0x100
[157707.740102] [<ffffffff810909c3>] ? update_group_power+0xf3/0x100
[157707.740102] [<ffffffff81090db2>] ? update_sd_lb_stats+0x3e2/0x5f0
[157707.740102] [<ffffffffa00f34a0>] obsf_tg+0x9c0/0x133c [xt_OBSF]
[157707.740102] [<ffffffff81090ff9>] ? find_busiest_group+0x39/0x4a0
[157707.740102] [<ffffffff81091541>] ? load_balance+0xe1/0x4a0
[157707.740102] [<ffffffffa00e41a5>] ipt_do_table+0x315/0x450 [ip_tables]
[157707.740102] [<ffffffffa00dd129>] ipt_mangle_out+0x99/0x100 [iptable_mangle]
[157707.740102] [<ffffffffa00dd1eb>] iptable_mangle_hook+0x5b/0x60 [iptable_mangle]
[157707.740102] [<ffffffff815aded5>] nf_iterate+0x85/0xc0
[157707.740102] [<ffffffff815b8e50>] ? ip_forward_options+0x200/0x200
[157707.740102] [<ffffffff815adf85>] nf_hook_slow+0x75/0x150
[157707.740102] [<ffffffff815b8e50>] ? ip_forward_options+0x200/0x200
[157707.740102] [<ffffffff815babb2>] __ip_local_out+0xa2/0xb0
[157707.740102] [<ffffffff815babd6>] ip_local_out+0x16/0x30
[157707.740102] [<ffffffff815bbefb>] ip_send_skb+0x1b/0x50
[157707.740102] [<ffffffff815df1d1>] udp_send_skb+0x111/0x2a0
[157707.740102] [<ffffffff815b9070>] ? ip_setup_cork+0x150/0x150
[157707.740102] [<ffffffff815e0286>] udp_sendmsg+0x316/0x960
[157707.740102] [<ffffffff815eabc4>] inet_sendmsg+0x64/0xb0
[157707.740102] [<ffffffff812f31b7>] ? apparmor_socket_sendmsg+0x17/0x20
[157707.740102] [<ffffffff8156a437>] sock_sendmsg+0x117/0x130
[157707.740102] [<ffffffff8119a510>] ? __pollwait+0xf0/0xf0
[157707.740102] [<ffffffff8119a510>] ? __pollwait+0xf0/0xf0
[157707.740102] [<ffffffff8119a510>] ? __pollwait+0xf0/0xf0
[157707.740102] [<ffffffff8156b58d>] ? move_addr_to_user+0xbd/0xd0
[157707.740102] [<ffffffff8156ce7a>] ? move_addr_to_kernel+0x5a/0xa0
[157707.740102] [<ffffffff8156d91d>] sys_sendto+0x13d/0x190
[157707.740102] [<ffffffff8103fcc9>] ? kvm_clock_read+0x19/0x20
[157707.740102] [<ffffffff8103fcd9>] ? kvm_clock_get_cycles+0x9/0x10
[157707.740102] [<ffffffff810a3bd7>] ? getnstimeofday+0x57/0xe0
[157707.740102] [<ffffffff810a3cca>] ? do_gettimeofday+0x1a/0x50
[157707.740102] [<ffffffff816a7029>] system_call_fastpath+0x16/0x1b
[157707.740102] Code: f7 f1 48 8b 8d 70 fe ff ff 4c 63 f2 41 89 d7 49 69 c6 68 01 00 00 48 01 c3 48 8b 83 58 01 00 00 48 2d 58 01 00 00 48 89 c2 eb 20 <44> 39 62 04 0f 85 c0 02 00 00 44 39 6a 08 0f 85 b6 02 00 00 48
[157707.740102] RIP [<ffffffffa00f3fa0>] pkt_queue+0x184/0x48a [xt_OBSF]
[157707.740102] RSP <ffff880212faf578>
[157707.736203] general protection fault: 0000 [#1] SMP
Says that you are doing something horrible in memory (e.g dereferencing a null pointer)
[157707.740102] RIP: 0010:[<ffffffffa00f3fa0>] [<ffffffffa00f3fa0>] pkt_queue+0x184/0x48a
This line is reporting to you the instruction pointer value when your module crashed; it says that it died inside a function named "pkt_queue" after an offset of "0x184".
(btw, the same value appears in the first crash dump, 388 in decimal = 0x184)
Now, you can use objdump to dump the assembly + debug information about your code and you add the address of the function pkt_queue to 0x184 and you get to the offending instruction.
Let's say your pkt_queue function appears(unreasonably hypothetical) at address 0x01 in objdump, it means you should look at line: 0x184 + 0x01 = 0x185 in the assembly to see what's going on.
Objdump allows you view the source + the assembly and line numbers:
objdump -S your_object_file.o this will not only list the assembly but also the corresponding source code assuming the debug symbols are added when compiling.
Oh and for your future reference:https://opensourceforu.com/2011/01/understanding-a-kernel-oops/
You can also use:
eu-addr2line -f -e object_file.o pkt_queue+0x184
Where -f tells the command that the function name is used with line number and -e is the executable or object file containing the line number.
there is also the script scripts/decode_stacktrace.sh in the kernel source code.
You should enable CONFIG_DEBUG_INFO then run the script:
./scripts/decode_stacktrace.sh /path/to/vmlinux /path/to/kernel/tree /path/to/modules/dir < dmesg.log
for example starting in the kernel source code root:
make O=~/kbuild/x86/ -j9
cd ~/kbuild/x86/
make INSTALL_MOD_PATH=~/modpath modules_install
cd -
./scripts/decode_stacktrace.sh ~/kbuild/x86/vmlinux . ~/modpath < crash.log
see https://lwn.net/Articles/592724/