I'm developing a kernel module that I want to run on my router. The router model is DGN2200v2 by Netgear. It's running Linux 2.6.30 on MIPS. My problem is that when I load my module it seems that my module_init isn't getting called. I tried to narrow it down by modifying my module_init to return -3 (which indicates an error?) and insmod still reports success. I can see my module in the output of lsmod, but I don't see my printk output using dmesg.
For starters, I wanted to create the simplest possible module:
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
static int my_init(void)
{
printk(KERN_EMERG "init_module() called\n");
return -3;
}
static void my_cleanup(void)
{
printk(KERN_EMERG "cleanup_module() called\n");
}
module_init(my_init);
module_exit(my_cleanup);
This is the Makefile I'm using:
TOOLCHAIN=/home/user/buildroot-2016.08/output/host/usr/bin/mips-buildroot-linux-uclibc-
ARCH=mips
CC = $(TOOLCHAIN)gcc
KBUILD_CFLAGS:=.
EXTRA_CFLAGS := -I/home/user/buildroot-2016.08/output/build/linux-headers-2.6.30/include\
-I/home/user/buildroot-2016.08/output/build/linux-headers-2.6.30/arch/mips/include/asm/mach-mipssim\
-I/home/user/buildroot-2016.08/output/build/linux-headers-2.6.30/arch/mips/include/asm/mach-generic\
-fno-pic -mno-abicalls -O2
obj-m := module.o
KDIR := /home/user/buildroot-2016.08/output/build/linux-headers-2.6.30
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
I'm running make like so:
make ARCH=mips CROSS_COMPILE=/home/user/buildroot-2016.08/output/host/usr/bin/mips-buildroot-linux-uclibc-
which passes successfully.
As you can see, I'm using Buildroot which I (hopefully) configured correctly. I can paste my .config if needed.
I ran objdump on my module and didn't find a problem. In particular, the module_init symbol seems to point to the same place as my my_init function, and it seems to have the code I expect it to:
module.ko: file format elf32-tradbigmips
module.ko
architecture: mips:isa32, flags 0x00000011:
HAS_RELOC, HAS_SYMS
start address 0x00000000
private flags = 50001001: [abi=O32] [mips32] [not 32bitmode] [noreorder]
MIPS ABI Flags Version: 0
ISA: MIPS32
GPR size: 32
CPR1 size: 0
CPR2 size: 0
FP ABI: Soft float
ISA Extension: None
ASEs:
None
FLAGS 1: 00000001
FLAGS 2: 00000000
Sections:
Idx Name Size VMA LMA File off Algn
0 .MIPS.abiflags 00000018 00000000 00000000 00000038 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA, LINK_ONCE_SAME_SIZE
1 .reginfo 00000018 00000000 00000000 00000050 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA, LINK_ONCE_SAME_SIZE
2 .note.gnu.build-id 00000024 00000018 00000018 00000068 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .text 00000040 00000000 00000000 00000090 2**4
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
4 .rodata.str1.4 00000038 00000000 00000000 000000d0 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .modinfo 0000005c 00000000 00000000 00000108 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
6 .data 00000000 00000000 00000000 00000170 2**4
CONTENTS, ALLOC, LOAD, DATA
7 .gnu.linkonce.this_module 0000014c 00000000 00000000 00000170 2**2
CONTENTS, ALLOC, LOAD, RELOC, DATA, LINK_ONCE_DISCARD
8 .bss 00000000 00000000 00000000 000002c0 2**4
ALLOC
9 .comment 00000040 00000000 00000000 000002c0 2**0
CONTENTS, READONLY
10 .pdr 00000040 00000000 00000000 00000300 2**2
CONTENTS, RELOC, READONLY
11 .gnu.attributes 00000010 00000000 00000000 00000340 2**0
CONTENTS, READONLY
12 .mdebug.abi32 00000000 00000000 00000000 00000350 2**0
CONTENTS, READONLY
SYMBOL TABLE:
00000000 l d .MIPS.abiflags 00000000 .MIPS.abiflags
00000000 l d .reginfo 00000000 .reginfo
00000018 l d .note.gnu.build-id 00000000 .note.gnu.build-id
00000000 l d .text 00000000 .text
00000000 l d .rodata.str1.4 00000000 .rodata.str1.4
00000000 l d .modinfo 00000000 .modinfo
00000000 l d .data 00000000 .data
00000000 l d .gnu.linkonce.this_module 00000000 .gnu.linkonce.this_module
00000000 l d .bss 00000000 .bss
00000000 l d .comment 00000000 .comment
00000000 l d .pdr 00000000 .pdr
00000000 l d .gnu.attributes 00000000 .gnu.attributes
00000000 l d .mdebug.abi32 00000000 .mdebug.abi32
00000000 l df *ABS* 00000000 module.c
00000000 l F .text 0000002c my_init
0000002c l F .text 00000014 my_cleanup
00000000 l .rodata.str1.4 00000000 $LC0
0000001c l .rodata.str1.4 00000000 $LC1
00000000 l df *ABS* 00000000 module.mod.c
00000000 l O .modinfo 00000023 __mod_srcversion23
00000024 l O .modinfo 00000009 __module_depends
00000030 l O .modinfo 0000002c __mod_vermagic5
00000000 g O .gnu.linkonce.this_module 0000014c __this_module
0000002c g F .text 00000014 cleanup_module
00000000 g F .text 0000002c init_module
00000000 *UND* 00000000 printk
Disassembly of section .MIPS.abiflags:
00000000 <.MIPS.abiflags>:
0: 00002001 movf a0,zero,$fcc0
4: 01000003 0x1000003
...
10: 00000001 movf zero,zero,$fcc0
14: 00000000 nop
Disassembly of section .reginfo:
00000000 <.reginfo>:
0: a2000014 sb zero,20(s0)
...
14: 00007fef 0x7fef
Disassembly of section .note.gnu.build-id:
00000018 <.note.gnu.build-id>:
18: 00000004 sllv zero,zero,zero
1c: 00000014 0x14
20: 00000003 sra zero,zero,0x0
24: 474e5500 c1 0x14e5500
28: c8e5d654 lwc2 $5,-10668(a3)
2c: cb477d3d lwc2 $7,32061(k0)
30: dfa48d71 ldc3 $4,-29327(sp)
34: c2ea16da ll t2,5850(s7)
38: f6bcae7d sdc1 $f28,-20867(s5)
Disassembly of section .text:
00000000 <init_module>:
0: 27bdffe8 addiu sp,sp,-24
4: 3c040000 lui a0,0x0
4: R_MIPS_HI16 $LC0
8: 3c020000 lui v0,0x0
8: R_MIPS_HI16 printk
c: afbf0014 sw ra,20(sp)
10: 24420000 addiu v0,v0,0
10: R_MIPS_LO16 printk
14: 0040f809 jalr v0
18: 24840000 addiu a0,a0,0
18: R_MIPS_LO16 $LC0
1c: 8fbf0014 lw ra,20(sp)
20: 2402fffd li v0,-3
24: 03e00008 jr ra
28: 27bd0018 addiu sp,sp,24
modinfo output also matches what I expect (same modinfo output as for another .ko that's found on the router, except for the srcversion which my module has but the other module on the router doesn't):
filename: /home/user/module/module.ko
srcversion: B0BADBA395A121CF49B74DC
depends:
vermagic: 2.6.30 mod_unload MIPS32_R1 32BIT
It's entirely possible that I messed something up in my Buildroot configuration, or something doesn't quite match the CPU type of the router, but my init code is so minimal that I'm out of ideas as to what could be wrong.
It turns out that the problem was related to a different kernel configuration between my development environment and the router. Specifically, my kernel was using CONFIG_UNUSED_SYMBOLS whereas the router's was not.
The reason this caused a problem even in a trivial module is that when the kernel loads a module it doesn't only look up the module_init symbol in the module's symbol table. Rather, it reads the module struct from the module (from the .gnu.linkonce.this_module section), and then calls the init module through that struct.
The offset of the init function pointer inside the module struct depends on the kernel configuration, which explains why the kernel can't find the init function if the configuration is different.
Thanks to Sam Protsenko for investing a lot of time in helping me crack this!
Related
I'm compiling someone else's code using GCC and the output from GCC shows a large .bss section:
text data bss dec
9468 1080 10892 21440
I run the following command to generate information related to each section so I can try and figure out what is taking up so much space in the .bss section:
objdump -x output.elf > output.dmp
Here is the output of the sections:
Sections:
Idx Name Size VMA LMA File off Algn
0 .mstack 00001000 20000000 20000000 00030000 2**0
ALLOC
1 .pstack 00001000 20001000 20001000 00030000 2**0
ALLOC
2 .nocache 00000000 30040000 30040000 0002042c 2**2
CONTENTS
3 .eth 00000000 30040000 30040000 0002042c 2**2
CONTENTS
4 .vectors 000002a0 08000000 08000000 00010000 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
5 .xtors 0000000c 080002a0 080002a0 000102a0 2**2
CONTENTS, ALLOC, LOAD, DATA
6 .text 00002054 080002b0 080002b0 000102b0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
7 .init 00000004 08002304 08002304 00012304 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
8 .fini 00000004 08002308 08002308 00012308 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
9 .rodata 00000200 0800230c 0800230c 0001230c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
10 .data 0000042c 24000000 0800250c 00020000 2**3
CONTENTS, ALLOC, LOAD, DATA
11 .bss 00000a8c 24000430 08002938 00020430 2**3
ALLOC
12 .ram0_init 00000000 24000ebc 24000ebc 0002042c 2**2
CONTENTS
13 .ram0 00000000 24000ebc 24000ebc 0002042c 2**2
CONTENTS
14 .ram1_init 00000000 30000000 30000000 0002042c 2**2
CONTENTS
15 .ram1 00000000 30000000 30000000 0002042c 2**2
CONTENTS
16 .ram2_init 00000000 30000000 30000000 0002042c 2**2
CONTENTS
17 .ram2 00000000 30000000 30000000 0002042c 2**2
CONTENTS
18 .ram3_init 00000000 30040000 30040000 0002042c 2**2
CONTENTS
19 .ram3 00000000 30040000 30040000 0002042c 2**2
CONTENTS
20 .ram4_init 00000000 38000000 38000000 0002042c 2**2
CONTENTS
21 .ram4 00000000 38000000 38000000 0002042c 2**2
CONTENTS
22 .ram5_init 00000000 20002000 20002000 0002042c 2**2
CONTENTS
23 .ram5 00000000 20002000 20002000 0002042c 2**2
CONTENTS
24 .ram6_init 00000000 00000000 00000000 0002042c 2**2
CONTENTS
25 .ram6 00000000 00000000 00000000 0002042c 2**2
CONTENTS
26 .ram7_init 00000000 38800000 38800000 0002042c 2**2
CONTENTS
27 .ram7 00000000 38800000 38800000 0002042c 2**2
CONTENTS
28 .ARM.attributes 00000030 00000000 00000000 0002042c 2**0
CONTENTS, READONLY
29 .comment 0000004c 00000000 00000000 0002045c 2**0
CONTENTS, READONLY
30 .debug_info 0001bdb6 00000000 00000000 000204a8 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
31 .debug_abbrev 000043c3 00000000 00000000 0003c25e 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
32 .debug_aranges 00000ae8 00000000 00000000 00040628 2**3
CONTENTS, READONLY, DEBUGGING, OCTETS
33 .debug_ranges 00000918 00000000 00000000 00041110 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
34 .debug_line 00007b18 00000000 00000000 00041a28 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
35 .debug_str 000031ff 00000000 00000000 00049540 2**0
CONTENTS, READONLY, DEBUGGING, OCTETS
36 .debug_frame 000027e8 00000000 00000000 0004c740 2**2
CONTENTS, READONLY, DEBUGGING, OCTETS
As you can see the .bss section only takes up 2700 bytes. Could someone please explain how I can track down where the extra bytes are coming from? Thank you.
Also not sure if this is relevant, but I'm using an ARM Cortex-M7 processor.
Nevermind. I figured out where the extra bytes were coming from.
The sections .mstack and .pstack have 4Kbytes allocated each and are being placed in RAM. When I take the 8Kbytes total (.mstack + .pstack) and add in the size of the .bss section, I get the GCC output value of 10892.
I have a arm board on which I am running yocto with kernel 4.1.15. While I am running my python program I get following kernel error frequently but randomly
Unable to handle kernel paging request at virtual address 7f101f7c
pgd = 80004000
[7f101f7c] *pgd=8c6c4811, *pte=00000000, *ppte=00000000
Internal error: Oops: 80000007 [#1] PREEMPT SMP ARM
Modules linked in: wilc3000(O) at_pwr_dev(O) pn5xx_i2c [last unloaded: at_pwr_dev]
CPU: 0 PID: 1336 Comm: DebugThread Tainted: G O 4.1.15-1.2.0+g77f6154
Hardware name: Freescale i.MX6 Ultralite (Device Tree)
task: 8c73b900 ti: 8c8d6000 task.ti: 8c8d6000
PC is at 0x7f101f7c
LR is at _raw_spin_unlock_irqrestore+0x28/0x54
pc : [<7f101f7c>] lr : [<807e1238>] psr: 600f0013
sp : 8c8d7f30 ip : 00000000 fp : 00000000
r10: 7f107d30 r9 : 7f107d20 r8 : 7f107f48
r7 : 00000000 r6 : 8c57b000 r5 : 7f107f48 r4 : 8c54aa00
r3 : 00000000 r2 : 00000000 r1 : 20000013 r0 : ffffffc2
Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
Control: 10c53c7d Table: 8c52c06a DAC: 00000015
Process DebugThread (pid: 1336, stack limit = 0x8c8d6210)
Stack: (0x8c8d7f30 to 0x8c8d8000) 7f20: 8c8063a0 00000000 8c8d6000 00000000
7f40: 00000000 00000000 00000000 8c975c40 8c54aa00 7f101f28 00000000 00000000
7f60: 00000000 8004d070 00000000 00000000 7ee95a5c 8c54aa00 00000000 00000000
7f80: 8c8d7f80 8c8d7f80 00000000 00000000 8c8d7f90 8c8d7f90 8c8d7fac 8c975c40
7fa0: 8004cf94 00000000 00000000 8000f528 00000000 00000000 00000000 00000000
7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 7a9ce301 72611f00
[<807e1238>] (_raw_spin_unlock_irqrestore) from [<00000000>] ( (null))
Code: bad PC value
How can I debug this error considering the fact that I don't have access to JTAG on this board. What is the meaning of Code: bad PC value? If there any to find anything regarding problem from this log?
pc : [<7f101f7c>] lr : [<807e1238>] psr: 600f0013
In order to translate it into source code line:
arm-none-linux-gnueabi-addr2line -f -e vmlinux 7f101f7c
You must use your addr2line command.
I have some assembler for the Microblaze that I want to load at address 0x00000000 (ie to ensure it is executed on a reset).
I have a linker script that should do this (I think):
SECTIONS
{
ENTRY(_start)
. = 0x0000;
.vectors.reset : { *(.vectors.reset) }
. = 0x0008;
.vectors.sw_exception : { *(.vectors.sw_exception) }
. = 0x0010;
.vectors.interrupt : { *(.vectors.interrupt) }
. = 0x0018;
.vectors.hw_exception : { *(.vectors.hw_exception) }
. = 0x100;
.text : { *(.text) }
.data : { *(.data) }
.bss : { *(.bss) }
}
But when the code is compiled it seems to be offset by 0x1000:
objdump -h startup.MICROBLAZE.elf
startup.MICROBLAZE.elf: file format elf32-big
Sections:
Idx Name Size VMA LMA File off Algn
0 .vectors.reset 00000008 00000000 00000000 00001000 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .vectors.sw_exception 00000008 00000008 00000008 00001008 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
2 .vectors.interrupt 00000008 00000010 00000010 00001010 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
3 .vectors.hw_exception 00000008 00000018 00000018 00001018 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
4 .text 00000020 00000100 00000100 00001100 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
Where does that offset come from and how can I supress/control it?
Edit: It seems the 0x1000 offset is the physical offset of the code section in the compiled file/object - is that correct?
According to your objdump listing everything is suppose to be laid out as you expect. I.e. VMA and LMA of your .text section are pointing to address 0x100.
The offset 0x1000 as you correctly guessed is offset of the .text section inside ELF file. But this section will be loaded to address 0x100.
If you try to disassemble your ELF with
$ objdump -S startup.MICROBLAZE.elf
you will see the proper instructions layout. It is also helpful to produce MAP file on a linkage stage with -Wl,-Map,output.map gcc flags.
This works with MSVC (compile and link).
extern void (_TMDLLENTRY * _TMDLLENTRY tpsetunsol _((void (_TMDLLENTRY *)(char _TM_FAR *, long, long)))) _((char _TM_FAR *, long, long));
But with GCC compiling is ok but linking phase fails with undefined reference pointing to this. Is this declaration already GCC compatible or is there possibly some other problem?
#define _TMDLLENTRY __stdcall
#define _TM_FAR
nm output from MSVC created object file:
00000000 N .debug$S
00000000 N .debug$S
00000000 N .debug$S
00000000 N .debug$S
00000000 N .debug$T
00000000 i .drectve
00000000 r .rdata
00000000 r .rdata
00000000 r .rdata
00000000 t .text
00000000 t .text
00000000 t .text
U #__security_check_cookie#4
00aa766f a #comp.id
00000001 a #feat.00
U ___security_cookie
U __imp__CloseClipboard#0
U __imp__EmptyClipboard#0
U __imp__GlobalAlloc#8
U __imp__GlobalLock#4
U __imp__GlobalUnlock#4
U __imp__MessageBoxA#16
U __imp__OpenClipboard#4
U __imp__SetClipboardData#8
00000000 T _CheckBroadcast#0
00000000 T _Inittpsetunsol#0
U _memcpy
U _memset
U _sprintf
U _tpchkunsol#0
U _tpsetunsol#4
00000000 T _vCSLMsgHandler#12
nm output from MinGW compiler:
nm Release/broadc.o
00000000 b .bss
00000000 d .data
00000000 N .debug_abbrev
00000000 N .debug_aranges
00000000 N .debug_info
00000000 N .debug_line
00000000 N .debug_loc
00000000 r .eh_frame
00000000 r .rdata
00000000 t .text
00000104 T _CheckBroadcast#0
U _CloseClipboard#0
U _EmptyClipboard#0
U _GlobalAlloc#8
U _GlobalLock#4
U _GlobalUnlock#4
0000010c T _Inittpsetunsol#0
U _MessageBoxA#16
U _OpenClipboard#4
U _SetClipboardData#8
U _sprintf
U _tpchkunsol#0
U _tpsetunsol
00000000 T _vCSLMsgHandler#12
Edit: Looking at _tpsetunsol symbol it is quite evident that MSVC and GCC produce different symbols.
GCC produces: U _tpsetunsol and MSVC: U _tpsetunsol#4
Is there a way to have GCC produce same symbol than MSVC?
Edit2: output from nm wtuxws32.lib (that is where tpsetunsol is defined)
WTUXWS32.dll:
00000000 i .idata$4
00000000 i .idata$5
00000000 t .text
00000000 I __imp__tpsetunsol#4
U __IMPORT_DESCRIPTOR_WTUXWS32
00000000 T _tpsetunsol#4
Is this impossible to get to work with GCC? Thanks.
I am a little confused as to where uninitialized global variables go in the ELF file. I have this simple program to test in which sections the variables will be located:
const int a = 11;
int b = 10;
int c;
int main()
{
return 0;
}
I know that uninitialized global variable should be put into .bss section of ELF file, but objdump -h gives me the following output:
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 0000000a 00000000 00000000 00000034 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000004 00000000 00000000 00000040 2**2
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 00000000 00000000 00000044 2**2
ALLOC
3 .rodata 00000004 00000000 00000000 00000044 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .comment 00000024 00000000 00000000 00000048 2**0
CONTENTS, READONLY
5 .note.GNU-stack 00000000 00000000 00000000 0000006c 2**0
CONTENTS, READONLY
So the variable a goes to .rodata, b goes to .data, and c goes nowhere? When i change the code to:
int c = 0;
everything is as expected - .bss section has the length 4, but what happens with the variable c when it is not initialized?
It goes into a "common section". You can see it with objdump -t or by using nm.
I'm not quite sure I understand what this is about, but the reference to the ld -warn-common flag says this:
int i;
A common symbol. If there are only
(one or more) common symbols for a
variable, it goes in the uninitialized
data area of the output file. The
linker merges multiple common symbols
for the same variable into a single
symbol. If they are of different
sizes, it picks the largest size. The
linker turns a common symbol into a
declaration, if there is a definition
of the same variable.
(Found via the nm man page.) There is more information after that in the man page itself.