There is a remote 64-bit *nix server that can compile a user-provided code (which should be written in Rust, but I don't think it matters since it uses LLVM). I don't know which compiler/linker flags it uses, but the compiled ELF executable looks weird - it has 4 LOAD segments:
$ readelf -e executable
...
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
...
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000004138 0x0000000000004138 R 0x1000
LOAD 0x0000000000005000 0x0000000000005000 0x0000000000005000
0x00000000000305e9 0x00000000000305e9 R E 0x1000
LOAD 0x0000000000036000 0x0000000000036000 0x0000000000036000
0x000000000000d808 0x000000000000d808 R 0x1000
LOAD 0x0000000000043da0 0x0000000000044da0 0x0000000000044da0
0x0000000000002290 0x00000000000024a0 RW 0x1000
...
On my own system all executables that I was looking at only have 2 LOAD segments:
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
...
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x00000000003000c0 0x00000000003000c0 R E 0x200000
LOAD 0x00000000003002b0 0x00000000005002b0 0x00000000005002b0
0x00000000000776c8 0x000000000009b200 RW 0x200000
...
What are the circumstances (compiler/linker versions, flags etc) under which a compiler might build an ELF with 4 LOAD segments?
What is the point of having 4 LOAD segments? I imagine that having a segment with read but not execute permission might help against certain exploits, but why have two such segments?
A typical BFD-ld or Gold linked Linux executable has 2 loadable segments, with the ELF header merged with .text and .rodata into the first RE segment, and .data, .bss and other writable sections merged into the second RW segment.
Here is the typical section to segment mapping:
$ echo "int foo; int main() { return 0;}" | clang -xc - -o a.out-gold -fuse-ld=gold
$ readelf -Wl a.out-gold
Elf file type is EXEC (Executable file)
Entry point 0x400420
There are 9 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000400040 0x0000000000400040 0x0001f8 0x0001f8 R 0x8
INTERP 0x000238 0x0000000000400238 0x0000000000400238 0x00001c 0x00001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x000000 0x0000000000400000 0x0000000000400000 0x0006b0 0x0006b0 R E 0x1000
LOAD 0x000e18 0x0000000000401e18 0x0000000000401e18 0x0001f8 0x000200 RW 0x1000
DYNAMIC 0x000e28 0x0000000000401e28 0x0000000000401e28 0x0001b0 0x0001b0 RW 0x8
NOTE 0x000254 0x0000000000400254 0x0000000000400254 0x000020 0x000020 R 0x4
GNU_EH_FRAME 0x00067c 0x000000000040067c 0x000000000040067c 0x000034 0x000034 R 0x4
GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0x10
GNU_RELRO 0x000e18 0x0000000000401e18 0x0000000000401e18 0x0001e8 0x0001e8 RW 0x8
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .dynsym .dynstr .gnu.hash .hash .gnu.version .gnu.version_r .rela.dyn .init .text .fini .rodata .eh_frame .eh_frame_hdr
03 .fini_array .init_array .dynamic .got .got.plt .data .bss
04 .dynamic
05 .note.ABI-tag
06 .eh_frame_hdr
07
08 .fini_array .init_array .dynamic .got .got.plt
This optimizes the number of mmaps that the kernel must perform to load such executable, but at a security cost: the data in .rodata shouldn't be executable, but is (because it's merged with .text, which must be executable). This may significantly increase the attack surface for someone trying to hijack a process.
Newer Linux systems, in particular using LLD to link binaries, prioritize security over speed, and put ELF header and .rodata into the first R-only segment, resulting in 3 load segments and improved security. Here is a typical mapping:
$ echo "int foo; int main() { return 0;}" | clang -xc - -o a.out-lld -fuse-ld=lld
$ readelf -Wl a.out-lld
Elf file type is EXEC (Executable file)
Entry point 0x201000
There are 10 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000200040 0x0000000000200040 0x000230 0x000230 R 0x8
INTERP 0x000270 0x0000000000200270 0x0000000000200270 0x00001c 0x00001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x000000 0x0000000000200000 0x0000000000200000 0x000558 0x000558 R 0x1000
LOAD 0x001000 0x0000000000201000 0x0000000000201000 0x000185 0x000185 R E 0x1000
LOAD 0x002000 0x0000000000202000 0x0000000000202000 0x001170 0x002005 RW 0x1000
DYNAMIC 0x003010 0x0000000000203010 0x0000000000203010 0x000150 0x000150 RW 0x8
GNU_RELRO 0x003000 0x0000000000203000 0x0000000000203000 0x000170 0x001000 R 0x1
GNU_EH_FRAME 0x000440 0x0000000000200440 0x0000000000200440 0x000034 0x000034 R 0x1
GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0
NOTE 0x00028c 0x000000000020028c 0x000000000020028c 0x000020 0x000020 R 0x4
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .rodata .dynsym .gnu.version .gnu.version_r .gnu.hash .hash .dynstr .rela.dyn .eh_frame_hdr .eh_frame
03 .text .init .fini
04 .data .tm_clone_table .fini_array .init_array .dynamic .got .bss
05 .dynamic
06 .fini_array .init_array .dynamic .got
07 .eh_frame_hdr
08
09 .note.ABI-tag
Not to be left behind, the newer BFD-ld (my version is 2.31.1) also makes ELF header and .rodata read-only, but fails to merge two R-only segments into one, resulting in 4 loadable segments:
$ echo "int foo; int main() { return 0;}" | clang -xc - -o a.out-bfd -fuse-ld=bfd
$ readelf -Wl a.out-bfd
Elf file type is EXEC (Executable file)
Entry point 0x401020
There are 11 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000040 0x0000000000400040 0x0000000000400040 0x000268 0x000268 R 0x8
INTERP 0x0002a8 0x00000000004002a8 0x00000000004002a8 0x00001c 0x00001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x000000 0x0000000000400000 0x0000000000400000 0x0003f8 0x0003f8 R 0x1000
LOAD 0x001000 0x0000000000401000 0x0000000000401000 0x00018d 0x00018d R E 0x1000
LOAD 0x002000 0x0000000000402000 0x0000000000402000 0x000110 0x000110 R 0x1000
LOAD 0x002e40 0x0000000000403e40 0x0000000000403e40 0x0001e8 0x0001f0 RW 0x1000
DYNAMIC 0x002e50 0x0000000000403e50 0x0000000000403e50 0x0001a0 0x0001a0 RW 0x8
NOTE 0x0002c4 0x00000000004002c4 0x00000000004002c4 0x000020 0x000020 R 0x4
GNU_EH_FRAME 0x002004 0x0000000000402004 0x0000000000402004 0x000034 0x000034 R 0x4
GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0x10
GNU_RELRO 0x002e40 0x0000000000403e40 0x0000000000403e40 0x0001c0 0x0001c0 R 0x1
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn
03 .init .text .fini
04 .rodata .eh_frame_hdr .eh_frame
05 .init_array .fini_array .dynamic .got .got.plt .data .bss
06 .dynamic
07 .note.ABI-tag
08 .eh_frame_hdr
09
10 .init_array .fini_array .dynamic .got
Finally, some of these choices are affected by the --(no)rosegment (or -Wl,z,noseparate-code for BFD ld) linker option.
Related
Well I have written a bootloader in assembly and trying to load a C kernel from it.
This is the bootloader:
bits 16
xor ax,ax
jmp 0x0000:boot
extern kernel_main
global boot
boot:
mov ah, 0x02 ; load second stage to memory
mov al, 1 ; numbers of sectors to read into memory
mov dl, 0x80 ; sector read from fixed/usb disk ;0 for floppy; 0x80 for hd
mov ch, 0 ; cylinder number
mov dh, 0 ; head number
mov cl, 2 ; sector number
mov bx, 0x8000 ; load into es:bx segment :offset of buffer
int 0x13 ; disk I/O interrupt
mov ax, 0x2401
int 0x15 ; enable A20 bit
mov ax, 0x3
int 0x10 ; set vga text mode 3
cli
lgdt [gdt_pointer] ; load the gdt table
mov eax, cr0
or eax,0x1 ; set the protected mode bit on special CPU reg cr0
mov cr0, eax
jmp CODE_SEG:boot2 ; long jump to the code segment
gdt_start:
dq 0x0
gdt_code:
dw 0xFFFF
dw 0x0
db 0x0
db 10011010b
db 11001111b
db 0x0
gdt_data:
dw 0xFFFF
dw 0x0
db 0x0
db 10010010b
db 11001111b
db 0x0
gdt_end:
gdt_pointer:
dw gdt_end - gdt_start
dd gdt_start
CODE_SEG equ gdt_code - gdt_start
DATA_SEG equ gdt_data - gdt_start
bits 32
boot2:
mov ax, DATA_SEG
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
; mov esi,hello
; mov ebx,0xb8000
;.loop:
; lodsb
; or al,al
; jz haltz
; or eax,0x0100
; mov word [ebx], ax
; add ebx,2
; jmp .loop
;haltz:
;hello: db "Hello world!",0
mov esp,kernel_stack_top
jmp kernel_main
cli
hlt
times 510 -($-$$) db 0
dw 0xaa55
section .bss
align 4
kernel_stack_bottom: equ $
resb 16384 ; 16 KB
kernel_stack_top:
and this is the C kernel:
__asm__("cli\n");
void kernel_main(void){
const char string[] = "012345678901234567890123456789012345678901234567890123456789012";
volatile unsigned char* vid_mem = (unsigned char*) 0xb8000;
int j=0;
while(string[j]!='\0'){
*vid_mem++ = (unsigned char) string[j++];
*vid_mem++ = 0x09;
}
for(;;);
}
Now I am compiling both the source separately into an ELF output file.
And linking them through a linker script and output a raw binary file and load it with qemu.
Linker script:
ENTRY(boot)
OUTPUT_FORMAT("binary")
SECTIONS{
. = 0x7c00;
.boot1 : {
*(.boot)
}
.kernel : AT(0x7e00){
*(.text)
*(.rodata)
*(.data)
_bss_start = .;
*(.bss)
*(COMMON)
_bss_end = .;
*(.comment)
*(.symtab)
*(.shstrtab)
*(.strtab)
}
/DISCARD/ : {
*(.eh_frame)
}
}
with a build script:
nasm -f elf32 boot.asm -o boot.o
/home/rakesh/Desktop/cross-compiler/i686-elf-4.9.1-Linux-x86_64/bin/i686-elf-gcc -m32 kernel.c -o kernel.o -e kernel_main -Ttext 0x0 -nostdlib -ffreestanding -std=gnu99 -mno-red-zone -fno-exceptions -nostdlib -Wall -Wextra
/home/rakesh/Desktop/cross-compiler/i686-elf-4.9.1-Linux-x86_64/bin/i686-elf-ld boot.o kernel.o -o kernel.bin -T linker3.ld
qemu-system-x86_64 kernel.bin
But I am running into a little problem.
notice that string in the C kernel
const char string[] = "012345678901234567890123456789012345678901234567890123456789012";
when its size is equal to or less than 64 bytes (along with the null termination). then the program works correctly.
however when the string size increases from 64 bytes then the program seems to not work
I was trying to debug it myself and observed that when the string size is less than or equal to 64 bytes then the output ELF file, the kernel.o have following contents :
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Intel 80386
Version: 0x1
Entry point address: 0x1
Start of program headers: 52 (bytes into file)
Start of section headers: 4412 (bytes into file)
Flags: 0x0
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 1
Size of section headers: 40 (bytes)
Number of section headers: 7
Section header string table index: 4
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .text PROGBITS 00000000 001000 0000bd 00 AX 0 0 1
[ 2] .eh_frame PROGBITS 000000c0 0010c0 000034 00 A 0 0 4
[ 3] .comment PROGBITS 00000000 0010f4 000011 01 MS 0 0 1
[ 4] .shstrtab STRTAB 00000000 001105 000034 00 0 0 1
[ 5] .symtab SYMTAB 00000000 001254 0000a0 10 6 6 4
[ 6] .strtab STRTAB 00000000 0012f4 00002e 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x001000 0x00000000 0x00000000 0x000f4 0x000f4 R E 0x1000
Section to Segment mapping:
Segment Sections...
00 .text .eh_frame
There is no dynamic section in this file.
There are no relocations in this file.
The decoding of unwind sections for machine type Intel 80386 is not currently supported.
Symbol table '.symtab' contains 10 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 00000000 0 NOTYPE LOCAL DEFAULT UND
1: 00000000 0 SECTION LOCAL DEFAULT 1
2: 000000c0 0 SECTION LOCAL DEFAULT 2
3: 00000000 0 SECTION LOCAL DEFAULT 3
4: 00000000 0 FILE LOCAL DEFAULT ABS kernel.c
5: 00000000 0 FILE LOCAL DEFAULT ABS
6: 00000001 188 FUNC GLOBAL DEFAULT 1 kernel_main
7: 000010f4 0 NOTYPE GLOBAL DEFAULT 2 __bss_start
8: 000010f4 0 NOTYPE GLOBAL DEFAULT 2 _edata
9: 000010f4 0 NOTYPE GLOBAL DEFAULT 2 _end
No version information found in this file.
However when the size of the string is more than 64 bytes the contents are like this:
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: EXEC (Executable file)
Machine: Intel 80386
Version: 0x1
Entry point address: 0x1
Start of program headers: 52 (bytes into file)
Start of section headers: 4432 (bytes into file)
Flags: 0x0
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 1
Size of section headers: 40 (bytes)
Number of section headers: 8
Section header string table index: 5
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .text PROGBITS 00000000 001000 000083 00 AX 0 0 1
[ 2] .rodata PROGBITS 00000084 001084 000041 00 A 0 0 4
[ 3] .eh_frame PROGBITS 000000c8 0010c8 000038 00 A 0 0 4
[ 4] .comment PROGBITS 00000000 001100 000011 01 MS 0 0 1
[ 5] .shstrtab STRTAB 00000000 001111 00003c 00 0 0 1
[ 6] .symtab SYMTAB 00000000 001290 0000b0 10 7 7 4
[ 7] .strtab STRTAB 00000000 001340 00002e 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)
There are no section groups in this file.
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x001000 0x00000000 0x00000000 0x00100 0x00100 R E 0x1000
Section to Segment mapping:
Segment Sections...
00 .text .rodata .eh_frame
There is no dynamic section in this file.
There are no relocations in this file.
The decoding of unwind sections for machine type Intel 80386 is not currently supported.
Symbol table '.symtab' contains 11 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 00000000 0 NOTYPE LOCAL DEFAULT UND
1: 00000000 0 SECTION LOCAL DEFAULT 1
2: 00000084 0 SECTION LOCAL DEFAULT 2
3: 000000c8 0 SECTION LOCAL DEFAULT 3
4: 00000000 0 SECTION LOCAL DEFAULT 4
5: 00000000 0 FILE LOCAL DEFAULT ABS kernel.c
6: 00000000 0 FILE LOCAL DEFAULT ABS
7: 00000001 130 FUNC GLOBAL DEFAULT 1 kernel_main
8: 00001100 0 NOTYPE GLOBAL DEFAULT 3 __bss_start
9: 00001100 0 NOTYPE GLOBAL DEFAULT 3 _edata
10: 00001100 0 NOTYPE GLOBAL DEFAULT 3 _end
No version information found in this file.
I noticed that the string is now in the .rodata section with a size of 41 hex or 65 bytes, which has to be mapped to a segment, possibly the 0th segment which is NULL.
And that the program is unable to find the .rodata.
I am unable to make it work. I understand the ELF structure but I don't know how to work with them.
Two serious issues cause most of the problems are:
You load the second sector of the disk to 0x0000:0x8000 when all of the code expect the kernel to be loaded after the bootloader at 0x0000:0x7e00
You compile your kernel.c straight to an executable name kernel.o. You should compile it to a proper object file so it can go through the expected linking phase when you run ld.
To fix the problem with the kernel being loaded into memory at the wrong memory location, change:
mov bx, 0x8000 ; load into es:bx segment :offset of buffer
to:
mov bx, 0x7e00 ; load into es:bx segment :offset of buffer
To fix the issue of compiling kernel.cto an executable ELF file named kernel.o remove the -e kernel_main -Ttext 0x0 and replace it with -c. -c option forces GCC to produce an object file that can be properly linked with LD. Change:
/home/rakesh/Desktop/cross-compiler/i686-elf-4.9.1-Linux-x86_64/bin/i686-elf-gcc -m32 kernel.c -o kernel.o -e kernel_main -Ttext 0x0 -nostdlib -ffreestanding -std=gnu99 -mno-red-zone -fno-exceptions -nostdlib -Wall -Wextra
to:
/home/rakesh/Desktop/cross-compiler/i686-elf-4.9.1-Linux-x86_64/bin/i686-elf-gcc -m32 -c kernel.c -o kernel.o -nostdlib -ffreestanding -std=gnu99 -mno-red-zone -fno-exceptions -Wall -Wextra
Reason for Failure with Longer Strings
The reason the string with less than 64 bytes worked is because the compiler generated code in a position independent way by initializing the array on the stack with immediate values. When the size reached 64 bytes the compiler placed the string into the .rodata section and then initialized the array on the stack by copying it from the .rodata. This made your code position dependent. Your code was loaded at the wrong offsets and had incorrect origin points yielding code referencing incorrect addresses, so it failed.
Other Observations
You should initialize your BSS (.bss) section to 0 before calling kernel_main. This can be done in assembly by iterating through all the bytes from offset _bss_start to offset _bss_end.
The .comment section will be emitted into your binary file wasting bytes as a result. You should put it in the /DISCARD/ section.
You should place the BSS section in your linker script after all the others so it doesn't take up space in kernel.bin
In boot.asm you should set SS:SP (stack pointer) near the beginning before reading disk sectors. It should be set to a place that won't interfere with your code. This is especially important when reading data into memory from disk since you don't know where the BIOS placed the current stack. You don't want to read on top of the current stack area. Setting it just below the bootloader at 0x0000:0x7c00 should work.
Before calling into C code you should clear the direction flag to ensure string instructions use forward movement. You can do this by using the CLD instruction.
In boot.asmyou can make your code more generic by using the boot drive number passed by the BIOS in the DL register rather than hard coding it to the value 0x80 (0x80 being the first hard drive)
You might consider turning on optimization with -O3, or using optimization level -Os to optimize for code size.
Your linker script doesn't quite work the way you expect although it produces the correct results. You never declared .boot section in your NASM file so nothing actually gets placed in the .boot1 output section in the linker script. It works because it gets included in the .text section in the .kernel output section.
It is preferable to remove the padding and boot signature from the assembly file and move it to the linker script
Instead of having your linker script output a binary file directly, it is more useful to output to the default ELF executable format. You can then use OBJCOPY to convert the ELF file to a binary file. This allows you to build with debug information which will appear as part of the ELF executable. The ELF executable can be used to symbolically debug your binary kernel in QEMU.
Rather than use LD directly for linking, use GCC. This has the advantage that the libgcc library can be added without specifying the full path to the library. libgcc is a set of routines that may be needed for C code generation with GCC
Revised source code, linker script and build commands with the observations above taken into account:
boot.asm:
bits 16
section .boot
extern kernel_main
extern _bss_start
extern _bss_len
global boot
jmp 0x0000:boot
boot:
; Place realmode stack pointer below bootloader where it doesn't
; get in our way
xor ax, ax
mov ss, ax
mov sp, 0x7c00
mov ah, 0x02 ; load second stage to memory
mov al, 1 ; numbers of sectors to read into memory
; Remove this, DL is already set by BIOS to current boot drive number
; mov dl, 0x80 ; sector read from fixed/usb disk ;0 for floppy; 0x80 for hd
mov ch, 0 ; cylinder number
mov dh, 0 ; head number
mov cl, 2 ; sector number
mov bx, 0x7e00 ; load into es:bx segment :offset of buffer
int 0x13 ; disk I/O interrupt
mov ax, 0x2401
int 0x15 ; enable A20 bit
mov ax, 0x3
int 0x10 ; set vga text mode 3
cli
lgdt [gdt_pointer] ; load the gdt table
mov eax, cr0
or eax,0x1 ; set the protected mode bit on special CPU reg cr0
mov cr0, eax
jmp CODE_SEG:boot2 ; long jump to the code segment
gdt_start:
dq 0x0
gdt_code:
dw 0xFFFF
dw 0x0
db 0x0
db 10011010b
db 11001111b
db 0x0
gdt_data:
dw 0xFFFF
dw 0x0
db 0x0
db 10010010b
db 11001111b
db 0x0
gdt_end:
gdt_pointer:
dw gdt_end - gdt_start
dd gdt_start
CODE_SEG equ gdt_code - gdt_start
DATA_SEG equ gdt_data - gdt_start
bits 32
boot2:
mov ax, DATA_SEG
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
; Zero out the BSS area
cld
mov edi, _bss_start
mov ecx, _bss_len
xor eax, eax
rep stosb
mov esp,kernel_stack_top
call kernel_main
cli
hlt
section .bss
align 4
kernel_stack_bottom: equ $
resb 16384 ; 16 KB
kernel_stack_top:
kernel.c:
void kernel_main(void){
const char string[] = "01234567890123456789012345678901234567890123456789012345678901234";
volatile unsigned char* vid_mem = (unsigned char*) 0xb8000;
int j=0;
while(string[j]!='\0'){
*vid_mem++ = (unsigned char) string[j++];
*vid_mem++ = 0x09;
}
for(;;);
}
linker3.ld:
ENTRY(boot)
SECTIONS{
. = 0x7c00;
.boot1 : {
*(.boot);
}
.sig : AT(0x7dfe){
SHORT(0xaa55);
}
. = 0x7e00;
.kernel : AT(0x7e00){
*(.text);
*(.rodata*);
*(.data);
_bss_start = .;
*(.bss);
*(COMMON);
_bss_end = .;
_bss_len = _bss_end - _bss_start;
}
/DISCARD/ : {
*(.eh_frame);
*(.comment);
}
}
Commands to build this bootloader and kernel:
nasm -g -F dwarf -f elf32 boot.asm -o boot.o
i686-elf-gcc -g -O3 -m32 kernel.c -c -o kernel.o -ffreestanding -std=gnu99 \
-mno-red-zone -fno-exceptions -Wall -Wextra
i686-elf-gcc -nostdlib -Wl,--build-id=none -T linker3.ld boot.o kernel.o \
-lgcc -o kernel.elf
objcopy -O binary kernel.elf kernel.bin
To symbolically debug the 32-bit kernel with QEMU you can launch QEMU this way:
qemu-system-i386 -fda kernel.bin -S -s &
gdb kernel.elf \
-ex 'target remote localhost:1234' \
-ex 'break *kernel_main' \
-ex 'layout src' \
-ex 'continue'
This will start up your kernel.bin file in QEMU and then remotely connect the GDB debugger. The layout should show the source code and break on kernel_main.
I would like to know if a binary has been linked using Position independent executable flag during linking.
Here is one way:
main.c
#include <stdio.h>
int main(void)
{
puts(__func__);
return 0;
}
Compile and link non-PIE:
$ gcc -Wall -c main.c
$ gcc -Wall -no-pie main.o
See the program headers (my ^^^^^^^^^-annotations):
$ readelf -l a.out
Elf file type is EXEC (Executable file)
^^^^^^^^^^^^^^^^^^^^^^
Entry point 0x400400
^^^^^^^^
| Absolute entry point
There are 9 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x00000000000001f8 0x00000000000001f8 R 0x8
INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238
0x000000000000001c 0x000000000000001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
^^^^^^^^^^^^^^^^^^
| Absolute load address
0x00000000000006c8 0x00000000000006c8 R E 0x200000
LOAD 0x0000000000000e10 0x0000000000600e10 0x0000000000600e10
0x0000000000000220 0x0000000000000228 RW 0x200000
DYNAMIC 0x0000000000000e20 0x0000000000600e20 0x0000000000600e20
0x00000000000001d0 0x00000000000001d0 RW 0x8
NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254
0x0000000000000044 0x0000000000000044 R 0x4
GNU_EH_FRAME 0x000000000000058c 0x000000000040058c 0x000000000040058c
0x000000000000003c 0x000000000000003c R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x0000000000000e10 0x0000000000600e10 0x0000000000600e10
0x00000000000001f0 0x00000000000001f0 R 0x1
...
...
Compile and link PIE:
$ gcc -Wall -fPIC -c main.c
$ gcc -Wall -pie main.o
See the program headers again:
$ readelf -l a.out
Elf file type is DYN (Shared object file)
^^^^^^^^^^^^^^^^^^^^^^^^
Entry point 0x530
^^^^^
| Offset from unknown load address
There are 9 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000000040 0x0000000000000040
0x00000000000001f8 0x00000000000001f8 R 0x8
INTERP 0x0000000000000238 0x0000000000000238 0x0000000000000238
0x000000000000001c 0x000000000000001c R 0x1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000
^^^^^^^^^^^^^^^^^^
| Unknown load address
0x0000000000000830 0x0000000000000830 R E 0x200000
LOAD 0x0000000000000db8 0x0000000000200db8 0x0000000000200db8
0x0000000000000258 0x0000000000000260 RW 0x200000
DYNAMIC 0x0000000000000dc8 0x0000000000200dc8 0x0000000000200dc8
0x00000000000001f0 0x00000000000001f0 RW 0x8
NOTE 0x0000000000000254 0x0000000000000254 0x0000000000000254
0x0000000000000044 0x0000000000000044 R 0x4
GNU_EH_FRAME 0x00000000000006ec 0x00000000000006ec 0x00000000000006ec
0x000000000000003c 0x000000000000003c R 0x4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 0x10
GNU_RELRO 0x0000000000000db8 0x0000000000200db8 0x0000000000200db8
0x0000000000000248 0x0000000000000248 R 0x1
I have encountered a pretty mysterious problem after rewriting a ELF binary. I have rewritten a binary using libelf library. Basically I am just replacing some instructions in .text with same number of NOPs. This doesn't change size of any sections, as is evident by readelf output also. However there is some strange mismatch, with respect to original file, in segments to sections mapping after rewriting.
readelf -l output before rewriting:
Elf file type is EXEC (Executable file)
Entry point 0x202a0
There are 8 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
EXIDX 0x000964 0x10020964 0x10020964 0x00230 0x00230 R 0x4
LOAD 0x010000 0x00020000 0x00020000 0x20000 0x20000 R E 0x10000
LOAD 0x000000 0x10020000 0x10020000 0x00c1c 0x00c1c R 0x10000
LOAD 0x000c20 0x10030c20 0x10030c20 0x00b18 0x010b4 RW 0x10000
NOTE 0x000134 0x10020134 0x10020134 0x0003c 0x0003c R 0x4
TLS 0x000c20 0x10030c20 0x10030c20 0x00478 0x00478 R 0x8
GNU_EH_FRAME 0x000b94 0x10020b94 0x10020b94 0x00014 0x00014 R 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RWE 0x10
Section to Segment mapping:
Segment Sections...
00 .ARM.exidx
01 .init .text .fini
02 .note.NaCl.ABI.arm .note.gnu.build-id .rodata .ARM.extab .ARM.exidx
.eh_frame_hdr .eh_frame
03 .tdata .init_array .fini_array .jcr .got .data .bss
04 .note.NaCl.ABI.arm .note.gnu.build-id
05 .tdata
06 .eh_frame_hdr
07
readelf -l after rewriting:
Elf file type is EXEC (Executable file)
Entry point 0x202a0
There are 8 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
EXIDX 0x000964 0x10020964 0x10020964 0x00230 0x00230 R 0x4
LOAD 0x010000 0x00020000 0x00020000 0x20000 0x20000 R E 0x10000
LOAD 0x000000 0x10020000 0x10020000 0x00c1c 0x00c1c R 0x10000
LOAD 0x000c20 0x10030c20 0x10030c20 0x00b18 0x010b4 RW 0x10000
NOTE 0x000134 0x10020134 0x10020134 0x0003c 0x0003c R 0x4
TLS 0x000c20 0x10030c20 0x10030c20 0x00478 0x00478 R 0x8
GNU_EH_FRAME 0x000b94 0x10020b94 0x10020b94 0x00014 0x00014 R 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RWE 0x10
Section to Segment mapping:
Segment Sections...
00
01 .fini .comment .ARM.attributes .debug_aranges .debug_info
.debug_abbrev
02
03 .bss
04
05
06
07
What might be the reason behind this?
What might be the reason behind this?
The most likely reason is that whatever tool you used overwrote more than just some instructions from .text.
In particular, it appears to have overwritten your section table. You can compare the before/after output from readelf -WS to confirm this, or even readelf -W --all to see what else might have been overwritten.
I'm having trouble finding appropriate documentation for the problem I'm having generating consistent HMACs in the kernel and user space. According to Robert Love in Linux Kernel Development, the Memory Descriptors mm->start_code and mm->end_code are supposed to contain the .text segment. Finding the .text segment in a static executable is well defined in the ELF documentation and is easy to get at. So, given the following two code snippets, one would expect to get a matching HMAC:
Kernel:
__mm = get_task_mm(__task);
__retcode = ntru_crypto_hmac_init(__crypto_context);
if(__retcode != NTRU_CRYPTO_HMAC_OK)
return 1;
__retcode = ntru_crypto_hmac_update(__crypto_context, (const uint8_t*)__mm->start_code,
__mm->end_code - __mm->start_code);
if(__retcode != NTRU_CRYPTO_HMAC_OK)
return 1;
__retcode = ntru_crypto_hmac_final(__crypto_context, __hmac);
if(__retcode != NTRU_CRYPTO_HMAC_OK)
return 1;
return 0;
Userland:
for (j = 0; j < file_hdr32.e_shnum; j++)
{
if (!strcmp(".text", strIndex + section_hdr32[j]->sh_name))
{
retcode = ntru_crypto_hmac_init(__crypto_context());
if(retcode != NTRU_CRYPTO_HMAC_OK)
{
syslog(LOG_ERR, "ntru_crypto_hmac_init error: retcode = %d, TID(0x%lx)",
retcode,pthread_self());
return 0;
}
retcode = ntru_crypto_hmac_update(__crypto_context(),
filebuf + section_hdr32[j]->sh_offset, section_hdr32[j]->sh_size);
if(retcode != NTRU_CRYPTO_HMAC_OK)
{
syslog(LOG_ERR, "Internal crypto error (%d)", retcode);
return 0;
}
retcode = ntru_crypto_hmac_final(__crypto_context(), _hmac);
if(retcode != NTRU_CRYPTO_HMAC_OK)
{
syslog(LOG_ERR, "Failed to finalize HMAC, TID(0x%lx)", pthread_self());
return 0;
}
return 1;
}
}
In both cases the .text segment is exactly where it's documented to be but they never match. I've generated userland HMACs for all 17,000 executable files on the system so even if the code segment in the kernel memory descriptor were pointing to a dependency, rather than the primary executable, I still should get a match. But no dice. There's something fundamentally different between the two .text segments and I was wondering if anyone out there knew what it was so I can save some time--any clues?
There's something fundamentally different between the two ".text" segments
Your problem is that you are ignoring the difference between segments and sections.
The ELF format is an executable and linking format. Segments are used for the former, sections for the latter (and linking here means static linking, i.e. build-time). Once the binary is linked, sections can be completely discarded from it, and only segments are needed at runtime. Segments are mmaped, not sections.
Now let's look at the difference between the two.
readelf -l /bin/date
Elf file type is EXEC (Executable file)
Entry point 0x402000
There are 9 program headers, starting at offset 64
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x00000000000001f8 0x00000000000001f8 R E 8
INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x000000000000d5ac 0x000000000000d5ac R E 200000
LOAD 0x000000000000de10 0x000000000060de10 0x000000000060de10
0x0000000000000440 0x0000000000000610 RW 200000
DYNAMIC 0x000000000000de38 0x000000000060de38 0x000000000060de38
0x00000000000001a0 0x00000000000001a0 RW 8
NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254
0x0000000000000044 0x0000000000000044 R 4
GNU_EH_FRAME 0x000000000000c700 0x000000000040c700 0x000000000040c700
0x00000000000002a4 0x00000000000002a4 R 4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 8
GNU_RELRO 0x000000000000de10 0x000000000060de10 0x000000000060de10
0x00000000000001f0 0x00000000000001f0 R 1
Section to Segment mapping:
Segment Sections...
00
01 .interp
02 .interp .note.ABI-tag .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame
03 .ctors .dtors .jcr .dynamic .got .got.plt .data .bss
04 .dynamic
05 .note.ABI-tag .note.gnu.build-id
06 .eh_frame_hdr
07
08 .ctors .dtors .jcr .dynamic .got
Above you can see that multiple sections (.interp, .note.ABI-tag, ... .text, ...) all got mapped into a single PT_LOAD segment. All these sections have the same protections, and all are "covered" by a single [mm->start_core, mm->end_code) region.
Compare this to the .text section:
readelf -WS /bin/date | grep '\.text'
[13] .text PROGBITS 0000000000401900 001900 0077f8 00 AX 0 0 16
You'll note that the section is smaller and begins at a different offset.
No wonder you get different HMAC then. Try computing HMAC in user-land over segments, and you should get a match.
All other factors being equal (eg optimisation level), how does having debug symbols in an ELF or SO affect:
Load time.
Runtime memory footprint.
Runtime performance?
And what could be done to mitigate any negative effects?
EDIT
I've seen this question but I find the discussion unhelpful, as the code optimization factor has confused the issue there. Why does my code run slower with multiple threads than with a single thread when it is compiled for profiling (-pg)?
The debug symbols are located in totally different sections from the code/data sections. You can check it with objdump:
$ objdump -h a.out
a.out: file format elf64-x86-64
Sections:
Idx Name Size VMA LMA File off Algn
0 .interp 0000001c 0000000000400200 0000000000400200 00000200 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
1 .note.ABI-tag 00000020 000000000040021c 000000000040021c 0000021c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
2 .note.gnu.build-id 00000024 000000000040023c 000000000040023c 0000023c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .hash 00000018 0000000000400260 0000000000400260 00000260 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .gnu.hash 0000001c 0000000000400278 0000000000400278 00000278 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .dynsym 00000048 0000000000400298 0000000000400298 00000298 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
6 .dynstr 00000038 00000000004002e0 00000000004002e0 000002e0 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
7 .gnu.version 00000006 0000000000400318 0000000000400318 00000318 2**1
CONTENTS, ALLOC, LOAD, READONLY, DATA
8 .gnu.version_r 00000020 0000000000400320 0000000000400320 00000320 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
9 .rela.dyn 00000018 0000000000400340 0000000000400340 00000340 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
10 .rela.plt 00000018 0000000000400358 0000000000400358 00000358 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
11 .init 00000018 0000000000400370 0000000000400370 00000370 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
12 .plt 00000020 0000000000400388 0000000000400388 00000388 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
13 .text 000001c8 00000000004003b0 00000000004003b0 000003b0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
14 .fini 0000000e 0000000000400578 0000000000400578 00000578 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
15 .rodata 00000004 0000000000400588 0000000000400588 00000588 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
16 .eh_frame_hdr 00000024 000000000040058c 000000000040058c 0000058c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
17 .eh_frame 0000007c 00000000004005b0 00000000004005b0 000005b0 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
18 .ctors 00000010 0000000000600630 0000000000600630 00000630 2**3
CONTENTS, ALLOC, LOAD, DATA
19 .dtors 00000010 0000000000600640 0000000000600640 00000640 2**3
CONTENTS, ALLOC, LOAD, DATA
20 .jcr 00000008 0000000000600650 0000000000600650 00000650 2**3
CONTENTS, ALLOC, LOAD, DATA
21 .dynamic 000001a0 0000000000600658 0000000000600658 00000658 2**3
CONTENTS, ALLOC, LOAD, DATA
22 .got 00000008 00000000006007f8 00000000006007f8 000007f8 2**3
CONTENTS, ALLOC, LOAD, DATA
23 .got.plt 00000020 0000000000600800 0000000000600800 00000800 2**3
CONTENTS, ALLOC, LOAD, DATA
24 .data 00000010 0000000000600820 0000000000600820 00000820 2**3
CONTENTS, ALLOC, LOAD, DATA
25 .bss 00000010 0000000000600830 0000000000600830 00000830 2**3
ALLOC
26 .comment 00000039 0000000000000000 0000000000000000 00000830 2**0
CONTENTS, READONLY
27 .debug_aranges 00000030 0000000000000000 0000000000000000 00000869 2**0
CONTENTS, READONLY, DEBUGGING
28 .debug_pubnames 0000001b 0000000000000000 0000000000000000 00000899 2**0
CONTENTS, READONLY, DEBUGGING
29 .debug_info 00000055 0000000000000000 0000000000000000 000008b4 2**0
CONTENTS, READONLY, DEBUGGING
30 .debug_abbrev 00000034 0000000000000000 0000000000000000 00000909 2**0
CONTENTS, READONLY, DEBUGGING
31 .debug_line 0000003b 0000000000000000 0000000000000000 0000093d 2**0
CONTENTS, READONLY, DEBUGGING
32 .debug_str 00000026 0000000000000000 0000000000000000 00000978 2**0
CONTENTS, READONLY, DEBUGGING
33 .debug_loc 0000004c 0000000000000000 0000000000000000 0000099e 2**0
CONTENTS, READONLY, DEBUGGING
You can see the extra sections (27 through 33). These sections won't be loaded at runtime, so there won't be any performance penalty. Using gdb, you can also examine them at runtime:
$ gdb ./a.out
(gdb) break main
(gdb) run
(gdb) info files
// blah blah ....
Local exec file:
`/home/kghost/a.out', file type elf64-x86-64.
Entry point: 0x4003b0
0x0000000000400200 - 0x000000000040021c is .interp
0x000000000040021c - 0x000000000040023c is .note.ABI-tag
0x000000000040023c - 0x0000000000400260 is .note.gnu.build-id
0x0000000000400260 - 0x0000000000400278 is .hash
0x0000000000400278 - 0x0000000000400294 is .gnu.hash
0x0000000000400298 - 0x00000000004002e0 is .dynsym
0x00000000004002e0 - 0x0000000000400318 is .dynstr
0x0000000000400318 - 0x000000000040031e is .gnu.version
0x0000000000400320 - 0x0000000000400340 is .gnu.version_r
0x0000000000400340 - 0x0000000000400358 is .rela.dyn
0x0000000000400358 - 0x0000000000400370 is .rela.plt
0x0000000000400370 - 0x0000000000400388 is .init
0x0000000000400388 - 0x00000000004003a8 is .plt
0x00000000004003b0 - 0x0000000000400578 is .text
0x0000000000400578 - 0x0000000000400586 is .fini
0x0000000000400588 - 0x000000000040058c is .rodata
0x000000000040058c - 0x00000000004005b0 is .eh_frame_hdr
0x00000000004005b0 - 0x000000000040062c is .eh_frame
0x0000000000600630 - 0x0000000000600640 is .ctors
0x0000000000600640 - 0x0000000000600650 is .dtors
0x0000000000600650 - 0x0000000000600658 is .jcr
0x0000000000600658 - 0x00000000006007f8 is .dynamic
0x00000000006007f8 - 0x0000000000600800 is .got
0x0000000000600800 - 0x0000000000600820 is .got.plt
0x0000000000600820 - 0x0000000000600830 is .data
0x0000000000600830 - 0x0000000000600840 is .bss
// blah blah ....
So the only penalty is that you need extra disk space to store this information. You can also use strip to remove the debug information:
$ strip a.out
Use objdump to check it again, you'll see the difference.
EDIT:
Instead of looking at sections, actually the loader loads elf file according to its Program Header, which can be seen by objdump -p (the following example uses using a different elf binary):
$ objdump -p /bin/cat
/bin/cat: file format elf64-x86-64
Program Header:
PHDR off 0x0000000000000040 vaddr 0x0000000000000040 paddr 0x0000000000000040 align 2**3
filesz 0x00000000000001f8 memsz 0x00000000000001f8 flags r-x
INTERP off 0x0000000000000238 vaddr 0x0000000000000238 paddr 0x0000000000000238 align 2**0
filesz 0x000000000000001c memsz 0x000000000000001c flags r--
LOAD off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**21
filesz 0x00000000000078bc memsz 0x00000000000078bc flags r-x
LOAD off 0x0000000000007c28 vaddr 0x0000000000207c28 paddr 0x0000000000207c28 align 2**21
filesz 0x0000000000000678 memsz 0x0000000000000818 flags rw-
DYNAMIC off 0x0000000000007dd8 vaddr 0x0000000000207dd8 paddr 0x0000000000207dd8 align 2**3
filesz 0x00000000000001e0 memsz 0x00000000000001e0 flags rw-
NOTE off 0x0000000000000254 vaddr 0x0000000000000254 paddr 0x0000000000000254 align 2**2
filesz 0x0000000000000044 memsz 0x0000000000000044 flags r--
EH_FRAME off 0x0000000000006980 vaddr 0x0000000000006980 paddr 0x0000000000006980 align 2**2
filesz 0x0000000000000274 memsz 0x0000000000000274 flags r--
STACK off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**4
filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
RELRO off 0x0000000000007c28 vaddr 0x0000000000207c28 paddr 0x0000000000207c28 align 2**0
filesz 0x00000000000003d8 memsz 0x00000000000003d8 flags r--
The program headers tell which segment will be loaded with what rwx flags; multiple sections with the same flags will be merged to a single segment.
BTW:
The loader doesn't care about sections when loading elf file, but it will look at several symbol-related sections to resolve symbols when needed.
You might want to look at Why does my code run slower with multiple threads than with a single thread when it is compiled for profiling (-pg)? for a quick explanations of how the debug symbols could affect optimization.
To answer your 3 questions:
Load time will be increased when the debug symbols are present over when not present
The on-disk footprint will be larger
If you compiled with zero optimization then you really lose nothing. If you set optimization, then the optimized code will be less optimized because of the debug symbols.