Subject: PPC Assembly Language - Linux Loadble Kernel Module
Detail: How access local TOC area (r2) when called from kernel in syscall table hook?
I have written a loadable kernel module for Linux that uses syscall table hooking to intercept system calls and log information about them before passing the call on to the original handler. This is part of a security product. My module runs well and is in production code running on a large variety of Linux kernel versions and distributions with both 32 and 64 bit kernels all running on x86 hardware.
I am trying to port this code to run on Linux for PPC processors and ran into a few problems. Using the Linux kernel source, it is easy enough to see how the system call table is implemented differently on PPC. I can replace entries in the table with function addresses from my own compiled handlers, no problem.
But, here's the issue I'm having trouble with. The PPC ABI uses this thing called a Table Of Contents (TOC) address which is stored in the CPU's R2 register and expects to address a module's global and local data by using an offset from the address (TOC address) contained in that register. This works fine in normal cases where a function call is made because the compiler knows to load the module's TOC address into the register before making the call (or its already there becasue normally your functions are called by your own code).
However, when I place the address of my own function (from my loaded kernel module at runtime) into the system call table, the kernel calls my handler with an R2 value that is not the one my compiled C code expects, so my code gets control without being able to access its data.
Does anybody know of any example code out there showing how to handle this situation? I cannot recompile the kernel. This should be a straightforward case of runtime syscall table hooking, but I have yet to figure it out, or find any examples specific to PPC.
Ideas include:
Hand coding an assembly language stub that saves the R2 value, loads the register with my local TOC address, executes my code, then restores the old value before calling the original handler. I don't have the depth of PPC assembly experience to do this, nor am I sure it would work.
Some magic gcc option that will generate my code without using TOC. There is a documented gcc option "-mno-toc" that doesn't work on my PPC6 Linux. It looks like it may only be an option for system V.4 and embedded PowerPC.
Any help is greatly appreciated !
Thanks!
Linux has a generic syscall audit infrastructure which works on powerpc and you can access from user space. Have you considered using that rather than writing a kernel module?
You need a stub to load r2. There are examples in the kernel source.
Related
I am working on porting an application that uses LLVM MCJIT (RPCS3) to macOS. I have run into an issue when calling existing functions from the JIT: calling such functions results in a segfault, as an offset of 0x100000000 is added to the address of each function. llvm::ExecutionEngine::addGlobalMapping is used to add the mappings for these existing functions. To confirm this, I tried replacing the address of one of these functions with 0x1234. When calling the function from the JIT, the application now segfaults at 0x100001234 (not 0x1234 as expected). This behaved as expected (segfault at 0x1234) on Linux. Where could this offset be coming from? Is it possible to manually specify the image base in MCJIT? I am not too familiar with LLVM MCJIT internals.
I hook execve in kernel mode(change system_call_table entry __NR_execve to my function). I want to check the ELF's assembly code. If it harmful, I'll return directly without executing it.
I am writing a linux module. In Linux kernel mode, I want to use objdump to disassembly the ELF file.
I want to go user mode to execute objdump, and go back to kernel mode. Is this possible?
Thank you.
Maybe you can split your project into two parts: kernel module and user-space application. So you can hook execve() in kernel, then tell your application about hook triggered, then do disassembling and checking in your application, send computed result back to kernel module, and then either continue or break execve() execution.
If you still want to run objdump from kernel -- check out call_usermodhelper().
See also this related question.
I wanted to build my own custom kernel with a different syscall table. (same syscalls but in different position/numbers)
I was working on kernel 3.2.29.
Changing the kernel was quite easy:
1) changing the syscall position in arch/x86/kernel/syscall_table_32.S
2) changing the syscall macro number in arch/x86/include/asm/unistd_32.h
3) compiling and installing the new kernel
I switched the syscalls around: sys_open took the place and number of sys_read, and vice versa.
I figured that if I compile glibc with the modified kernel headers, I could have a running system, but unfortunately, it wasn't enough and my system won't boot.
Am I missing something? What else do I need to do in order to have a running system?
The steps I have taken are:
1) building and installing the kernel as described in my question
2) extracting the new kernel headers using make headers_install INSTALL_HDR_PATH=[path]
3) building glibc with the parameter --with-headers=[path/include]
4) I used a live cd to access the file system externally in order to install the new glibc, using the make install install_root=[the original file system] (so the system won't break during the install)
I hope that the new glibc was built properly, but I am not sure.
After that, when booting the system, the boot stops in the (initrafms) shell screen:
I guess I need to rebuild the initrd, but how do I compile it according to the new syscall table?
You will have to rebuild everything. Even if all your binaries are dynamically linked, it is possible that the old syscalls were inlined into the binary because many of the C functions are just return syscall(__NR_somecall,...).
You could do this manually, but it could be difficult to keep the toolchains straight unless you use a cross compilable toolchain like buildroot, aboriginal or similar. Pick whichever best suits you (I prefer Rob Landley's aboriginal - http://landley.net/aboriginal/ )
Then to make your initrd just expand the old one using {z,bz,xz}cat oldinit.rd |cpio -id; rm oldinit.rd. Replace the old kernel modules, libs and binaries with the new and cpio and compress it back (cpio needs the -H newc option) ... or now you can rebuild your kernel and point the initramfs to that directory, but wouldn't recommend that if your initrd may need changed frequently such as if for instance you were testing out a whole new syscall structure and having to debug a lot.
Scrambling the system call numbers is really going to hurt. You'll at least need to rebuild all of the statically linked binaries on your system and your initrd (if you use one).
You haven't said at what point the boot fails, but even if the kernel comes up it is likely that the critical programs contained in the initrd compressed ramdisk would fail because they have the original syscall numbers hard-coded. You will need to rebuild and repackage those as well.
You might consider first replacing init with a static hello-world type of program to verify that your kernel can support a userspace at all; then look into the details of making all the complexity of a modern linux userspace match.
you had to learn to reading the messsage of pansic dump and show us what kenrel panic. Without this information, people can hardly help you or provide you useful suggestion.
I encounter an access violation when calling a DLL inside LabVIEW. Let's call the DLL "extcode.dll". I don't have its code, it comes from an external manufacturer.
Running it in Windbg, it stopped with the message:
(724.1200): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
ntdll!RtlNewSecurityObjectWithMultipleInheritance+0x12a:
And call stack is:
ntdll!RtlNewSecurityObjectWithMultipleInheritance+0x12a
ntdll!MD5Final+0xedfc
ntdll!RtlFindClearBitsAndSet+0xdf4
ntdll!RtlFindClearBitsAndSet+0x3a8
ntdll!RtlFindClearBitsAndSet+0x4b9
ntdll!RtlCreateProcessParametersEx+0x829
ntdll!LdrLoadDll+0x9e
KERNELBASE!LoadLibraryExW+0x19c
KERNELBASE!LoadLibraryExA+0x51
LabVIEW!ChangeVINameWrapper+0x36f5
LabVIEW!ChangeVINameWrapper+0x3970
LabVIEW!ExtFuncDynLibWrapper+0x211
Note that dependencies of extcode.dll are loaded before access violation.
The situation is random, but when it happens all subsequent tries lead to it.
The code is a simple LabVIEW function calling a function in the DLL, and prototype is super simple (int function(void)) so it cannot be an misconfiguration of the call parameters, nor pointer arithmetics. I checked every combination of calling conventions and error checking levels.
The DLL runs perfectly fine when called in other environments (.NET and C).
I found that RtlFindClearBitsAndSet is related to bit array manipulations
What does it make you think about? Do you think it is a problem in extcode.dll, LabVIEW, or Windows?
PS: I use LabVIEW 2010 64 bit, on Windows 7 64 bit (and extcode.dll is 64 bit). I didn't manage to reproduce it on 32 bit system.
11/18 EDIT
I ended up making a standalone exe that wraps the DLL; LabVIEW communicates with it through pipes. It works perfectly, but I stil don't understand why loading a DLL into LabVIEW can crash.
If it works ok when called from C, you can quit working with Windbg because the DLL is probably ok. Something is wrong with how the DLL is being called, and once the DLL overwrites some of LabView's memory it is all over, even though it might take 1000 iterations before something actually goes kablooey.
First check your calling conventions, C or StdCall. C calling convention is the default and StdCall is almost certainly what you want. (Check the DLL header file.) LabView 2009 apparently did some auto-checking and fixing of calling conventions, but the switch to LLVM in LV 2010 has made this impossible; now it just tanks.
If it still tanks after changing this, check your calling arguments again. what you are passing, scalars or pointer data? You cannot access memory allocated by the DLL from LabView without doing some sneaky things, although you can allocate memory (i.e. byte array) in LabView and pass a pointer to it to the DLL for it to modify.
Also, if you are getting a pointer (such as a refnum) from an earlier call to DLL and returning it, check your pointer size. LabView's Call Library function now has a "pointer size integer" type, which generates the appropriately-sized type depending on whether it is invoked in 32-bit or 64-bit LabView. (It is always 64 bits on the wire, because that has to be defined at compile time.) The fact that your DLL works in 32 suggests this is a possibility.
Also keep in mind that C structs are often aligned by the (C) compiler. If you are passing a pointer to a struct made of a Uint8 and an UInt16, the C compiler will allocate 32 bits (or maybe even 64 bits) for this. You'll have to pad your struct (cluster) in LabView to make it match, or write a wrapper DLL to assemble the struct.
-Rob
An access violation (0xc0000005) will also be reported if DEP (Data Execution Prevention) is enabled on your machine and disapproves of something your binary (EXE or DLL) is trying to do. DEP is usually off by default on Windows XP, but active on Windows Vista / Windows 7.
DEP is a hardware-supported security measure designed to prevent malicious code executing some bytes that previously were considered "just some data"; I've had a few run-ins with it, all of which required re-compiling the offending binaries with a recent version of Microsoft Visual Studio; this allows to you set a flag which defines whether or not your binary supports DEP.
Some useful resources:
I found this MSDN blog entry
very helpful for gaining an
understanding of what DEP is and does
You might also want to consult this
technet article on how to turn
DEP on and off on Windows XP.
This is hard to diagnose remotely, but here are a couple of ideas.
The fact that your function takes no arguments means that either the function is truly trivial, or there is some stored state in the dll that takes into account previous function calls. Maybe the crash in this function is only an indicator, and you have a problem with a previous function call? Is there an initilization routine you're not calling?
If you only have problem when using 64 bit labview, my first guess would be that there's a problem with the 64 bit version of the dll, but if you are sure you don't have any problem with the exact same calls when using the dll in other environments, I'm stumped. One possibility is that you are using the wrong calling convention (stdcall vs. cdecl) in labview.
Have you tried importing the dll and header using the labview import wizard? This might help avoid silly mistakes with the prototypes.
One other thing to try: right click on the DLL call, choose configure and make sure you're running in the UI thread instead of any thread. Sometimes this helps.
When working with git and cygwin under NTFS, i found that sometimes the executable bit is not set (or un-set during checkout or some file operations) - inside cygwin cd to the folder and do
chmod a+rwx *.dll
and check if it changes a thing (and check if you want it this way!). I found this question while searching for LoadLibrary() failing with GetLastError() returning 5 (not "0xc0000005" btw) and solved the issue with this chmod call.
I just want to ask, I know that standart system calls in Linux are done by int instruction pointing into Interrupt Vector Table. I assume this is similiar on Windows. But, how do you call some higher-level specific system routines? Such as how do you tell Windows to create a window? I know this is handled by the code in the dll, but what actually happend at assembler-instruction level? Does the routine in dll calls software interrupt by int instruction, or is there any different approach to handle this? Thanks.
Making a Win32 call to create a window is not really related to an interrupt. The client application is already linked with the .dll that provides the call which exposes the address for the linker to use. Since you are asking about the difference in calling mechanism, I'm limiting the discussion here to those Win32 calls that are available to any application as opposed to kernel-level calls or device drivers. At an assembly language level, it would be the same as any other function call since most Win32 calls are user-level calls which internally make the needed kernel calls. The linker provides the address of the Win32 function as the target for some sort of branching instruction, the specifics would depend on the compiler.
[Edit]
It looks like you are right about the interrupts and the int. vector table. CodeGuru has a good article with the OS details on how NT kernel calls work. Link:
http://www.codeguru.com/cpp/w-p/system/devicedriverdevelopment/article.php/c8035
Windows doesn't let you call system calls directly because system call numbers can change between builds, if a new system call gets added the others can shift forward or if a system call gets removed others can shift backwards. So to maintain backwards compatability you call the win32 or native functions in DLLs.
Now there are 2 groups of system calls, those serviced by the kernel (ntoskrnl) and by the win32 kernel layer (win32k).
Kernel system call stubs are easily accessible from ntdll.dll, while win32k ones are not exported, they're private within user32.dll. Those stubs contain the system call number and the actual system call instruction which does the job.
So if you wanted to create a window, you'd call CreateWindow in user32.dll, then the extended version CreateWindowEx gets called for backwards compatability, that calls the private system call stub NtUserCreateWindowEx, which invokes code within the win32k window manager.