I would like to debug a user defined function (called prepareTheOutputRecord implemented in C/C++ that is a part of user defined function in postgres. Here's how I achieve this with gdb:
The function prepareTheOutputRecord resides in libMyExtenstion.so file in the lib directory of postgresql server
I start the psql shell, retrieve the pid of the process
postgres=# SELECT pg_backend_pid();
pg_backend_pid
- - - - - - - - - - - - - -
4120
(1 row)
Run the gdb with the attached pid:
gdb -p 4120
Search now the .so file, how the function is exactly called:
nm -as libMyExtenstion.so | grep prepareTheOputRecord
00000000002633fe t _ZN6libafd6LIBAFD22prepareTheOutputRecordEP20FunctionCallInfoData
Set a breakpoint in gdb and run the program:
(gdb) b _ZN6libafd6LIBAFD22prepareTheOutputRecordEP20FunctionCallInfoData
Function "_ZN6libafd6LIBAFD22prepareTheOutputRecordEP20FunctionCallInfoData" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (_ZN6libafd6LIBAFD22prepareTheOutputRecordEP20FunctionCallInfoData) pending.
(gdb) c
Execute the SQL in psql. At a certain point the breakpoint is hit in gdb:
Breakpoint 1, prepareTheOutputRecord (this=0x1116410, fcinfo=0x7fff3a41e150)
at ../Dir/file.cpp:1736
1736 funcctx = SRF_PERCALL_SETUP();
(gdb)
Continue debugging the code.
I want to do exactly the same in kdbg. For that I loaded the postgres executable, attached to the process, loaded the cpp file, set the breakpoint with the mouse at the function, continued the execution of the postgres process, but the breakpoint was never hit :( I repeated the same with .so file (instead of postgres executable) without any success. I even tried to set the breakpoint to _ZN6libafd6LIBAFD22prepareTheOutputRecordEP20FunctionCallInfoData (without the mouse clicks) but the program does not stop in kdbg :(
I believe the problem was that the kdb was not run as the root (or postgres). Due to wrong permissions the symbols were not loaded and therefore no breakpoint was shown (nor allowed to be placed at a function)
Related
I’m new for spike and RISC V. I’m trying to do some dynamic instruction trace with spike. These instructions are from a sample.c file. I have tried the following commands:
$ riscv64-unknown-elf-gcc simple.c -g -o simple.out
$ riscv64-unknown-elf-objdump -d --line-numbers -S simple.out
But these commands display the assembled instructions in an out file, which is not I want. I need to trace the dynamic executed instruction in runtime. I find only two relative commands in spike host option:
-g - track histogram of PCs
-l - generate a log of execution
I’m not sure if the result is what I expected as above.
Does anyone have an idea how to do the dynamic instruction trace in spike?
Thanks a lot!
Yes, you can call spike with -l to get a trace of all executed instructions.
Example:
$ spike -l --isa=RV64gc ~/riscv/pk/riscv64-unknown-elf/bin/pk ./hello 2> ins.log
Note that this trace also contains all instructions executed by the proxy-kernel - rather than just the trace of your user program.
The trace can still be useful, e.g. you can search for the start address of your code (i.e. look it up in the objdump output) and consume the trace from there.
Also, when your program invokes a syscall you see something like this in the trace:
[.. inside your program ..]
core 0: 0x0000000000010088 (0x00000073) ecall
core 0: exception trap_user_ecall, epc 0x0000000000010088
core 0: 0x0000000080001938 (0x14011173) csrrw sp, sscratch, sp
[.. inside the pk ..]
sret
[.. inside your program ..]
That means you can skip to the sycall instruction (that are executed in the pk) by searching for the next sret.
Alternatively, you can call spike with -d to enter debug mode. Then you can set a breakpoint on the first instruction of interest in your program (until pc 0 YOURADDRESS - look up the address in the objdump output) and single step from there (by hitting return multiple times). See also the help screen by entering h at the spike prompt.
I am writing a bpf filter to prevent certain netlink messages. I am trying to debug the bpf code. Is there any debug tool that could help me?
I was initially thinking of using nlmon to capture netlink messages:
From https://jvns.ca/blog/2017/09/03/debugging-netlink-requests/
# create the network interface
sudo ip link add nlmon0 type nlmon
sudo ip link set dev nlmon0 up
sudo tcpdump -i nlmon0 -w netlink.pcap # capture your packets
Then use ./bpf_dbg (
https://github.com/cloudflare/bpftools/blob/master/linux_tools/bpf_dbg.c)
1) ./bpf_dbg to enter the shell (shell cmds denoted with '>'):
2) > load bpf 6,40 0 0 12,21 0 3 20... (this is the bpf code I intend to debug)
3) > load pcap netlink.pcap
4) > run /disassemble/dump/quit (self-explanatory)
5) > breakpoint 2 (sets bp at loaded BPF insns 2, do run then;
multiple bps can be set, of course, a call to breakpoint
w/o args shows currently loaded bps, breakpoint reset for
resetting all breakpoints)
6) > select 3 (run etc will start from the 3rd packet in the pcap)
7) > step [-, +] (performs single stepping through the BPF)
Did anyone try this before?
Also, I was not able to make nlmon module to load on my linux kernel(Is there a doc for this?)
I am running kernel version Linux version 4.10.0-40-generic
The nlmon module seems to be present in the kernel source:
https://elixir.free-electrons.com/linux/v4.10/source/drivers/net/nlmon.c#L41
But, when I search inside, /lib/modules/ for nlmon.ko I dont find anything.
instance-1:/lib/modules$ find . | grep -i nlmon
instance-1:/lib/modules$
I have a virtual Windows 7 x64 machine on a Windows 10 host, and I kernel debug it with windbg 10.0.10586.567. I'm running my own application on it, which I have full source and private symbols for. Whenever I break in and ask for stack traces of the app's threads, the backtrace always stops when one of my application's binaries are "hit."
So for instance, if I break in, switch to the process, and request a stacktrace with !thread [thread address] 1f, I get something like this (note the "early" zero return address at the last line):
fffff880`0534e870 fffff800`026d6992 nt!KiSwapContext+0x7a
fffff880`0534e9b0 fffff800`026d81a2 nt!KiCommitThreadWait+0x1d2
fffff880`0534ea40 fffff800`029c7a2e nt!KeDelayExecutionThread+0x186
fffff880`0534eab0 fffff800`026d08d3 nt!NtDelayExecution+0x59
fffff880`0534eae0 00000000`76e7165a nt!KiSystemServiceCopyEnd+0x13 (TrapFrame # fffff880`0534eae0)
00000000`00276708 000007fe`fcf91203 ntdll!NtDelayExecution+0xa
00000000`00276710 00000001`410e7dd9 KERNELBASE!SleepEx+0xab
00000000`002767b0 00000000`00000000 MyApp!MainMessageLoop+0x4b1 [d:\whatever\path\myapplication.cpp # 3024]
This looks very similar to when you you are missing a binary while debugging a user-mode dump (lack of unwind data) of an x64 process, except in that case the stack trace usually does not stop "this sudden", rather it goes astray at that point, and shows bogus values.
Some extra info/things I tried:
I have the correct symbol paths set up (both the Microsoft symbol server, and a local folder on the host with matching PDBs, even though the latter is not needed for just the stack trace)
I have a binary path set up (.exepath) containing matching binaries on the host (I've made absolutely sure of this; copied the binaries directly from the guest to the host machine)
If I put a breakpoint in one of the app's exported DLL functions, then when the debugger breaks in, I get a one-liner stack trace like this: 0000000000274b40 0000000000000000 MyAppDLL!SomeExportedFunction+0x32 [d:\whatever\path\myapplicationDLL.cpp # 232]
I've tried virtually every combination of commands to get a stacktrace (.process /i, .process /r /p, !process -1 7, .reloads, .reload /users, .reload /f MyApp.exe, !thread [address] 1f, etc.) with no success
Tried with an older version of windbg (6.11.0001.404) as well, same result
Also tried on Windows 8.1 as a guest with the very same binaries, same result
!sym noisy output (irrelevant lines omitted):
0: kd>.process /i [address]
0: kd>g
0: kd>.reload /user
0: kd> !process -1 2
0: kd> !thread [address] 1f
[...]
DBGHELP: d:\symbolcache\MyApp.pdb\76931C5A6C284779AD2F916CA324617E1\MyApp.pdb already cached
DBGHELP: MyApp - private symbols & lines
[...]
lmvm MyApp output:
[...]
Loaded symbol image file: MyApp.exe
Image path: C:\MyApp\MyApp.exe
[...]
Any ideas?
I accidentally stumbled into a linker switch that solves this problem: /DEBUGTYPE with the PDATA argument. If you link your binaries with this switch, unwind information will be copied into your PDBs.
I recompiled/relinked the application in question with /DEBUGTYPE:CV,PDATA (/DEBUGTYPE:CV is the default if /DEBUG is specified, see the documentation), now everything works like a charm, I always get full call stacks.
One strange aspect of this: windbg happily uses unwind data found in the PDBs, but ignores the very same data in the mapped binaries (both on the host machine).
This is not a perfect solution to the problem (or any solution at all, one might say), but I'm providing this provisional answer with a workaround.
You should be able to get the information you want, albeit not so well-formatted using something like dps #rsp L10.
In x86-64 you don't have a parallel of the x86 ebp-chain, but the return addresses are still on the stack. Those will give you the functions in the stack, and the values between them will be the arguments passed to the functions (and saved registers on the stack, etc.). A random example from Google (as I'm not on my Windows machine right now):
0:017> dps #rsp
00000000`1bb0fbb8 00000000`00000020
00000000`1bb0fbc0 00000000`00000000
00000000`1bb0fbc8 00000000`008bc6c6 Dolphin!ReadDataFromFifoOnCPU+0xb6 [d:\sources\comex\source\core\videocommon\fifo.cpp # 245]
00000000`1bb0fbd0 00000000`1ba0ffeb
00000000`1bb0fbd8 00000000`00000020
00000000`1bb0fbe0 00000000`00000020
00000000`1bb0fbe8 00000000`00000800
00000000`1bb0fbf0 00000000`1ba0ffeb
00000000`1bb0fbf8 00000000`008c2ff5 Dolphin!InterpretDisplayListPreprocess+0x45 [d:\sources\comex\source\core\videocommon\opcodedecoding.cpp # 87]
00000000`1bb0fc00 00000000`00000000
00000000`1bb0fc08 00000000`008bc041 Dolphin!RunGpu+0x81 [d:\sources\comex\source\core\videocommon\fifo.cpp # 389]
00000000`1bb0fc10 00000000`8064cbc0
00000000`1bb0fc18 00000000`1bb0fcc0
00000000`1bb0fc20 00000000`00000000
00000000`1bb0fc28 00000000`008c2dda Dolphin!OpcodeDecoder_Preprocess+0x14a [d:\sources\comex\source\core\videocommon\opcodedecoding.cpp # 326]
00000000`1bb0fc30 00000000`8064cbe0
Given that you have symbols, the return addresses are easily distinguishable.
The unwind data is lazy loaded for user mode modules, so it's not going to be mapped unless someone needs it. Unfortunately the kernel debugger doesn't force the information to be present for user images, so sometimes you get this behavior. You can see if the data is mapped or not by dumping the PE header (!dh) and checking the state of the Exception Directory (!pte imagename+offset).
Given that you own the app, try forcing the information to be resident by doing a stack walk NOP somewhere in your app:
PVOID stack[2];
(VOID)CaptureStackBackTrace(0, 2, (PVOID*)&stack, NULL);
That doesn't guarantee the entire directory will be present, but usually good enough.
I want to debug a program in kernel mode, and I want to break on the entry point of the program like ollydbg. But I can't break it with bp because the program is not start and the symbol can't be loaded. I have found some way to do it but I think it's not so good.
1.Break on the CreateProcess function in kernel. But I don't know which function exactly should I break and I think there is a long way between CreateProcess and the entry point of the program.
2.Change the entry point of the program with cc. But it needs other tools and I should change the code where the byte changed back. I think it is annoying.
3.With the help of ollydbg. Debugging the program with ollydbg in a virtual machine which is debugged with windbg. I don't think that it is a good idea.
4.Use sxe ld. It can be found on Listing 3.29 in <<Advanced Windows Debugging>>. I have tried it but I found that it only works on the first time. And I don't know what exactly should I do after the break.
5.Break on the entry function with bu. But I don't know what exactly I should do either. For example, how to load the symbol?
6.Use .create. I don't know whether it is properly or not to do what I said.
I think that it is a common use to break on the entry point of a program when debug in kernel mode with windbg , and I think that there must be a good way to do that with the powerful windbg. What's the best way to do it?
By the way, I want to debug a program in kernel mode because I want to get the token vaule of the program. I found that the windbg can identify the token with !token in user mode, but I don't know how to get the value of token in user mode. It seems that I can only get the value of token in the kernel mode, right or wrong?
you can run any exe in the target via ntsd -d to debug it from the kernel mode debugger running in the host
assuming you are running a virtual machine mytarget inside myhost
install windbg in myhost
set symbol path for myhost viz srv*x:\xxxx*http:\xxxxxxxxxxxx
create a kernel connection in the host (choose the best shown below is a serial connnection)
X:\xxxx\windbg.exe -k com:pipe,port=\\.\pipe\debugPipe,resets=0,reconnect
install windbg in mytarget
open a shared folder z:\ pointing to the symbolcache folder in myhost
set symbolpath in mytarget pointing to the shared folder
run ntsd -d calc.exe
kd will break on $exentry of calc.exe with Input Prompt
as long as Input prompt is shown you are using kd like a native usermode debugger
so if you set a bp calc!Winmain and issue g kd will break on calc.exe winmain
to get to kd session use .breakin
messy stuff but will work well once you get accustomed (ie memorizing the docs)
a sample run
kd> g <-------------- kd session running in myhost
CommandLine: calc.exe
Symbol search path is: srv*z:\
*http://msdl.microsoft.com/download/symbols
ntdll!DbgBreakPoint:
7c90120e cc int 3
.sympath
NOTE: The symbol path for this ntsd is relative to where
ntsd.exe is running, not where kd.exe is running.
Symbol search path is: srv*z:\
*http://msdl.microsoft.com/download/symbols
Expanded Symbol search path is: srv*z:\
*http://msdl.microsoft.com/download/symbols
.reload /f calc.exe
lm m calc
start end module name
01000000 0101f000 calc (pdb symbols) z:\calc.pdb\3B7D84101\calc.pdb
0:000> version <--------------------usermode session in kd via ntsd -d
version
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Live user mode: <Local>
command line: 'ntsd -d calc.exe' Debugger Process 0x3F8
? $exentry;? calc!WinmainCrtstartup
Evaluate expression: 16852085 = 01012475
Evaluate expression: 16852085 = 01012475
as to your original request i am not sure what token you are interested to find
if getting the EPROCESS->Token of your exe is the only requirement you dont have to run any kd session
you can get the token of all running process in myhost with a local kernel debugging session (either using kd -kl or by using livekd from sysinternals)
here is a simple script which fetches the sid of all running process employing the above technique
:\>cat sid.txt
!for_each_process "r $t0 =(##c++(((nt!_eprocess *) ##Process )->Token.Object)) &
##(~7); r $t1 = ##c++(((nt!_token *) #$t0 )->UserAndGroups->Sid);!sid #$t1 1; ?
? (char *)((nt!_eprocess *) ##Process )->ImageFileName "
:\>kd -kl -c "$$>a< sid.txt;q"
result
WARNING: Local kernel debugging requires booting with kernel
debugging support (/debug or bcdedit -debug on) to work optimally.
lkd> kd: Reading initial command '$$>a< sid.txt;q'
SID is: S-1-5-18 (Well Known Group: NT AUTHORITY\SYSTEM)
char * 0x8ac729a4
"System"
SID is: S-1-5-18 (Well Known Group: NT AUTHORITY\SYSTEM)
char * 0x8a35729c
"smss.exe"
SID is: S-1-5-20 (Well Known Group: NT AUTHORITY\NETWORK SERVICE)
char * 0x8a3619ac
"svchost.exe"
SID is: S-1-5-19 (Well Known Group: NT AUTHORITY\LOCAL SERVICE)
char * 0x8a36ef14
"svchost.exe"
SID is: S-1-5-21-602162358-1801674531-1417001333-1003 (User: XXXXXX\Admin)
char * 0x8a261b64
"explorer.exe"
Use the method described in the Windbg help file for debugging WinLogon. Substitute your user mode app for WinLogon:
Windbg | Help | Contents | Windows Debugging | Debugging Techniques | Specialized Debugging Techniques | Debugging WinLogon
IFEO will start your user mode app and attach ntsd.exe. From ntsd.exe, you can set a break point on image entry with bu $exentry then g to continue.
At any point that ntsd.exe is broken into your user mode process, you can issue .breakin command to switch to kernel mode debugging.
In the book "Rootkit Arsenal" page 84 (Chapter 3) mentions:
..., we can view the contents of the
target machine's descriptor registers
using the command with the 0x100 mask:
kd> rM 0x100
and a paragraph below:
Note that the same task can be
accomplished by specifying the GDTR
components explicitly: kd> r gdtr ....
I run Windbg on my Win XP (inside VMWare) and choose the Kernel Debug -> Local.
My problem is in case of first command, windbg errors with:
lkd> rM 0x100
^ Operation not supported in current debug session 'rM 0x100'
and in the second command:
lkd> r gdtr
^ Bad register error in 'r gdtr'
Can anyone guide me ?
Right, you can't look at registers in a local kernel debug session. LiveKD works and you can also get the address indirectly through the PCR (!pcr).
-scott
I think I've found the solution:
Use two computers for kernel debugging instead of Local Kernel Debug.
(I used VMWare and am debugging through the COM port/named pipe)
I am thinking why this facility/feature (Local Kernel Debugging) is there if it's not complete ?