Does WinDBG provide a way of displaying type information (dt?) for a plain heap address?
I am looking at a scenario where paged heap and user mode stack traces are not available. While analysing the heap I was tracking down excessive heap usage and I found various objects in the heap that were referencing a certain memory address.
Mainly
!heap -srch [UserPtr] +0x30
returned objects like this:
0:004> !heap -srch 00000224ceade3a0+30
_HEAP # 224ceaa0000
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
00000224ceae3ac0 045c 0000 [00] 00000224ceae3ad0 045ac - (busy)
_HEAP # 224ceaa0000
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
00000224ceae8080 0346 045c [00] 00000224ceae8090 03454 - (busy)
Now due to missing user mode stack traces !heap -p -a [UserPtr] returns nothing.
So here I am, I know that actually 0x00000224ceae3ad0 and 0x00000224ceae8090 are the culprits. Well actually [UserPtr]+0x30 (48 bytes) for a reason I yet have to find out but I am stuck here. (I created a small sample where I was able to verify the address, so at least in the sample I am quite sure that the address are correct)
Is there something useful I can do at this point to gain more information about those address beside !address [Addr] which does not really help much.
Since I know size and address I could wade around in the heap and see what I can find but that seems random.
Any hints on how to get more data would be great.
For what it is worth: I am on a NT Heap with LFH.
Edit:
In my sample I of course know the data types but in the real world scenario I am facing these types are unknown to me, even with access to code it might be not obvious which instance of what objects points to this address beside coincidental matches of sizes.
if the UserPtr contained a vftable windbg will normally decode it in the output as below
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
029b1428 0030 0007 [00] 029b1430 00168 - (busy)
dbgeng!Debugger::DataModel::Host::HostModule::`vftable'
as raymond commented you can use dps like dps poi(UserPtr) or print individual entries
0:004> .printf "%y\n" , poi(029b1430)
dbgeng!Debugger::DataModel::Host::HostModule::`vftable' (59cde364)
0:004> .printf "%y\n" , poi(poi(029b1430)+2c)
dbgeng!Debugger::DataModel::Host::BaseSymbol::GetFullyQualifiedName (59f79670)
0:004> .printf "%y\n" , poi(poi(029b1430)+28)
dbgeng!Debugger::DataModel::Host::HostModule::IsEqual (59f77680)
Related
I'm studying the internals of the Windows kernel and one of the things I'm looking into is how paging and virtual addresses work in Windows. I was experimenting with windbg's !vtop function when I noticed something strange I was getting an impossible physical address?
For example here is my output of a !process 0 0 command:
PROCESS fffffa8005319b30
SessionId: none Cid: 0104 Peb: 7fffffd8000 ParentCid: 0004
DirBase: a8df3000 ObjectTable: fffff8a0002f6df0 HandleCount: 29.
Image: smss.exe
when I run !vtop a8df3000 fffffa8005319b30. I get the following result:
lkd> !vtop a8df3000 fffffa8005319b30
Amd64VtoP: Virt fffffa80`05319b30, pagedir a8df3000
Amd64VtoP: PML4E a8df3fa8
Amd64VtoP: PDPE 2e54000
Amd64VtoP: PDE 2e55148
Amd64VtoP: Large page mapped phys 1`3eb19b30
Virtual address fffffa8001f07310 translates to physical address 13eb19b30
The problem I have with this is that my VM that I'm running this test on only has 4GB and 13eb19b30 is 5,346,794,288...
When I run !dd 13eb19b30 and dd fffffa8001f07310 I get the same result so windows seems to be able to access this physical address somehow... Does anyone know how this is done?
I found this post on Cheat Engine that looks like he had a similar problem to me. But they found no solution in that case either
I see You have posted this is RESE also i saw it there didn't understand exactly what you are trying to do.
i see a few discrepancies
you seemed to have used a PFN a8df3000 but it seems windbg seems to be using a PFN of 187000 instead
btw pfn iirc should be dirbase & 0xfffff000
also for virtual address you seem to using the EPROCESS address of your process
are you sure that this is the right virtual address you want to use ?
also it seems you are using lkd which is local kernel debugging prompt
and i hope you understand that lkd is not real kernel debugging
So I think I was finally able to come with a reasonable answer to the problem. It turns out that vmware doesn't seem to actually expose to the VM contiguous memory but instead segments it into different memory "runs". I was able to confirm this by using the volatility:
$ python vol.py -f ~/Desktop/Win7SP1x64-d8737a34.vmss vmwareinfo --verbose | less
Magic: 0xbad1bad1 (Version 1)
Group count: 0x5c
File Offset PhysMem Offset Size
----------- -------------- ----------
0x000010000 0x000000000000 0xc0000000
0x0c0010000 0x000100000000 0xc0000000
Here is a volatility github wiki article that goes into more detail about: volatility
My heap buffer of interest was allocated as follows:
0:047> !heap -p -a 1d7cd1f0
address 1d7cd1f0 found in
_DPH_HEAP_ROOT # 5251000
in busy allocation ( DPH_HEAP_BLOCK: UserAddr UserSize - VirtAddr VirtSize)
1cf8f5b0: 1d7cc008 3ff8 - 1d7cb000 6000
68448e89 verifier!AVrfDebugPageHeapAllocate+0x00000229
76e465ee ntdll!RtlDebugAllocateHeap+0x00000030
76e0a793 ntdll!RtlpAllocateHeap+0x000000c4
76dd5dd0 ntdll!RtlAllocateHeap+0x0000023a
000ca342 TEST+0x0002a342
000be639 TEST+0x0001e639
As you can see, it was allocated using HeapAlloc(). When I run the !address command on the pointer of this heap I get:
ProcessParametrs 01699928 in range 01699000 0169a000
Environment 016976e8 in range 01697000 01698000
1d790000 : 1d7cb000 - 00005000
Type 00020000 MEM_PRIVATE
Protect 00000004 PAGE_READWRITE
State 00001000 MEM_COMMIT
Usage RegionUsageIsVAD
It claims to be in RegionUsageIsVAD. According to this stackoverflow answer, RegionUsageIsVAD generally means two things:
This is a .NET application in which case, the CLR allocates this
block of memory.
The application calls VirtualAlloc to allocate a
bloc of memory.
My scenario does not fit either one of these cases. I confirmed that CLR wasn't used by running .cordll -ve -u -l to which I got:
CLR DLL status: No load attempts
What does RegionUsageIsVAD mean in this case?
i reread your question thinking i would update what i commented
but upon closer look it seems there are lot of holes
it appears you copied things and didnt paste right
where is that pointer on heap ?
01699928 which version of windbg are you using
since i couldn't confirm i cooked up a simple program
enabled hpa in gflags and executed the exe under windbg
below is the screen shot
except what you paste as isregionvad ( this line is output under kernel !address not in user !address ) every thing else appears to be similar in the screenshot
I'm disassembling "Test Drive III". It's a 1990 DOS game. The *.EXE has MZ format.
I've never dealt with segmentation or DOS, so I would be grateful if you answered some of my questions.
1) The game's system requirements mention 286 CPU, which has protected mode. As far as I know, DOS was 90% real mode software, yet some applications could enter protected mode. Can I be sure that the app uses the CPU in real mode only? IOW, is it guaranteed that the segment registers contain actual offset of the segment instead of an index to segment descriptor?
2) Said system requirements mention 1 MB of RAM. How is this amount of RAM even meant to be accessed if the uppermost 384 KB of the address space are reserved for stuff like MMIO and ROM? I've heard about UMBs (using holes in UMA to access RAM) and about HMA, but it still doesn't allow to access the whole 1 MB of physical RAM. So, was precious RAM just wasted because its physical address happened to be reserved for UMA? Or maybe the game uses some crutches like LIM EMS or XMS?
3) Is CS incremented automatically when the code crosses segment boundaries? Say, the IP reaches 0xFFFF, and what then? Does CS switch to the next segment before next instruction is executed? Same goes for SS. What happens when SP goes all the way down to 0x0000?
4) The MZ header of the executable looks like this:
signature 23117 "0x5a4d"
bytes_in_last_block 117
blocks_in_file 270
num_relocs 0
header_paragraphs 32
min_extra_paragraphs 3349
max_extra_paragraphs 65535
ss 11422
sp 128
checksum 0
ip 16
cs 8385
reloc_table_offset 30
overlay_number 0
Why does it have no relocation information? How is it even meant to run without address fixups? Or is it built as completely position-independent code consisting from program-counter-relative instructions? The game comes with a cheat utility which is also an MZ executable. Despite being much smaller (8448 bytes - so small that it fits into a single segment), it still has relocation information:
offset 1
segment 0
offset 222
segment 0
offset 272
segment 0
This allows IDA to properly disassemble the cheat's code. But the game EXE has nothing, even though it clearly has lots of far pointers.
5) Is there even such thing as 'sections' in DOS? I mean, data section, code (text) section etc? The MZ header points to the stack section, but it has no information about data section. Is data and code completely mixed in DOS programs?
6) Why even having a stack section in EXE file at all? It has nothing but zeroes. Why wasting disk space instead of just saying, "start stack from here"? Like it is done with BSS section?
7) MZ header contains information about initial values of SS and CS. What about DS? What's its initial value?
8) What does an MZ executable have after the exe data? The cheat utility has whole 3507 bytes in the end of the executable file which look like
__exitclean.__exit.__restorezero._abort.DGROUP#.__MMODEL._main._access.
_atexit._close._exit._fclose._fflush._flushall._fopen._freopen._fdopen
._fseek._ftell._printf.__fputc._fputc._fputchar.__FPUTN.__setupio._setvbuf
._tell.__MKNAME._tmpnam._write.__xfclose.__xfflush.___brk.___sbrk._brk._sbrk
.__chmod.__close._ioctl.__IOERROR._isatty._lseek.__LONGTOA._itoa._ultoa.
_ltoa._memcpy._open.__open._strcat._unlink.__VPRINTER.__write._free._malloc
._realloc.__REALCVT.DATASEG#.__Int0Vector.__Int4Vector.__Int5Vector.
__Int6Vector.__C0argc.__C0argv.__C0environ.__envLng.__envseg.__envSize
Is this some kind of debugging symbol information?
Thank you in advance for your help.
Re. 1. No, you can't be sure until you prove otherwise to yourself. One giveaway would be the presence of MOV CR0, ... in the code.
Re. 2. While marketing materials aren't to be confused with an engineering specification, there's a technical reason for this. A 286 CPU could address more than 1M of physical address space. The RAM was only "wasted" in real mode, and only if an EMM (or EMS) driver wasn't used. On 286 systems, the RAM past 640kb was usually "pushed up" to start at the 1088kb mark. The ISA and on-board peripherals' memory address space was mapped 1:1 into the 640-1024kb window. To use the RAM from the real mode needed an EMM or EMS driver. From protected mode, it was simply "there" as soon as you set up the segment descriptor correctly.
If the game actually needed the extra 384kb of RAM over the 640kb available in the real mode, it's a strong indication that it either switched to protected mode or required the services or an EMM or EMS driver.
Re. 3. I wish I remembered that. On reflection, I wish not :) Someone else please edit or answer separately. Hah, I did know it at some point in time :)
Re. 4. You say "[the code] has lots of instructions like call far ptr 18DCh:78Ch". This implies one of three things:
Protected mode is used and the segment part of the address is a selector into the segment descriptor table.
There is code there that relocates those instructions without DOS having to do it.
There is code there that forcibly relocates the game to a constant position in the address space. If the game doesn't use DOS to access on-disk files, it can remove DOS completely and take over, gaining lots of memory in the process. I don't recall whether you could exit from the game back to the command prompt. Some games where "play until you reboot".
Re. 5. The .EXE header does not "point" to any stack, there is no stack section you imply, the concept of sections doesn't exist as far as the .EXE file is concerned. The SS register value is obtained by adding the segment the executable was loaded at with the SS value from the header.
It's true that the linker can arrange sections contiguously in the .EXE file, but such sections' properties are not included in the .EXE header. They often can be reverse-engineered by inspecting the executable.
Re. 6. The SS and SP values in the .EXE header are not file pointers. The EXE file might have a part that maps to the stack, but that's entirely optional.
Re. 7. This is already asked and answered here.
Re. 8. This looks like a debug symbol list. The cheat utility was linked with the debugging information left in. You can have completely arbitrary data there - often it'd various resources (graphics, music, etc.).
This is question on JTAG.
I am trying to set a read/write breakpoint in a range of addresses.
Command I give in b.set window is as follows(and selecting read/write)
A:0x8500000..0xd300000
But when i list it(b.list) it shows as follows :
AN:0x0:0x8500000--0xd300000
Why is 0x0 getting appended? So, is my original range of addresses at which i wanted to put the breakpoint is altered?
Additional Information : I am using Lauterbach Trace 32. CPU is ARm Krait
Thank you.
0x00 is the space ID. It is 0 for complete kernel space. For User spaces this space id is the process ID. Space Id's are nothing but Address space identifiers, used by TLB to differentiate the pagetable ebtries.
We have a slow memory leak in our application and I've already gone through the following steps in trying to analyize the cause for the leak:
Enabling user mode stack trace database in GFlags
In Windbg, typing the following command: !heap -stat -h 1250000 (where 1250000 is the address of the heap that has the leak)
After comparing multiple dumps, I see that a memory blocks of size 0xC are increasing over time and are probably the memory that is leaked.
typing the following command: !heap -flt s c
gives the UserPtr of those allocations and finally:
typing !heap -p -a address on some of those addresses always shows the following allocation call stack:
0:000> !heap -p -a 10576ef8
address 10576ef8 found in
_HEAP # 1250000
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
10576ed0 000a 0000 [03] 10576ef8 0000c - (busy)
mscoreei!CLRRuntimeInfoImpl::`vftable'
7c94b244 ntdll!RtlAllocateHeapSlowly+0x00000044
7c919c0c ntdll!RtlAllocateHeap+0x00000e64
603b14a4 mscoreei!UtilExecutionEngine::ClrHeapAlloc+0x00000014
603b14cb mscoreei!ClrHeapAlloc+0x00000023
603b14f7 mscoreei!ClrAllocInProcessHeapBootstrap+0x0000002e
603b1614 mscoreei!operator new[]+0x0000002b
603d402b +0x0000005f
603d5142 mscoreei!GetThunkUseState+0x00000025
603d6fe8 mscoreei!_CorDllMain+0x00000056
79015012 mscoree!ShellShim__CorDllMain+0x000000ad
7c90118a ntdll!LdrpCallInitRoutine+0x00000014
7c919a6d ntdll!LdrpInitializeThread+0x000000c0
7c9198e6 ntdll!_LdrpInitialize+0x00000219
7c90e457 ntdll!KiUserApcDispatcher+0x00000007
This looks like thread initialization call stack but I need to know more than this.
What is the next step you would recommend to do in order to put the finger at the exact cause for the leak.
The stack recorded when using GFlags is done without utilizing .pdb and often not correct.
Since you have traced the leak down to a specific size on a given heap, you can try
to set a live break in RtlAllocateHeap and inspect the stack in windbg with proper symbols. I have used the following with some success. You must edit it to suit your heap and size.
$$ Display stack if heap handle eq 0x00310000 and size is 0x1303
$$ ====================================================================
bp ntdll!RtlAllocateHeap "j ((poi(#esp+4) = 0x00310000) & (poi(#esp+c) = 0x1303) )'k';'gc'"
Maybe you then get another stack and other ideas for the offender.
The first thing is that the new operator is the new [] operator so is there a corresponding delete[] call and not a plain old delete call?
If you suspect this code I would put a test harness around it, for instance put it in a loop and execute it 100 or 1000 times, does it still leak and proportionally.
You can also measure the memory increase using process explorer or programmatically using GetProcessInformation.
The other obvious thing is to see what happens when you comment out this function call, does the memory leak go away? You may need to do a binary chop if possible of the code to reduce the likely suspect code by half (roughly) each time by commenting out code, however, changing the behaviour of the code may cause more problems or dependant code path issues which can cause memory leaks or strange behaviour.
EDIT
Ignore the following seeing as you are working in a managed environment.
You may also consider using the STL or better yet boost reference counted pointers like shared_ptr or scoped_array for array structures to manage the lifetime of the objects.