How can I calculate heap chunk size from raw bytes read from memory.
I tried below thing.
0:001> !heap
Index Address Name Debugging options enabled
1: 00500000
2: 00280000
3: 008f0000
4: 00ab0000
5: 00cc0000
0:001> !heap -a 00500000
..
..
Heap entries for Segment00 in Heap 00500000
address: psize . size flags state (requested size)
00500000: 00000 . 00588 [101] - busy (587)
00500588: 00588 . 00240 [101] - busy (23f)
005007c8: 00240 . 00020 [101] - busy (18)
005007e8: 00020 . 00ca0 [101] - busy (c94)
..
..
!heap -a 00500000 shows that size of first chunk is 588 bytes.
If we dump the chunk header using dt _HEAP_ENTRY, it somehow shows size is 0x3822
0:001> dt _HEAP_ENTRY 00500000
ntdll!_HEAP_ENTRY
+0x000 Size : 0x3822
+0x002 Flags : 0xfc ''
+0x003 SmallTagIndex : 0xbb ''
+0x000 SubSegmentCode : 0xbbfc3822 Void
+0x004 PreviousSize : 0x1849
+0x006 SegmentOffset : 0 ''
+0x006 LFHFlags : 0 ''
+0x007 UnusedBytes : 0x1 ''
+0x000 FunctionIndex : 0x3822
+0x002 ContextValue : 0xbbfc
+0x000 InterceptorValue : 0xbbfc3822
+0x004 UnusedBytesLength : 0x1849
+0x006 EntryOffset : 0 ''
+0x007 ExtendedBlockSignature : 0x1 ''
+0x000 Code1 : 0xbbfc3822
+0x004 Code2 : 0x1849
+0x006 Code3 : 0 ''
+0x007 Code4 : 0x1 ''
+0x000 AgregateCode : 0x01001849`bbfc3822
When I dump the address 0x00500000 I find first two bytes are 22 and 38.
00500000 22 38 fc bb 49 18 00 01 ee ff ee ff 00 00 00 00 a8 00 "8..I.............
00500012 50 00 a8 00 50 00 00 00 50 00 00 00 50 00 00 01 00 00 P...P...P...P.....
00500024 88 05 50 00 00 00 60 00 cf 00 00 00 01 00 00 00 00 00 ..P...`...........
00500036 00 00 f0 0f 53 00 f0 0f 53 00 02 00 00 00 00 00 00 00 ....S...S.........
00500048 00 00 00 00 00 00 10 00 93 38 fd 0b 49 18 00 00 17 ff .........8..I.....
0050005a bb 44 00 00 00 00 00 fe 00 00 ff ee ff ee 00 00 10 00 .D................
0050006c 00 20 00 00 00 08 00 00 00 20 00 00 2e 04 00 00 ff ef . ....... ........
0050007e fd 7f 01 00 38 01 00 00 00 00 00 00 00 00 00 00 00 00 ....8.............
00500090 e8 0f 53 00 e8 0f 53 00 0f 00 00 00 f8 ff ff ff a0 00 ..S...S...........
005000a2 50 00 a0 00 50 00 10 00 50 00 10 00 50 00 00 00 00 00 P...P...P...P.....
My question is how does 22 and 38 (or 0x3822) becomes 0x588
Summary: heap entries are now encoded, the key is in the heap itself.
Let's say I have a heap at 0x00d60000:
0:000> !heap -a 00d60000
Index Address Name Debugging options enabled
2: 00d60000
Segment at 00d60000 to 00d70000 (00001000 bytes committed)
Flags: 40000061
ForceFlags: 40000061
Granularity: 8 bytes
Segment Reserve: 00100000
Segment Commit: 00002000
DeCommit Block Thres: 00000200
DeCommit Total Thres: 00002000
Total Free Size: 00000149
Max. Allocation Size: 7ffdefff
Lock Variable at: 00000000
Next TagIndex: 0000
Maximum TagIndex: 0000
Tag Entries: 00000000
PsuedoTag Entries: 00000000
Virtual Alloc List: 00d6009c
Uncommitted ranges: 00d6008c
00d61000: 0000f000 (61440 bytes)
FreeList[ 00 ] at 00d600c0: 00d605a0 . 00d605a0
00d60598: 00118 . 00a48 [104] - free
Segment00 at 00d60000:
Flags: 00000000
Base: 00d60000
First Entry: 00d60480
Last Entry: 00d70000
Total Pages: 00000010
Total UnCommit: 0000000f
Largest UnCommit:00000000
UnCommitted Ranges: (1)
Heap entries for Segment00 in Heap 00d60000
address: psize . size flags state (requested size)
00d60000: 00000 . 00480 [101] - busy (47f)
00d60480: 00480 . 00118 [107] - busy (100), tail fill
00d60598: 00118 . 00a48 [104] free fill
00d60fe0: 00a48 . 00020 [111] - busy (1d)
00d61000: 0000f000 - uncommitted bytes.
There's a busy block at 0x00d60480: its allocated size is 0x118 (the size of the previous block is 0x480).
If we dump this block we can see it's encoded:
0:000> dt _heap_entry 00d60480
ntdll!_HEAP_ENTRY
+0x000 Size : 0x7387
+0x002 Flags : 0xf5 ''
+0x003 SmallTagIndex : 0x64 'd'
+0x000 SubSegmentCode : 0x64f57387
+0x004 PreviousSize : 0xb95d
+0x006 SegmentOffset : 0 ''
+0x006 LFHFlags : 0 ''
+0x007 UnusedBytes : 0x18 ''
+0x000 FunctionIndex : 0x7387
+0x002 ContextValue : 0x64f5
+0x000 InterceptorValue : 0x64f57387
+0x004 UnusedBytesLength : 0xb95d
+0x006 EntryOffset : 0 ''
+0x007 ExtendedBlockSignature : 0x18 ''
+0x000 Code1 : 0x64f57387
+0x004 Code2 : 0xb95d
+0x006 Code3 : 0 ''
+0x007 Code4 : 0x18 ''
+0x004 Code234 : 0x1800b95d
+0x000 AgregateCode : 0x1800b95d`64f57387
Back to the heap, pay a particular attention to the field named "Encoding " (at offset 0x50):
0:000> dt _heap encoding
ntdll!_HEAP
+0x050 Encoding : _HEAP_ENTRY
Dumping the whole _HEAP structure:
0:000> dt _heap 00d60000
ntdll!_HEAP
+0x000 Entry : _HEAP_ENTRY
+0x008 SegmentSignature : 0xffeeffee
+0x00c SegmentFlags : 0
+0x010 SegmentListEntry : _LIST_ENTRY [ 0xd600a4 - 0xd600a4 ]
+0x018 Heap : 0x00d60000 _HEAP
+0x01c BaseAddress : 0x00d60000 Void
+0x020 NumberOfPages : 0x10
+0x024 FirstEntry : 0x00d60480 _HEAP_ENTRY
+0x028 LastValidEntry : 0x00d70000 _HEAP_ENTRY
+0x02c NumberOfUnCommittedPages : 0xf
+0x030 NumberOfUnCommittedRanges : 1
+0x034 SegmentAllocatorBackTraceIndex : 0
+0x036 Reserved : 0
+0x038 UCRSegmentList : _LIST_ENTRY [ 0xd60ff0 - 0xd60ff0 ]
+0x040 Flags : 0x40000061
+0x044 ForceFlags : 0x40000061
+0x048 CompatibilityFlags : 0
+0x04c EncodeFlagMask : 0x100000
+0x050 Encoding : _HEAP_ENTRY
+0x058 Interceptor : 0
+0x05c VirtualMemoryThreshold : 0xfe00
+0x060 Signature : 0xeeffeeff
+0x064 SegmentReserve : 0x100000
+0x068 SegmentCommit : 0x2000
+0x06c DeCommitFreeBlockThreshold : 0x200
+0x070 DeCommitTotalFreeThreshold : 0x2000
+0x074 TotalFreeSize : 0x149
+0x078 MaximumAllocationSize : 0x7ffdefff
+0x07c ProcessHeapsListIndex : 2
+0x07e HeaderValidateLength : 0x248
+0x080 HeaderValidateCopy : (null)
+0x084 NextAvailableTagIndex : 0
+0x086 MaximumTagIndex : 0
+0x088 TagEntries : (null)
+0x08c UCRList : _LIST_ENTRY [ 0xd60fe8 - 0xd60fe8 ]
+0x094 AlignRound : 0x17
+0x098 AlignMask : 0xfffffff8
+0x09c VirtualAllocdBlocks : _LIST_ENTRY [ 0xd6009c - 0xd6009c ]
+0x0a4 SegmentList : _LIST_ENTRY [ 0xd60010 - 0xd60010 ]
+0x0ac AllocatorBackTraceIndex : 0
+0x0b0 NonDedicatedListLength : 0
+0x0b4 BlocksIndex : 0x00d60248 Void
+0x0b8 UCRIndex : (null)
+0x0bc PseudoTagEntries : (null)
+0x0c0 FreeLists : _LIST_ENTRY [ 0xd605a0 - 0xd605a0 ]
+0x0c8 LockVariable : (null)
+0x0cc CommitRoutine : 0x7944d754 long +7944d754
+0x0d0 FrontEndHeap : (null)
+0x0d4 FrontHeapLockCount : 0
+0x0d6 FrontEndHeapType : 0 ''
+0x0d7 RequestedFrontEndHeapType : 0 ''
+0x0d8 FrontEndHeapUsageData : (null)
+0x0dc FrontEndHeapMaximumIndex : 0
+0x0de FrontEndHeapStatusBitmap : [257] ""
+0x1e0 Counters : _HEAP_COUNTERS
+0x23c TuningParameters : _HEAP_TUNING_PARAMETERS
Dumping the encoding field as two DWORDs:
0:000> dd 00d60000 + 0x50 L2
00d60050 40f273a4 0000b9cd
Now dumping the heap entry as two DWORDs:
0:000> dd 00d60480 L2
00d60480 64f57387 1800b95d
Let's XOR them:
0:000> ? 40f273a4 ^ 64f57387
Evaluate expression: 604438563 = 24070023
0:000> ? 0000b9cd ^ 1800b95d
Evaluate expression: 402653328 = 18000090
Now just writing a fake _HEAP_ENTRY so we can 'dt' it:
0:000> ed 00d604b0
00d604b0 00000000 24070023
24070023
00d604b4 00000000 18000090
18000090
00d604b8 00000000
0:000> dt _HEAP_ENTRY 00d604b0
ntdll!_HEAP_ENTRY
+0x000 Size : 0x23
+0x002 Flags : 0x7 ''
+0x003 SmallTagIndex : 0x24 '$'
+0x000 SubSegmentCode : 0x24070023
+0x004 PreviousSize : 0x90
+0x006 SegmentOffset : 0 ''
+0x006 LFHFlags : 0 ''
+0x007 UnusedBytes : 0x18 ''
+0x000 FunctionIndex : 0x23
+0x002 ContextValue : 0x2407
+0x000 InterceptorValue : 0x24070023
+0x004 UnusedBytesLength : 0x90
+0x006 EntryOffset : 0 ''
+0x007 ExtendedBlockSignature : 0x18 ''
+0x000 Code1 : 0x24070023
+0x004 Code2 : 0x90
+0x006 Code3 : 0 ''
+0x007 Code4 : 0x18 ''
+0x004 Code234 : 0x18000090
+0x000 AgregateCode : 0x18000090`24070023
Size field is 0x23, granularity is 8 bytes (as reported by the !heap -a command output). The real size of the block is the Size field value multiplied by the granularity, so:
0:000> ? 23 * 8
Evaluate expression: 280 = 00000118
It also works for the size of the previous block (reported to be 0x480):
0:000> ? 0x90 * 8
Evaluate expression: 11552 = 00000480
We found the same sizes.
Granularity
Granularity (as given by the !heap -a command output) is not indicated by a specific field, it's just the size of a HEAP_ENTRY structure:
8 bytes on x86 systems (or WOW64):
0:000> ?? sizeof(_HEAP_ENTRY)
unsigned int64 8
16 bytes on x64 systems:
0:000> ?? sizeof(_HEAP_ENTRY)
unsigned int64 0x10
Form Vista and later, the heap entries are scrambled so it’s a hard task
to do any calculations.
Check this link read about randomization.
The DT command are therefore unable do display any sensible information at all.
Take a look at the offsets:
0:001> dt _HEAP_ENTRY
+0x000 Size
+0x000 FunctionIndex
+0x000 InterceptorValue
+0x000 AgregateCode
A lot of elements with same offset, hence same memory.
Also observe your
+0x004 PreviousSize : 0x1849
Does not correspond with the psize of 0000 from !heap –a.
On Win XP and earlier your technique was possible, but here the
_HEAP_ENTRY-> Size
was number of heap blocks, usually of 8 bytes.
Edit:
I’m not aware of any manual method to decode the heap entry, but I guess it’s possible.
I have used the !heap –i command to do it for me.
First:
!heap –i <heap> , in your case !heap –i 00500000
Then
!heap –I <heap entry> , in your case !heap –I 00500588 (for second entry)
Sample:
address: psize . size flags state (requested size)
00240000: 00000 . 00588 [101] - busy (587)
00240588: 00588 . 00240 [101] - busy (23f)
....
0:000> !heap -i 00240000
Heap context set to the heap 0x00240000
0:000> !heap -i 00240588
Detailed information for block entry 00240588
Assumed heap : 0x00240000 (Use !heap -i NewHeapHandle to change)
Header content : 0x32343AD9 0x0100B0F1 (decoded : 0x49010048 0x010000B1)
Owning segment : 0x00240000 (offset 0)
Block flags : 0x1 (busy )
Total block size : 0x48 units (0x240 bytes)
Requested size : 0x23f bytes (unused 0x1 bytes)
Previous block size: 0xb1 units (0x588 bytes)
Block CRC : OK - 0x49
Previous block : 0x00240000
Next block : 0x002407c8
See also : this link
Related
I am trying to get a simple USB device to work on macOS. The device works on Windows, through the use of usbser.sys driver: a simple .inf file is provided and that seems to do the trick (with a .cat file). My reading of usbser.sys is that it will take ownership of USB Communication Device Class (CDC) devices.
I have written user-space drivers for many different devices before by deconstructing USB devices on macOS. My strategy has always been the same: use USB Prober.app (or more recently PyUSB) to get a listing of USB devices with all their configuration parameters, interfaces, pipes & endpoints. The information obtained is enough to fully grasped the device: I can see if the device can be serialized (i.e., to get /dev/cu.serialxxx through some driver) or not, I can see the USB configurations and the USB interfaces. Of course, when possible (that is, when a USB kernel extension takes ownership of the device and serializes it), I use POSIX libraries the serialized version because it is simpler. However, when needed, I use IOKit to access the device directly and find (after trial and error) the right pipes/endpoints for communication.
But I am completely baffled: the device, when connected (it is a laser from Cobolt Lasers) does not show up at all, not even as a USB raw device although it works perfectly well (and every time) on a Windows 10 machine. It is a CDC device, as I validated on Windows. It should be taken by at least the AppleUSBCDC.kext, but no: it does not show up in System Report, in USB Prober or PyUSB listing so of course it does not show up at all in /dev/. I am quite puzzled because although I am no USB expert, I thought I understood that a USB device does not need to be taken by a driver to show up at least in System Report, which would be enough for me to write my own user-space driver using IOKit (which I have done for other devices in the past).
My questions:
Should I expect my device to show up at least in System Report without a driver? What is the solution if I am missing something?
Why isn't AppleUSBCDC.kext not taking the CDC device as a last resort?
Is there an equivalent to usbser.sys on macOS (Mojave) if it's not AppleUSBCDC.kext ?
EDIT 1:
Following the comments, I used sudo dmesg and saw errors during the enumeration process (see below). My working hypothesis at this point is that the USB device descriptor is not quite valid, but that is not a problem on Windows although it is on macOS.
EDIT 2:
Ot appears to be an invalid USB device descriptor, although I am unable to find the exact error in the descriptor. I posted another question regarding a USB device descriptor validator.
I am pasting here the USB information obtained by connecting the device to a Windows machine which to me, clearly indicates that AppleUSBCDC.kext should really take the device and serialize it because CDC devices are (always?) serializable:
=========================== USB Port8 ===========================
Connection Status : 0x01 (Device is connected)
Port Chain : 3-8
======================== USB Device ========================
+++++++++++++++++ Device Information ++++++++++++++++++
Friendly Name : Cobolt Laser (COM7)
Device Description : Cobolt Laser
Device Path : \\?\usb#vid_25dc&pid_0006#000013245678#{a5dcbf10-6530-11d2-901f-00c04fb951ed}
Device ID : USB\VID_25DC&PID_0006\000013245678
Hardware IDs : USB\VID_25DC&PID_0006&REV_0100 USB\VID_25DC&PID_0006
Driver KeyName : {4d36e978-e325-11ce-bfc1-08002be10318}\0006 (GUID_DEVCLASS_PORTS)
Driver : system32\DRIVERS\usbser.sys (Version: 6.1.7601.18247 Date: 2013-08-28)
Driver Inf : C:\Windows\inf\oem107.inf
Legacy BusType : PNPBus
Class : Ports
Class GUID : {4d36e978-e325-11ce-bfc1-08002be10318} (GUID_DEVCLASS_PORTS)
Interface GUID : {a5dcbf10-6530-11d2-901f-00c04fb951ed} (GUID_DEVINTERFACE_USB_DEVICE)
Service : usbser
Enumerator : USB
Location Info : Port_#0008.Hub_#0001
Location IDs : PCIROOT(0)#PCI(1400)#USBROOT(0)#USB(8)
Container ID : {1ddbadd3-ec62-5d3a-a00a-64685c39b2e8}
Manufacturer Info : Cobolt AB
Capabilities : 0x94 (Removable, UniqueID, SurpriseRemovalOK)
Status : 0x0180600A (DN_DRIVER_LOADED, DN_STARTED, DN_DISABLEABLE, DN_REMOVABLE, DN_NT_ENUMERATOR, DN_NT_DRIVER)
Problem Code : 0
Power State : D0 (supported: D0, D3, wake from D0)
COM-Port : COM7 (\Device\USBSER000)
+++++++++++++++++ Registry USB Flags +++++++++++++++++
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\UsbFlags\25DC00060100
osvc : REG_BINARY 00 00
SkipContainerIdQuery : REG_BINARY 01 00 00 00
---------------- Connection Information ---------------
Connection Index : 0x08 (8)
Connection Status : 0x01 (DeviceConnected)
Current Config Value : 0x01
Device Address : 0x0B (11)
Is Hub : 0x00 (no)
Number Of Open Pipes : 0x03 (3)
Device Bus Speed : 0x01 (Full-Speed)
Pipe0ScheduleOffset : 0x00 (0)
Pipe1ScheduleOffset : 0x00 (0)
Pipe2ScheduleOffset : 0x00 (0)
Data (HexDump) : 08 00 00 00 12 01 00 02 02 00 00 08 DC 25 06 00 .............%..
00 01 01 02 03 01 01 01 00 0B 00 03 00 00 00 01 ................
00 00 00 07 05 82 03 40 00 08 00 00 00 00 07 05 .......#........
01 02 40 00 01 00 00 00 00 07 05 81 02 40 00 01 ..#..........#..
00 00 00 00 ....
---------------------- Device Descriptor ----------------------
bLength : 0x12 (18 bytes)
bDescriptorType : 0x01 (Device Descriptor)
bcdUSB : 0x200 (USB Version 2.00)
bDeviceClass : 0x02 (Communications and CDC Control)
bDeviceSubClass : 0x00
bDeviceProtocol : 0x00 (No class specific protocol required)
bMaxPacketSize0 : 0x08 (8 bytes)
idVendor : 0x25DC
idProduct : 0x0006
bcdDevice : 0x0100
iManufacturer : 0x01 (String Descriptor 1)
Language 0x0409 : "Cobolt AB"
iProduct : 0x02 (String Descriptor 2)
Language 0x0409 : "Cobolt Laser Driver MLD"
iSerialNumber : 0x03 (String Descriptor 3)
Language 0x0409 : "000013245678"
bNumConfigurations : 0x01 (1 Configuration)
Data (HexDump) : 12 01 00 02 02 00 00 08 DC 25 06 00 00 01 01 02 .........%......
03 01 ..
------------------ Configuration Descriptor -------------------
bLength : 0x09 (9 bytes)
bDescriptorType : 0x02 (Configuration Descriptor)
wTotalLength : 0x0043 (67 bytes)
bNumInterfaces : 0x02 (2 Interfaces)
bConfigurationValue : 0x01 (Configuration 1)
iConfiguration : 0x00 (No String Descriptor)
bmAttributes : 0x80
D7: Reserved, set 1 : 0x01
D6: Self Powered : 0x00 (no)
D5: Remote Wakeup : 0x00 (no)
D4..0: Reserved, set 0 : 0x00
MaxPower : 0x32 (100 mA)
Data (HexDump) : 09 02 43 00 02 01 00 80 32 09 04 00 00 01 02 02 ..C.....2.......
01 00 05 24 00 10 01 05 24 01 01 01 04 24 02 06 ...$....$....$..
05 24 06 00 01 07 05 82 03 40 00 08 09 04 01 00 .$.......#......
02 0A 00 00 00 07 05 01 02 40 00 01 07 05 81 02 .........#......
40 00 01 #..
---------------- Interface Descriptor -----------------
bLength : 0x09 (9 bytes)
bDescriptorType : 0x04 (Interface Descriptor)
bInterfaceNumber : 0x00
bAlternateSetting : 0x00
bNumEndpoints : 0x01 (1 Endpoint)
bInterfaceClass : 0x02 (Communications and CDC Control)
bInterfaceSubClass : 0x02 (Abstract Control Model)
bInterfaceProtocol : 0x01 (AT Commands defined by ITU-T V.250 etc)
iInterface : 0x00 (No String Descriptor)
Data (HexDump) : 09 04 00 00 01 02 02 01 00 .........
-------------- CDC Interface Descriptor ---------------
bFunctionLength : 0x05 (5 bytes)
bDescriptorType : 0x24 (Interface)
bDescriptorSubType : 0x00 (Header Functional Descriptor)
bcdCDC : 0x110 (CDC Version 1.10)
Data (HexDump) : 05 24 00 10 01 .$...
-------------- CDC Interface Descriptor ---------------
bFunctionLength : 0x05 (5 bytes)
bDescriptorType : 0x24 (Interface)
bDescriptorSubType : 0x01 (Call Management Functional Descriptor)
bmCapabilities : 0x01
D7..2: : 0x00 (Reserved)
D1 : : 0x00 (sends/receives call management information only over the Communication Class interface)
D0 : : 0x01 (handles call management itself)
bDataInterface : 0x01
Data (HexDump) : 05 24 01 01 01 .$...
-------------- CDC Interface Descriptor ---------------
bFunctionLength : 0x04 (4 bytes)
bDescriptorType : 0x24 (Interface)
bDescriptorSubType : 0x02 (Abstract Control Management Functional Descriptor)
bmCapabilities : 0x06
D7..4: : 0x00 (Reserved)
D3 : : 0x00 (not supports the notification Network_Connection)
D2 : : 0x01 (supports the request Send_Break)
D1 : : 0x01 (supports the request combination of Set_Line_Coding, Set_Control_Line_State, Get_Line_Coding, and the notification Serial_State)
D0 : : 0x00 (not supports the request combination of Set_Comm_Feature, Clear_Comm_Feature, and Get_Comm_Feature)
Data (HexDump) : 04 24 02 06 .$..
-------------- CDC Interface Descriptor ---------------
bFunctionLength : 0x05 (5 bytes)
bDescriptorType : 0x24 (Interface)
bDescriptorSubType : 0x06 (Union Functional Descriptor)
bControlInterface : 0x00
bSubordinateInterface[0] : 0x01
Data (HexDump) : 05 24 06 00 01 .$...
----------------- Endpoint Descriptor -----------------
bLength : 0x07 (7 bytes)
bDescriptorType : 0x05 (Endpoint Descriptor)
bEndpointAddress : 0x82 (Direction=IN EndpointID=2)
bmAttributes : 0x03 (TransferType=Interrupt)
wMaxPacketSize : 0x0040 (64 bytes)
bInterval : 0x08 (8 ms)
Data (HexDump) : 07 05 82 03 40 00 08 ....#..
---------------- Interface Descriptor -----------------
bLength : 0x09 (9 bytes)
bDescriptorType : 0x04 (Interface Descriptor)
bInterfaceNumber : 0x01
bAlternateSetting : 0x00
bNumEndpoints : 0x02 (2 Endpoints)
bInterfaceClass : 0x0A (CDC-Data)
bInterfaceSubClass : 0x00
bInterfaceProtocol : 0x00
iInterface : 0x00 (No String Descriptor)
Data (HexDump) : 09 04 01 00 02 0A 00 00 00 .........
----------------- Endpoint Descriptor -----------------
bLength : 0x07 (7 bytes)
bDescriptorType : 0x05 (Endpoint Descriptor)
bEndpointAddress : 0x01 (Direction=OUT EndpointID=1)
bmAttributes : 0x02 (TransferType=Bulk)
wMaxPacketSize : 0x0040 (64 bytes)
bInterval : 0x01 (ignored)
Data (HexDump) : 07 05 01 02 40 00 01 ....#..
----------------- Endpoint Descriptor -----------------
bLength : 0x07 (7 bytes)
bDescriptorType : 0x05 (Endpoint Descriptor)
bEndpointAddress : 0x81 (Direction=IN EndpointID=1)
bmAttributes : 0x02 (TransferType=Bulk)
wMaxPacketSize : 0x0040 (64 bytes)
bInterval : 0x01 (ignored)
Data (HexDump) : 07 05 81 02 40 00 01 ....#..
----------------- Device Qualifier Descriptor -----------------
Error : ERROR_GEN_FAILURE
-------------------- String Descriptors -------------------
------ String Descriptor 0 ------
bLength : 0x04 (4 bytes)
bDescriptorType : 0x03 (String Descriptor)
Language ID[0] : 0x0409 (English - United States)
Data (HexDump) : 04 03 09 04 ....
------ String Descriptor 1 ------
bLength : 0x14 (20 bytes)
bDescriptorType : 0x03 (String Descriptor)
Language 0x0409 : "Cobolt AB"
Data (HexDump) : 14 03 43 00 6F 00 62 00 6F 00 6C 00 74 00 20 00 ..C.o.b.o.l.t. .
41 00 42 00 A.B.
------ String Descriptor 2 ------
bLength : 0x30 (48 bytes)
bDescriptorType : 0x03 (String Descriptor)
Language 0x0409 : "Cobolt Laser Driver MLD"
Data (HexDump) : 30 03 43 00 6F 00 62 00 6F 00 6C 00 74 00 20 00 0.C.o.b.o.l.t. .
4C 00 61 00 73 00 65 00 72 00 20 00 44 00 72 00 L.a.s.e.r. .D.r.
69 00 76 00 65 00 72 00 20 00 4D 00 4C 00 44 00 i.v.e.r. .M.L.D.
------ String Descriptor 3 ------
bLength : 0x1A (26 bytes)
bDescriptorType : 0x03 (String Descriptor)
Language 0x0409 : "000013245678"
Data (HexDump) : 1A 03 30 00 30 00 30 00 30 00 31 00 33 00 32 00 ..0.0.0.0.1.3.2.
34 00 35 00 36 00 37 00 38 00 4.5.6.7.8.
----- String Descriptor 0xEE -----
bLength : 0x02 (2 bytes)
bDescriptorType : 0x03 (String Descriptor)
Language 0x0409 : "" *!*CAUTION zero length
Data (HexDump) : 02 03 ..
Regularly I need to investigate dumpfiles, always in the same way, and I'd like to automate this. I'm using Windbg as a tool, and I'm thinking of using Windbg scripting.
I have done some first attempts with PYKD, but I don't like the overhead that much, so I've opted for the standard Windbg scripting, but this is getting into a nightmare, let me show you what I want to do:
0:001> kb
# RetAddr : Args to Child : Call Site
00 00007ffc`26272685 : ffffffff`fffffffe 00007ff7`06e563f0 00007ff7`00000000 0000005a`1fb6fd70 : user32!NtUserGetMessage+0xa
01 00007ff7`06d87596 : 00000000`00000008 00007ff7`06e5d048 00000000`00007c1c 0000005a`00000004 : user32!GetMessageW+0x25
02 00007ff7`06d87673 : 0000005a`1f2b3710 00007ff7`06e5c7d0 0000005a`1f2ac270 00000000`00000002 : <Application>!CServiceModule::Run+0x8ee [sourcefile.cpp # 1905]
03 00007ffc`26875ada : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : <Application>!CServiceModule::ServiceMain+0x63 [sourcefile.cpp # 1379]
04 00007ffc`26ef13d2 : 00007ffc`26875aa0 0000005a`1f2ac270 00000000`00000000 00000000`00000000 : sechost!ScSvcctrlThreadA+0x3a
05 00007ffc`270454f4 : 00007ffc`26ef13b0 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x22
06 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34
0:001> dx Debugger.Sessions[0].Processes[26520].Threads[14812].Stack.Frames[2].SwitchTo();dv /t /v
// The second command "dx ..." happens by clicking on the DML link of the line, corresponding with the line, containing "CServiceModule::Run".
Now, how to do this in a Windbg script?
Launch the kb command and put the result in a variable. (How to do that? I already tried using as and aS but neither seems to work)
If I can't get the whole kb result in one variable, try to use a .foreach, as in following example:
.foreach ( token { kb} )
{
.printf "TEST\n"
}
=> even that is not working: kb has about 8 lines of results, while the word TEST is printed 78 times.
Instead of printing the word "TEST", print the variable token and see what it looks like.
=> How to do that? I already tried .printf "%msu \n" , ${token}, or #$token, ... but nothing is working.
Even if I get this working: how can I do string manipulation, decimal/hexadecimal number conversion, ...?
Not to forget: how do I compile such a script? I'm currently trying to run the script in Windbg, which sometimes gives compilation errors, but those are very unreadable (I add a new line, causing a problem, but the complication error (not a typo) does not even mention that newly added line).
You might say: just have a look at the examples, mentioned under this URL, but I can't find one example of a standard Windbg command, being run, and have its result stored in a variable (which is the first thing I need to do).
In case I get my script working, I might turn this post into a general "Windbg scripting FAQ" for the mentioned (and newly added) questions.
Edit after first answer
I realise that I've mistaken with the kb command: the actual command I need to use is ~* k, giving following result:
(Small remark: I've just a screenshot, instead of a text copy, in order to emphasize the DML hyperlinks)
As you can see, there are some DML results, and I would like to "click" on the line, containing CServiceModule::Run. When I do this by hand, there seems to be a translation towards the following command:
dx Debugger.Sessions[0].Processes[26520].Threads[14812].Stack.Frames[2].SwitchTo()
Here 26520 is the HexToDec conversion of 6798,
and 14812 is the HexToDec conversion of 39DC.
(Both to be retrieved from the screenshot's Id: 6798.39dc)
So I "need" the string manipulation and the HexToDec conversion in order to simulate DML clicking. If, however, you know an easier way to do put this "click" action into a script, I'd very very thankful!
for .foreach space is the delimiter
this command is not meant for reading lines
if you want to print the token use it as is in a alias interpreter
0:001> kb
# ChildEBP RetAddr Args to Child
00 0147fa44 7706f20f 7642d6f7 00000000 00000000 ntdll!DbgBreakPoint
01 0147fa74 75d1ed6c 00000000 0147fac0 770337eb ntdll!DbgUiRemoteBreakin+0x3c
02 0147fa80 770337eb 00000000 7642d643 00000000 kernel32!BaseThreadInitThunk+0xe
03 0147fac0 770337be 7706f1d3 00000000 00000000 ntdll!__RtlUserThreadStart+0x70
04 0147fad8 00000000 7706f1d3 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b
0:001> r $t0 =0 ;.foreach (token { kb } ) { r $t0 = #$t0+1 ; .printf "\"token\" %2d ${token}\n" , #$t0 }
"token" 1 #
"token" 2 ChildEBP
"token" 3 RetAddr
"token" 4 Args
"token" 5 to
"token" 6 Child
"token" 7 00
"token" 8 0147fa44
"token" 9 7706f20f
"token" 10 7642d6f7
"token" 11 00000000
"token" 12 00000000
"token" 13 ntdll!DbgBreakPoint
"token" 14 01
"token" 15 0147fa74
"token" 16 75d1ed6c
"token" 17 00000000
"token" 18 0147fac0
"token" 19 770337eb
"token" 20 ntdll!DbgUiRemoteBreakin+0x3c
"token" 21 02
"token" 22 0147fa80
"token" 23 770337eb
"token" 24 00000000
"token" 25 7642d643
"token" 26 00000000
"token" 27 kernel32!BaseThreadInitThunk+0xe
"token" 28 03
"token" 29 0147fac0
"token" 30 770337be
"token" 31 7706f1d3
"token" 32 00000000
"token" 33 00000000
"token" 34 ntdll!__RtlUserThreadStart+0x70
"token" 35 04
"token" 36 0147fad8
"token" 37 00000000
"token" 38 7706f1d3
"token" 39 00000000
"token" 40 00000000
"token" 41 ntdll!_RtlUserThreadStart+0x1b
i do not understand your further queries so iam assuming your intent below
0:001> $$ putting the result of kb into a variable use javascript
0:001> dx #$foo = Debugger.Utility.Control.ExecuteCommand("kb")
#$foo = Debugger.Utility.Control.ExecuteCommand("kb")
[0x0] : # ChildEBP RetAddr Args to Child
[0x1] : 00 0147fa44 7706f20f 7642d6f7 00000000 00000000 ntdll!DbgBreakPoint
[0x2] : 01 0147fa74 75d1ed6c 00000000 0147fac0 770337eb ntdll!DbgUiRemoteBreakin+0x3c
[0x3] : 02 0147fa80 770337eb 00000000 7642d643 00000000 kernel32!BaseThreadInitThunk+0xe
[0x4] : 03 0147fac0 770337be 7706f1d3 00000000 00000000 ntdll!__RtlUserThreadStart+0x70
[0x5] : 04 0147fad8 00000000 7706f1d3 00000000 00000000 ntdll!_RtlUserThreadStart+0x1b
0:001> dx -r0 #$foo[1]
#$foo[1] : 00 0147fa44 7706f20f 7642d6f7 00000000 00000000 ntdll!DbgBreakPoint
0:001> dx -r0 #$foo[2]
#$foo[2] : 01 0147fa74 75d1ed6c 00000000 0147fac0 770337eb ntdll!DbgUiRemoteBreakin+0x3c
again i don't know what you want to convert to hex or dec
so assumption below
trying to dec a hex from stack as well uppercasing strings
a script like this
function log(a1,a2)
{
host.diagnostics.debugLog(a1 +" " +a2 + "\n");
}
function exec (cmdstr)
{
return host.namespace.Debugger.Utility.Control.ExecuteCommand(cmdstr);
}
function kbtok()
{
var temp = exec("kb")
for ( line of temp )
{
var token = line.split(" ")
for (tok of token)
{
//log(tok)
try {
var num = host.parseInt64(tok,16)
log(tok , num)
}
catch(err) {
log(tok.toUpperCase(), " ")
}
}
}
}
using it like
.load jsprovider
.scriptload ---path----
dx #$scriptcontents.kbtok()
sample output for stack in first paragraph
0:001> dx #$scriptContents.kbtok()
#
CHILDEBP
RETADDR
ARGS
TO
CHILD
00 0
013bfef4 20709108
7706f20f 1996943887
763ed3a7 1983828903
00000000 0
00000000 0
NTDLL!DBGBREAKPOINT
01 1
013bff24 20709156
75d1ed6c 1976692076
00000000 0
013bff70 20709232
770337eb 1996699627
NTDLL!DBGUIREMOTEBREAKIN+0X3C
02 2
013bff30 20709168
770337eb 1996699627
00000000 0
763ed3f3 1983828979
00000000 0
KERNEL32!BASETHREADINITTHUNK+0XE
03 3
013bff70 20709232
770337be 1996699582
7706f1d3 1996943827
00000000 0
00000000 0
NTDLL!__RTLUSERTHREADSTART+0X70
04 4
013bff88 20709256
00000000 0
7706f1d3 1996943827
00000000 0
00000000 0
NTDLL!_RTLUSERTHREADSTART+0X1B
#$scriptContents.kbtok()
you do not compile a script
scripts are interpreted
for a windbg script use $$>a< ---path to script----
for javascript use .scriptload command
I'm working on a script that needs to extract a few information from smbiosDump from ESXi. The smbiosDump looks something like this:
Dumping live SMBIOS data!
BIOS Info: #0
Size: 0x00018
Vendor: "FUJITSU // American Megatrends Inc."
Version: "V5.0.0.11 R1.2.0 for D3373-B1x"
Date: "03/16/2017"
Start Address: 0xf0000
ROM Size: 16384 kB
Features: 0x0d03000000013c099880
PCI supported
BIOS flashable
BIOS shadowing allowed
CD boot supported
Selectable boot supported
EDD spec supported
Print Screen supported
8042 Keyboard Services supported
Serial Services supported
Printer Services supported
ACPI supported
USB Legacy supported
BIOS Boot Spec supported
Enable targeted content distribution
UEFI supported
System BIOS release: 1.2
Port Connector: #4
Type: 0x08 (Serial Port 16550 Compatible)
Internal Designator: "Serial"
External Designator: "Serial 1"
External Connector: 0x08 (DB-9 pin male)
Port Connector: #5
Type: 0x1c (Video Port)
Internal Designator: "VGA"
External Designator: "Rear: Video"
External Connector: 0x07 (DB-15 pin female)
Port Connector: #6
Type: 0x1f (Network Port)
Internal Designator: "LAN i210-AT"
External Designator: "LAN 1"
External Connector: 0x0b (RJ-45)
Port Connector: #7
Type: 0x1f (Network Port)
Internal Designator: "LAN i210-AT"
External Designator: "LAN 2"
External Connector: 0x0b (RJ-45)
Port Connector: #8
Type: 0x1f (Network Port)
Internal Designator: "iRMC-S4"
External Designator: "LAN M"
External Connector: 0x0b (RJ-45)
Port Connector: #9
Type: 0x10 (USB)
Internal Designator: "USB 2.0 REAR"
External Designator: "Rear: USB 1"
External Connector: 0x12 (Access Bus [USB])
Port Connector: #10
Type: 0x10 (USB)
Internal Designator: "USB 2.0 REAR"
External Designator: "Rear: USB 2"
External Connector: 0x12 (Access Bus [USB])
Port Connector: #11
Type: 0x10 (USB)
Internal Designator: "USB 2.0 REAR"
External Designator: "Rear: USB 3"
External Connector: 0x12 (Access Bus [USB])
Port Connector: #12
Type: 0x20 (SATA)
Internal Designator: "SATA 1-4"
Internal Connector: 0xff (Other)
Port Connector: #13
Type: 0x20 (SATA)
Internal Designator: "SATA 5 DOM"
Internal Connector: 0x22 (Other)
Port Connector: #14
Type: 0x20 (SATA)
Internal Designator: "SATA 6 DOM"
Internal Connector: 0x22 (Other)
Port Connector: #15
Type: 0x10 (USB)
Internal Designator: "UFM"
Internal Connector: 0xff (Other)
Port Connector: #16
Type: 0x10 (USB)
Internal Designator: "DAT"
Internal Connector: 0x12 (Access Bus [USB])
Port Connector: #17
Type: 0xff (Other)
Internal Designator: "TPM HDR"
Internal Connector: 0xff (Other)
System Slot: #18
Designation: "SLOT 1"
Type: 0xa5 (PCI Express)
Bus Width: 0x08 (x1)
Status: 0x03 (Available)
Length: 0x03 (Short)
Slot ID: 1
Characteristics: 0x0104 (3.3 V, PME#)
System Slot: #19
Designation: "SLOT2"
Type: 0xa5 (PCI Express)
Bus Width: 0x0a (x4)
Status: 0x04 (In Use)
Length: 0x03 (Short)
Slot ID: 2
Characteristics: 0x0104 (3.3 V, PME#)
System Slot: #20
Designation: "SLOT3"
Type: 0xa5 (PCI Express)
Bus Width: 0x0b (x8)
Status: 0x04 (In Use)
Length: 0x03 (Short)
Slot ID: 3
Characteristics: 0x0104 (3.3 V, PME#)
System Slot: #21
Designation: "SLOT4"
Type: 0xa5 (PCI Express)
Bus Width: 0x0b (x8)
Status: 0x04 (In Use)
Length: 0x03 (Short)
Slot ID: 4
Characteristics: 0x010c (3.3 V, Shared, PME#)
On Board Devices: #22
Video: "VGA iRMC4"
Ethernet: "LAN i210-AT"
Ethernet: "LAN i210-AT"
OEM Strings: #23
FUJITSU
FUJITSU
FUJITSU TECHNOLOGY SOLUTIONS
System Config Options (Jumpers & Switches) #24:
PWD-CLR: Password Skip active
RCVR: Recovery BIOS active
Hardware Security: #25
Power-on Password: 0x00 (Disabled)
Keyboard Password: 0x02 (Not Implemented)
Admin Password: 0x00 (Disabled)
Front Panel Reset: 0x02 (Not Implemented)
Type 32 Record: #26
Data 00: 20 14 1a 00 00 00 00 00 00 00 00 00 00 00 00 00
Data 10: 00 00 00 00
Type 41 Record: #27
Data 00: 29 0b 1b 00 01 83 01 00 00 03 00
String 1: "VGA iRMC4"
Type 41 Record: #28
Data 00: 29 0b 1c 00 01 85 01 00 00 04 00
String 1: "LAN i210-AT"
Type 41 Record: #29
Data 00: 29 0b 1d 00 01 85 02 00 00 05 00
String 1: "LAN i210-AT"
Type 38 Record: #30
Data 00: 26 12 1e 00 01 20 20 ff a3 0c 00 00 00 00 00 00
Data 10: 00 00
Cache Info: #31
Designation: "L1 Cache"
Level: L1
State: Enabled
Mode: 0x00 (Write Through)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x02 (Unknown)
Type: 0x04 (Data)
Associativity: 0x07 (8-way Set-Associative)
Max. Size: 128 kB
Current Size: 128 kB
Supported SRAM Types: 0x0002 (Unknown)
Current SRAM Type: 0x0002 (Unknown)
Cache Info: #32
Designation: "L1 Cache"
Level: L1
State: Enabled
Mode: 0x00 (Write Through)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x02 (Unknown)
Type: 0x03 (Instruction)
Associativity: 0x07 (8-way Set-Associative)
Max. Size: 128 kB
Current Size: 128 kB
Supported SRAM Types: 0x0002 (Unknown)
Current SRAM Type: 0x0002 (Unknown)
Cache Info: #33
Designation: "L2 Cache"
Level: L2
State: Enabled
Mode: 0x00 (Write Through)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x05 (Single-bit)
Type: 0x05 (Unified)
Associativity: 0x05 (4-way Set-Associative)
Max. Size: 1024 kB
Current Size: 1024 kB
Supported SRAM Types: 0x0002 (Unknown)
Current SRAM Type: 0x0002 (Unknown)
Cache Info: #34
Designation: "L3 Cache"
Level: L3
State: Enabled
Mode: 0x01 (Write Back)
Location: 0x00 (Internal, Not Socketed)
ECC: 0x05 (Single-bit)
Type: 0x05 (Unified)
Associativity: 0x09 (12-way Set-Associative)
Max. Size: 6144 kB
Current Size: 6144 kB
Supported SRAM Types: 0x0002 (Unknown)
Current SRAM Type: 0x0002 (Unknown)
Processor Info: #35
Payload length: 0x30
Socket: "CPU1"
Socket Type: 0x01 (Other)
Socket Status: Populated
Type: 0x03 (CPU)
Family: 0xb3 (Xeon)
Manufacturer: "Intel(R) Corporation"
Version: "Intel(R) Xeon(R) CPU E3-1280 v6 # 3.90GHz"
Processor ID: 0xbfebfbff000906e9
Status: 0x01 (Enabled)
Voltage: 1.1 V
External Clock: 100 MHz
Max. Speed: 4200 MHz
Current Speed: 3900 MHz
L1 Cache: #32
L2 Cache: #33
L3 Cache: #34
Core Count: #4
Core Enabled Count: #4
Thread Count: #8
Physical Memory Array: #36
Use: 0x03 (System memory)
Location: 0x03 (Motherboard)
Slots: 4
Max. Size: 64 GB
ECC: 0x05 (Single-bit)
Memory Device: #37
Location: "DIMM CHA3"
Bank: "BANK 0"
Manufacturer: "Samsung"
Serial: "002FEFAAF"
Asset Tag: "0123456789"
Part Number: "M391A1K43BB1-CRC"
Memory Array: #36
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x0080 (Synchronous)
Data Width: 64 bits (+8 ECC bits)
Size: 8 GB
Speed: 2400 MHz
Memory Device: #38
Location: "DIMM CHA1"
Bank: "BANK 1"
Manufacturer: "Samsung"
Serial: "002FEFABA"
Asset Tag: "0123456789"
Part Number: "M391A1K43BB1-CRC"
Memory Array: #36
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x0080 (Synchronous)
Data Width: 64 bits (+8 ECC bits)
Size: 8 GB
Speed: 2400 MHz
Memory Device: #39
Location: "DIMM CHB4"
Bank: "BANK 2"
Manufacturer: "Samsung"
Serial: "002FEFB14"
Asset Tag: "0123456789"
Part Number: "M391A1K43BB1-CRC"
Memory Array: #36
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x0080 (Synchronous)
Data Width: 64 bits (+8 ECC bits)
Size: 8 GB
Speed: 2400 MHz
Memory Device: #40
Location: "DIMM CHB2"
Bank: "BANK 3"
Manufacturer: "Samsung"
Serial: "002FEFAB7"
Asset Tag: "0123456789"
Part Number: "M391A1K43BB1-CRC"
Memory Array: #36
Form Factor: 0x09 (DIMM)
Type: 0x1a (DDR4)
Type Detail: 0x0080 (Synchronous)
Data Width: 64 bits (+8 ECC bits)
Size: 8 GB
Speed: 2400 MHz
Memory Array Mapping: #41
Memory Array: #36
Partition Width: 4
Start Address: 0x0000000000000000
End Address: 0x0000000800000000
Type 176 Record: #42
Data 00: b0 24 2a 00 00 00 73 33 00 11 33 00 02 31 ff ff
Data 10: 72 fd ad 6a 08 39 a4 89 04 46 a1 4d f9 4a 28 3f
Data 20: 01 00 01 00
Type 177 Record: #43
Data 00: b1 2a 2b 00 12 00 00 00 12 00 00 00 12 05 00 00
Data 10: 0b 01 02 00 00 00 00 00 00 00 00 12 00 00 00 00
Data 20: 00 00 00 00 00 00 00 00 00 00
Type 188 Record: #44
Data 00: bc 08 2c 00 30 39 31 24
Type 136 Record: #45
Data 00: 88 06 2d 00 5a 5a
System Info: #46
Manufacturer: "FUJITSU"
Product: "PRIMERGY TX1320 M3"
Serial: "YMBK000042"
UUID: 09000800070006000500040003000200
Wake-up: 0x06 (Power Switch)
Board Info: #47
Manufacturer: "FUJITSU"
Product: "D3373-B1"
Version: "S26361-D3373-B12 WGS03 GS51"
Serial: "50896663"
Type: 0x0a (Motherboard)
Features: 0x09
Hosting Board
Replaceable
Chassis: #3
Chassis Info: #48
Manufacturer: "FUJITSU"
Version: "TX1320M3F5"
Serial: "YMBK000042"
Asset Tag: "System Asset Tag"
Type: 0x11 (Main Server Chassis)
Bootup State: 0x03 (Safe)
Power Supply State: 0x03 (Safe)
Thermal State: 0x03 (Safe)
Security Status: 0x03 (None)
Memory Device Mapping: #49
Memory Device: #37
Array Mapping: #41
Interleave Pos: 1
Interleaved Depth: 2
Start Address: 0x0000000000000000
End Address: 0x0000000200000000
Memory Device Mapping: #50
Memory Device: #38
Array Mapping: #41
Interleave Pos: 1
Interleaved Depth: 2
Start Address: 0x0000000400000000
End Address: 0x0000000600000000
Memory Device Mapping: #51
Memory Device: #39
Array Mapping: #41
Interleave Pos: 2
Interleaved Depth: 2
Start Address: 0x0000000200000000
End Address: 0x0000000400000000
Memory Device Mapping: #52
Memory Device: #40
Array Mapping: #41
Interleave Pos: 2
Interleaved Depth: 2
Start Address: 0x0000000600000000
End Address: 0x0000000800000000
Type 221 Record: #53
Data 00: dd 1a 35 00 03 01 00 04 01 00 01 00 02 00 00 00
Data 10: 00 48 00 03 00 00 05 00 00 00
String 1: "Reference Code - CPU"
String 2: "uCode Version"
String 3: "TXT ACM version"
Type 221 Record: #54
Data 00: dd 44 36 00 09 01 00 04 01 00 01 00 02 03 ff ff
Data 10: ff ff ff 04 00 ff ff ff 31 00 05 00 ff ff ff 31
Data 20: 00 06 00 ff ff ff ff ff 07 00 3e 00 00 00 00 08
Data 30: 00 34 00 00 00 00 09 00 3e 00 00 00 00 0a 00 34
Data 40: 00 00 00 00
String 1: "Reference Code - SKL PCH"
String 2: "PCH-CRID Status"
String 3: "Disabled"
String 4: "PCH-CRID Original Value"
String 5: "PCH-CRID New Value"
String 6: "OPROM - RST - RAID"
String 7: "SKL PCH H Bx Hsio Version"
String 8: "SKL PCH H Dx Hsio Version"
String 9: "SKL PCH LP Bx Hsio Version"
String 10: "SKL PCH LP Cx Hsio Version"
Type 221 Record: #55
Data 00: dd 36 37 00 07 01 00 04 01 00 01 00 02 00 04 01
Data 10: 00 01 00 03 00 04 01 00 00 00 04 05 ff ff ff ff
Data 20: ff 06 00 ff ff ff 05 00 07 00 ff ff ff 05 00 08
Data 30: 00 ff ff ff ff ff
String 1: "Reference Code - SA - System Agent"
String 2: "Reference Code - MRC"
String 3: "SA - PCIe Version"
String 4: "SA-CRID Status"
String 5: "Disabled"
String 6: "SA-CRID Original Value"
String 7: "SA-CRID New Value"
String 8: "OPROM - VBIOS"
Type 221 Record: #56
Data 00: dd 60 38 00 0d 01 00 00 00 00 00 00 02 00 ff ff
Data 10: ff ff ff 03 04 ff ff ff ff ff 05 06 ff ff ff ff
Data 20: ff 07 08 ff ff ff ff ff 09 00 00 00 00 00 00 0a
Data 30: 00 ff ff ff ff 00 0b 00 ff ff 00 00 00 0c 00 ff
Data 40: ff ff ff ff 0d 00 ff ff ff ff ff 0e 00 ff ff ff
Data 50: ff ff 0f 00 ff ff ff ff ff 10 11 01 02 02 03 00
String 1: "Lan Phy Version"
String 2: "Sensor Firmware Version"
String 3: "Debug Mode Status"
String 4: "Disabled"
String 5: "Performance Mode Status"
String 6: "Disabled"
String 7: "Debug Use USB(Disabled:Serial)"
String 8: "Disabled"
String 9: "ICC Overclocking Version"
String 10: "UNDI Version"
String 11: "EC FW Version"
String 12: "GOP Version"
String 13: "BIOS Guard Version"
String 14: "Base EC FW Version"
String 15: "EC-EC Protocol Version"
String 16: "Royal Park Version"
String 17: "BP1.2.2.0_RP03"
Group Associations: #57
Group Name: "Firmware Version Info"
Items: #53, #54, #55, #56
Language Info: #58
Languages: en|US|iso8859-1
Current: en|US|iso8859-1
On different systems the dump is different, either longer or shorter.
I need to extract a few information from this dump file and from other similar dump files, the L1 (or L2, or L3) Cache Max. Size information. and make it print out something like this:
Level: L1
Max Size: 128 kb
I would also like to note that the Cache Info numbering (#32, #31, etc) is different for every system. What is #31 here could be #41 on another system, or #29. So trying to use the numbering as a filter will not work unfortunately..
(The same goes for L2 or L3, but on a different script.)
(Since on my original question, i had to make a new post, here is the other page: How can I extract a section of a file based on matching only one line?)
Could anyone please help me out with this?
You can pipe your dump to egrep :
…|egrep 'Level|Max. Size'
This will select only lines containing either Level or Max. Size.
If you don't want to bother removing Max. Size lines not associated to a Level by hand, you can pipe further into awk :
…|egrep 'Level|Max. Size'|awk '$1!=prevline{print $0}{prevline=$1;}'
This will check that the first word ($1) is not the same as in prevline, then print the whole line, and store the first word in prevline.
I have already encountered some Oops in my developer's life and whereas I am familiar with some information that I can retrieve from these Oops, there are still pieces of information I can't understand and therefore, can't use to solve problems.
Below you will find an Oops example and I will describe what I can deduce from it. Then, I will ask what the remaining info can teach me about the problem.
[ 716.485951] BUG: unable to handle kernel paging request at fc132158
[ 716.485973] IP: [<fc1936e7>] ubi_change_vtbl_record+0x87/0x1c0 [ubi]
[ 716.485986] *pdpt = 00000000019e6001 *pde = 000000002c558067 *pte = 0000000000000000
[ 716.485997] Oops: 0002 [#1] SMP
[ 716.486004] Modules linked in: ubi(O) mtdchar nandsim nand mtd nand_ids nand_bch bch nand_ecc bnep rfcomm bluetooth parport_pc ppdev lp parport nfsd nfs_acl auth_rpcgss nfs fscache lockd sunrpc binfmt_misc dm_crypt snd_hda_codec_hdmi snd_hda_codec_analog kvm_intel snd_hda_intel snd_hda_codec snd_hwdep kvm snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event hid_generic snd_seq cdc_acm snd_timer snd_seq_device mei tpm_tis snd mac_hid serio_raw soundcore lpc_ich snd_page_alloc microcode coretemp usbhid hid nouveau usb_storage ttm drm_kms_helper drm floppy e1000e i2c_algo_bit mxm_wmi video wmi
[ 716.486128] Pid: 3994, comm: ubimkvol Tainted: G O 3.8.0-rc3+ #3 LENOVO 6239AS8/LENOVO
[ 716.486136] EIP: 0060:[<fc1936e7>] EFLAGS: 00010246 CPU: 0
[ 716.486144] EIP is at ubi_change_vtbl_record+0x87/0x1c0 [ubi]
[ 716.486151] EAX: 000000ac EBX: eb5ea000 ECX: 0000002b EDX: 00000000
[ 716.486157] ESI: eb4d1d74 EDI: fc132158 EBP: eb4d1d40 ESP: eb4d1d20
[ 716.486164] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 716.486170] CR0: 8005003b CR2: fc132158 CR3: 27542000 CR4: 000407f0
[ 716.486176] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 716.486183] DR6: ffff0ff0 DR7: 00000400
[ 716.486188] Process ubimkvol (pid: 3994, ti=eb4d0000 task=ec01d9b0 task.ti=eb4d0000)
[ 716.486195] Stack:
[ 716.486199] e755f000 eb4d1d2c c11cad11 eb4d1d34 eb543c00 eb5ea000 00000000 eb4d1e20
[ 716.486215] eb4d1e30 fc195412 e755f000 fc1adf01 eb5ea26c 00000002 0000009e eb5ea480
[ 716.486232] 00000002 e755f22c e755f2ac e755f000 eb4d1d74 2a000000 01000000 00000000
[ 716.486248] Call Trace:
[ 716.486257] [<c11cad11>] ? sysfs_create_file+0x21/0x30
[ 716.486266] [<fc195412>] ubi_create_volume+0x4b2/0x790 [ubi]
[ 716.486277] [<fc19967a>] ubi_cdev_ioctl+0x5da/0xac0 [ubi]
[ 716.486285] [<c117202a>] ? link_path_walk+0x5a/0x7d0
[ 716.486294] [<fc1990a0>] ? vol_cdev_ioctl+0x440/0x440 [ubi]
[ 716.486842] [<c1177e12>] do_vfs_ioctl+0x82/0x5b0
[ 716.487703] [<c1171ced>] ? final_putname+0x1d/0x40
[ 716.488564] [<c1171ced>] ? final_putname+0x1d/0x40
[ 716.489422] [<c1171ced>] ? final_putname+0x1d/0x40
[ 716.489891] [<c1171eb4>] ? putname+0x24/0x40
[ 716.489891] [<c1167239>] ? do_sys_open+0x169/0x1d0
[ 716.489891] [<c11783b0>] sys_ioctl+0x70/0x80
[ 716.489891] [<c16205cd>] sysenter_do_call+0x12/0x38
[ 716.489891] Code: ac 00 00 00 03 bb c8 04 00 00 f7 c7 01 00 00 00 0f 85 ee 00 00 00 f7 c7 02 00 00 00 0f 85 ca 00 00 00 89 c1 31 d2 c1 e9 02 a8 02 <f3> a5 74 0b 0f b7 16 66 89 17 ba 02 00 00 00 a8 01 74 07 0f b6
[ 716.489891] EIP: [<fc1936e7>] ubi_change_vtbl_record+0x87/0x1c0 [ubi] SS:ESP 0068:eb4d1d20
[ 716.489891] CR2: 00000000fc132158
[ 716.516453] ---[ end trace 473b15a7780e19ea ]---
It seems that the kernel wanted to access a wrong page. Now,
The Oops code 0002 tells me that it occurred while trying to read something in user-mode.
The Instruction Pointer is at ubi_change_vtbl_record, which means the offending instruction is located in this function.
I can deduce the path that lead to the faulting function from the
call trace (an ioctl launched from process ubimkvol)
From there, Is the "stack" a dump of the raw stack of the task ? I can see that some values mentioned are also function addresses found in the call trace. Then, I got fancy looking values like EAX, EBX ... DR7. I think they are CPU registers but still, I don't know what they really are.
Finally, the following line gets me lost :
[ 716.485986] *pdpt = 00000000019e6001 *pde = 000000002c558067 *pte = 0000000000000000
What are pdpt, pde and pte ? I feel they are information about the page fault but I could not retrieve further information after some googling around.
Yes, EAX, etc. are 32-bit x86 processor registers. pdpt (page directory pointer table), pde (page directory entry), and pte (page table entry) are all paging structures.
IP (also EIP for 32-bit or RIP for 64-bit processors) is the instruction pointer at the time of the Oops.
The stack is the raw stack for this processor. Each processor will have its own stack. Note that on this architecture the stack grows down (addresses start with 0xfxxxxxx).
Correct me if I am wrong but,
OOPS 0002 means no page found when writing in kernel mode:
bit 0 == 0 means no page found, 1 means a protection fault
bit 1 == 0 means read, 1 means write
bit 2 == 0 means kernel, 1 means user-mode
Hopefully I've been struggling with this long enough to ask a question here...
I can't connect ATmega1280 with a cheaper version of JTAGICE mkII like this one. I'm trying to first set fuses for JTAG debugging. I've already done udev config as described here (and here).
EDIT: I'm able to connect my hardware on my computer via my JTAGICE in Windows using AVRStudio, so I'm already positive the wiring is ok.
All I'm getting is:
$ avrdude -P usb -c jtagmkii -p m1280 -U hfuse:w:0x1a:m
avrdude: jtagmkII_program_enable(): bad response to enter progmode command: RSP_ILLEGAL_JTAG_ID
avrdude: jtagmkII_program_enable(): bad response to enter progmode command: RSP_ILLEGAL_JTAG_ID
avrdude: JTAGEN fuse disabled?
avrdude: initialization failed, rc=-1
Double check connections and try again, or use -F to override
this check.
avrdude done. Thank you.
I've read this can help solving JTAGICE problems. Sadly, I don't understand how it could help, but maybe you do:
$ avarice --part atmega1280 --mkII --jtag usb --read-fuses --read-lockbits --debug
AVaRICE version 2.10, Feb 8 2010 07:22:18
Defaulting JTAG bitrate to 250 kHz.
Found JTAG ICE, serno: 070000004693
JTAG config starting.
Attempting synchronisation at bitrate 19200
command[0x01, 1]: 01
recv: 0x1b
recv: 0x00
recv: 0x00
recv: 0x1c
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 28 bytes
read: 86 01 ff 26 04 00 ff 26 04 01 07 00 00 00 46 93 4a 54 41 47 49 43 45 6d 6b 49 49 00
recv: 0xb9
recv: 0xfe
CRC OK
Got message seqno 0 (command_sequence == 0)
response: 86 01 FF 26 04 00 FF 26 04 01 07 00 00 00 46 93 4A 54 41 47 49 43 45 6D 6B 49 49 00
Found a device: JTAGICEmkII
Serial number: 07:00:00:00:46:93
JTAG ICE mkII sign-on message:
Communications protocol version: 1
M_MCU:
boot-loader FW version: 255
firmware version: 4.38
hardware version: 0
S_MCU:
boot-loader FW version: 255
firmware version: 4.38
hardware version: 1
command[0x02, 1]: 02 03 01
recv: 0x1b
recv: 0x01
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: 80
recv: 0xcd
recv: 0x83
CRC OK
Got message seqno 1 (command_sequence == 1)
response: 80
command[0x0a, 1]: 0A 01
recv: 0x1b
recv: 0x02
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: 80
recv: 0x1d
recv: 0x09
CRC OK
Got message seqno 2 (command_sequence == 2)
response: 80
Automatic device detection:
command[0x03, 1]: 03 0E
recv: 0x1b
recv: 0xff
recv: 0xff
recv: 0x06
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 6 bytes
read: e0 fe ff 00 00 00
recv: 0x82
recv: 0x16
CRC OK
Got message seqno 65535 (command_sequence == 3)
got asynchronous event: 0xe0
recv: 0x1b
recv: 0x03
recv: 0x00
recv: 0x05
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 5 bytes
read: 81 ff ff ff ff
recv: 0x24
recv: 0x19
CRC OK
Got message seqno 3 (command_sequence == 3)
response: 81 FF FF FF FF
JTAG id = 0xFFFFFFFF : Ver = 0xf : Device = 0xffff : Manuf = 0x7ff
Reported JTAG device ID: 0xFFFF
Looking for device: atmega1280
Configured for device ID: 0x9703 atmega1280 -- FORCED with atmega1280
command[0x0c, 1]: 0C FF FF FF FF FF 3D B9 F8 00 00 00 00 00 00 00 00 FF FF 1F E0 FF 1D A9 F8 00 00 00 00 00 00 00 00 73 FF 3F FF F7 3F F7 3F F7 3F 5F 3F 37 37 36 00 00 00 00 00 FF 0F 00 00 F7 3F 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 73 FF 3F F8 F7 3F F7 3F F7 3F 5F 2F 36 36 36 00 00 00 00 00 FF 0F 00 00 F7 3F 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 57 3B 00 01 08 00 FE 00 00 36 01 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3E 3D 00 02 00 00 00 00 00 01 00 00 02 00 00 00 3C 1F 00
recv: 0x1b
recv: 0x04
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: 80
recv: 0xac
recv: 0x14
CRC OK
Got message seqno 4 (command_sequence == 4)
response: 80
JTAG config complete.
command[0x14, 1]: 14
recv: 0x1b
recv: 0x05
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xd0
recv: 0x29
CRC OK
Got message seqno 5 (command_sequence == 5)
response: A9
command[0x14, 2]: 14
recv: 0x1b
recv: 0x06
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0x00
recv: 0xa3
CRC OK
Got message seqno 6 (command_sequence == 6)
response: A9
command[0x14, 3]: 14
recv: 0x1b
recv: 0x07
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xbf
recv: 0x22
CRC OK
Got message seqno 7 (command_sequence == 7)
response: A9
command[0x14, 4]: 14
recv: 0x1b
recv: 0x08
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0x0d
recv: 0x93
CRC OK
Got message seqno 8 (command_sequence == 8)
response: A9
command[0x14, 5]: 14
recv: 0x1b
recv: 0x09
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xb2
recv: 0x12
CRC OK
Got message seqno 9 (command_sequence == 9)
response: A9
command[0x14, 6]: 14
recv: 0x1b
recv: 0x0a
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0x62
recv: 0x98
CRC OK
Got message seqno 10 (command_sequence == 10)
response: A9
command[0x14, 7]: 14
recv: 0x1b
recv: 0x0b
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xdd
recv: 0x19
CRC OK
Got message seqno 11 (command_sequence == 11)
response: A9
command[0x14, 8]: 14
recv: 0x1b
recv: 0x0c
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xd3
recv: 0x85
CRC OK
Got message seqno 12 (command_sequence == 12)
response: A9
command[0x14, 9]: 14
recv: 0x1b
recv: 0x0d
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0x6c
recv: 0x04
CRC OK
Got message seqno 13 (command_sequence == 13)
response: A9
command[0x14, 10]: 14
recv: 0x1b
recv: 0x0e
recv: 0x00
recv: 0x01
recv: 0x00
recv: 0x00
recv: 0x00
recv: 0x0e
sDATA: reading 1 bytes
read: a9
recv: 0xbc
recv: 0x8e
CRC OK
Got message seqno 14 (command_sequence == 14)
response: A9
JTAG ICE: Cannot synchronise
The device is being recognized on my system
$ lsusb
Bus 002 Device 012: ID 03eb:2103 Atmel Corp. JTAG ICE mkII
It's also not behind any USB hub. I triple checked all connections. Any clues?
avrice is connecting to the unit fine, but the unit doesn't seem to be communicating with the target device. Are you sure it is connected and powered on?
This document may help you decipher what the jtag unit is responding.