I done a memory dump with elf format using Virtualbox manager.
VBoxManage debugvm "image_name" dumpguestcore --filename test.elf
It worked well. Then I try to analyze the dump with volatility.
The imageinfo worked well and get the result.
volatility-2.2.standalone.exe -f test.elf imageinfo
Volatile Systems Volatility Framework 2.2
Determining profile based on KDBG search...
Suggested Profile(s) : WinXPSP2x86, WinXPSP3x86 (Instantiated with WinXPSP2x86)
AS Layer1 : FileAddressSpace (C:\work\volatility\test.elf)
PAE type : No PAE
DTB : 0x2f3000L
KDBG : 0x5461d0
Number of Processors : 0
Image Type (Service Pack) : -
KUSER_SHARED_DATA : 0xffdf0000L
It is failed When I tried to using pslist.
volatility-2.2.standalone.exe -f test.elf --profile=WinXPSP3x86 pslist
Volatile Systems Volatility Framework 2.2
No suitable address space mapping found
Tried to open image as:
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
AMD64PagedMemory: No base Address Space
JKIA32PagedMemory: No base Address Space
JKIA32PagedMemoryPae: No base Address Space
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
LimeAddressSpace: Invalid Lime header signature
WindowsHiberFileSpace32: No xpress signature found
WindowsCrashDumpSpace64: Header signature invalid
WindowsCrashDumpSpace32: Header signature invalid
AMD64PagedMemory: Incompatible profile WinXPSP3x86 selected
JKIA32PagedMemory: Failed valid Address Space check
JKIA32PagedMemoryPae: Failed valid Address Space check
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
FileAddressSpace: Must be first Address Space
Could anyone help to look at the issue why Volatility could not find "suitable address space mapping found" ???
Great thanks!!
Solved, virtualbox memory dump used ELF64 format, but volatility 2.2 didn't support it. The plugin here http://wiki.yobi.be/wiki/RAM_analysis could be used to support ELF64 format.
Related
I am trying to test a IO kit sample app that implements a filter scheme. See section 10.7.10 here for context. As the source there is hard to cut and paste (and when I tried I got errors), I am using the source from here which builds.
In the last figure on that page (1038) it has a series of steps for creating and mounting a new HFS volume. The idea is to see that the KEXT is working and doing encryption. I have built the sample KEXT and tested with it, but I always get "No mountable file systems" error when I make the "open" call. I have verified the KEXT is properly loaded.
If I remove the "-partitionType" I am able to manipulate the volume normally (though this doesn't involve the KEXT).
Can someone please suggest what might be going wrong here?
Part of my confusion in figure 1038's script is the "detach" command that is run before the open. It fails for me, but I am not sure if that is what is causing the "open" after it to fail, or is unrelated.
$ sudo kextload /System/Library/Extensions/IOKitDriverTestApp.kext/
$ hdiutil create -size 32m -partitionType osxbook_HFS /tmp/crypto.dmg
created: /tmp/crypto.dmg
$ hdiutil attach -nomount /tmp/crypto.dmg
/dev/disk2 Apple_partition_scheme
/dev/disk2s1 Apple_partition_map
/dev/disk2s2 osxbook_HFS
$ diskutil list
/dev/disk0 (internal, physical):
[cut]
/dev/disk1 (synthesized):
[cut]
/dev/disk2 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme +33.6 MB disk2
1: Apple_partition_map 32.3 KB disk2s1
2: osxbook_HFS 33.5 MB disk2s2
$ newfs_hfs -v Crypto /dev/rdisk2s2
Initialized /dev/rdisk2s2 as a 32 MB case-insensitive HFS Plus volume
$ hdiutil detach /dev/disk2
hdiutil: detach failed - No such file or directory
$ open /tmp/crypto.dmg
==> Get popup window with error "crypto.dmg" "No mountable file systems"
UPDATE: the question originally targeted the case when the KEXT was not loaded. Since I got that to work I changed it to reflect the case with the KEXT, which is more important anyway.
UPDATE2: It seems that eventually the kext is getting unloaded (possibly crashing). Sometimes it seems around the time of the "open", sometimes after. I tried adding statements like IOLog("CSD: init"), but none of them show up in the system log or console log. Looking for how else I can triage this (without needing a separate machine).
UPDATE3: Below is the output of when I try to start the KEXT. There are a bunch of warnings but since I have the signing check disabled I think it is still loading. Also I see it loaded via kextstat.
The other strange thing is I don't see any signs of this KEXT in the output of "ioreg", nor do I see any of the print statements from IOLog().
My-MacBook-Pro:Extensions myuser$ sudo kextutil -v /System/Library/Extensions/IOKitDriverTestApp.kext/
Defaulting to kernel file '/System/Library/Kernels/kernel'
Kext with invalid signatured (-67050) allowed: <OSKext 0x7fa53703ec70 [0x7fff8fe6faf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
Code Signing Failure: code signature is invalid
Warnings:
Personality CFBundleIdentifier differs from containing kext's (not necessarily a mistake, but rarely done):
SimpleCryptoDisk
Personality CFBundleIdentifier names a kext that can't be found:
'SimpleCryptoDisk' -> 'com.osxbook.driver.SimpleCryptoDisk'
Warnings:
Personality CFBundleIdentifier differs from containing kext's (not necessarily a mistake, but rarely done):
SimpleCryptoDisk
Personality CFBundleIdentifier names a kext that can't be found:
'SimpleCryptoDisk' -> 'com.osxbook.driver.SimpleCryptoDisk'
/System/Library/Extensions/IOKitDriverTestApp.kext appears to be loadable (not including linkage for on-disk libraries).
Loading /System/Library/Extensions/IOKitDriverTestApp.kext.
/System/Library/Extensions/IOKitDriverTestApp.kext successfully loaded (or already loaded).
Invalid signature -67050 for kext <OSKext 0x7fa53703ec70 [0x7fff8fe6faf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
UPDATE4: I adjusted the bundle ID in the plist so it matches, and now it is not giving the "names a kext that can't be found" error:
My-MacBook-Pro:Debug myuser$ sudo kextutil -v /System/Library/Extensions/IOKitDriverTestApp.kext/
Password:
Defaulting to kernel file '/System/Library/Kernels/kernel'
Kext with invalid signatured (-67050) allowed: <OSKext 0x7f8be663ecf0 [0x7fff866bcaf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
Code Signing Failure: code signature is invalid
/System/Library/Extensions/IOKitDriverTestApp.kext appears to be loadable (not including linkage for on-disk libraries).
Loading /System/Library/Extensions/IOKitDriverTestApp.kext.
/System/Library/Extensions/IOKitDriverTestApp.kext successfully loaded (or already loaded).
Invalid signature -67050 for kext <OSKext 0x7f8be663ecf0 [0x7fff866bcaf0]> { URL = "file:///System/Library/Extensions/IOKitDriverTestApp.kext/", ID = "com.osxbook.driver.IOKitDriverTestApp" }
UPDATE5:
Now my OS consistently crashes every time I load the KEXT and then execute "hdiutil attach -nomount /tmp/crypto.dmg" (after creating /tmp/crypto.dmg).
It seems clear my KEXT is running now, but for some reason I still don't see the IOLog() statements anywhere. Will have to look through the system logs and see if there is any indication why it is crashing.
UPDATE6: I have been able to get a symbolicated kernel panic, but it is pointing to a different KEXT (which I don't think I have the source too). Below is the relevant part of it.
I tried using another version of the same test KEXT (here) without any changes (I have removed the IOLog() statements for now since they weren't showing up).
I see a note this was tested on Mac OS 10.6, but has anyone gotten this sample to work on 10.13.16 (High Sierra)?
...
0xffffff800c94f540 : 0xffffff800cb8776f mach_kernel : _kernel_trap + 0x70f
0xffffff800c94f6b0 : 0xffffff800ca1e1e0 mach_kernel : _return_from_trap + 0xe0
0xffffff800c94f6d0 : 0xffffff7f8d527251 com.apple.iokit.IOStorageFamily : __ZN9IOStorage8completeEP19IOStorageCompletioniy + 0x27
0xffffff81f7a4b980 : 0xffffff7f8d51a0c4 com.apple.iokit.IOStorageFamily : __ZN20IOBlockStorageDriver24prepareRequestCompletionEPvS0_iy + 0xc2
0xffffff81f7a4b9e0 : 0xffffff7f903afa7a com.apple.driver.DiskImages : __ZN13IOHDIXCommand8completeEiy + 0x26
0xffffff81f7a4ba00 : 0xffffff7f903adbfe com.apple.driver.DiskImages : __ZN22IOHDIXHDDriveOutKernel12processReplyEPK13HDIReplyOOL64P18IOMemoryDescriptor + 0x2be
0xffffff81f7a4ba60 : 0xffffff7f903aee9b com.apple.driver.DiskImages : __ZN32IOHDIXHDDriveOutKernelUserClient14processReply64EPK13HDIReplyOOL64 + 0xd3
0xffffff81f7a4bac0 : 0xffffff800d0c3959 mach_kernel : _shim_io_connect_method_structureI_structureO + 0x1c9
0xffffff81f7a4bb20 : 0xffffff800d0c1ae0 mach_kernel : __ZN12IOUserClient14externalMethodEjP25IOExternalMethodArgumentsP24IOExternalMethodDispatchP8OSObjectPv + 0x340
0xffffff81f7a4bb70 : 0xffffff800d0ca567 mach_kernel : _is_io_connect_method + 0x217
...
UPDATE7: I gave up on trying to use the SimpleCryptoDisk.cpp test app and started from scratch, adding a little bit of code at a time. I've had some success (the driver starts and I can see logging), though I don't really have anything working yet.
If I get stuck on this I may post another question, but I'll go ahead and mark pmdj's answer as correct since he answered my original question.
If anyone knows of a modified version of SimpleCryptoDisk that works on High Sierra I'd still like to know, however.
To answer your original questions:
[Why "No mountable file systems" when not loading the kext and using partition type "osxbook_HFS"]
As you can see, the Apple Partition Scheme uses arbitrary text strings for identifying partition type. macOS uses this string verbatim as the "content hint" which selects which file system type to attempt to automount on that volume. The HFS+ file system is only looking for Apple_HFS (and Apple_HFSX for case-sensitive)
The idea behind the kext is that it itself matches the newly defined content hint and publishes a new storage nub object which in turn has the HFS+ content hint for the filesystem to pick up.
Why hdiutil detach was failing
$ hdiutil detach /dev/disk2
hdiutil: detach failed - No such file or directory
On current versions of macOS, you need to rewrite the command as either:
$ hdiutil detach disk2
or
$ hdiutil detach /Volumes/VOLUMENAME
I can't immediately answer your current question of why it's not working with the kext, but I can suggest you look at whether your kext is behaving as expected with regard to the IO Registry. Check with IORegistryExplorer or ioreg whether it's matching the volume with the special partition type, and whether it's creating and registering the virtual (decrypted) volume with the Apple_HFS content hint. Add the ioreg output (ioreg -w 0 -lirc YOUR_DRIVERS_CLASSNAME) to your question if unsure.
I am working on microzed 7010 board, I have manualy compiled kernel, u-boot, fsbl, and .bit (vivado). Board is booting well with all setup (without using petalinux). But i have noticed that kernel is not Uncompressing kernel... with zImage nor uImage. whereas i can see bootlogs with that of petalinux's images.
INPUT :
1 . zImage env is
zImage=tftpboot 0x3000000 zImage && tftpboot 0x2A00000 system.dtb && bootz 0x3000000 - 0x2A00000
2 . Boot log is =>
Zynq> run zImage
[2017-10-25 15:57:11
ethernet#e000b000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
[2017-10-25 15:57:15
Zynq> run zImage
[2017-10-25 15:57:22
Using ethernet#e000b000 device
TFTP from server 172.16.9.187; our IP address is 172.16.9.25
Filename 'zImage'.
Load address: 0x3000000
Loading:#####################################################################################################################################################################################################################################
3.9 MiB/s
done
Bytes transferred = 3913840 (3bb870 hex)
Using ethernet#e000b000 device
TFTP from server 172.16.9.187; our IP address is 172.16.9.25
Filename 'system.dtb'.
Load address: 0x2a00000
Loading: #
3.3 MiB/s
done
Bytes transferred = 13644 (354c hex)
Kernel image # 0x3000000 [ 0x000000 - 0x3bb870 ]
## Flattened Device Tree blob at 02a00000
Booting using the fdt blob at 0x2a00000
Loading Device Tree to 1fff9000, end 1ffff54b ... OK
Starting kernel ...
Booting Linux on physical CPU 0x0
Linux version 4.6.0-xilinx-00003-g2762bc9 (pritam#pritam) (gcc version 5.2.1 20151005 (Linaro GCC 5.2-2015.11-2) ) #3 SMP PREEMPT Wed Oct 25 10:28:387
[2017-10-25 15:57:24
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
3 . In u-boot I have given bootz support
4 . uImage is formed by
mkimage -A arm -O linux -T kernel -C none -a 0x10000000 -e 0x10000000 -n "Linux kernel" -d arch/arm/boot/zImage uImage
What causing it not to uncompress kernel ? Is u-boot compressing the kernel and booting ?
Any help will be appreciated.
Thanks and regards,
Pritam
Board is booting well with all setup (without using petalinux). But i have noticed that kernel is not Uncompressing kernel... with zImage nor uImage.
Some kernels can perform this step silently. The fact that you load a zImage (or a zImage in a uImage), and then see the Linux kernel version line means that the kernel has been uncompressed successfully and is executing
What causing it not to uncompress kernel ?
Your presumption that the kernel is not being uncompressed is simply wrong.
The zImage or uImage files that you are using are compressed kernel images. Since the kernel is actually executing (as evidenced by the boot log that you posted), the kernel must have silently uncompressed and proceeded to boot.
If the kernel did not uncompress (as you assert), then the kernel could not boot successfully (as you reported).
Is u-boot compressing the kernel and booting ?
No, U-Boot is not involved in uncompressing a zImage file.
A zImage is a self-extracting compressed Image file.
Depending on how the kernel was configured, the uncompression of the zImage file can be silent or verbose.
I have cloned the source codes from petalinux downloads. The boot logs, I got from images built by petalinux, shows Uncompressing kernel .... message. " Starting kernel ... Uncompressing Linux... done, booting the kernel. Booting Linux on physical CPU 0x0 Linux version 4.6.0-xilinx (pritam#pritam) (gcc version 5.2.1" So i am expecting it to show "uncompressing kernel " message
Using the same source code is only one requisite for building a duplicate kernel.
You also need to build with the same configuration.
The silent or verbose uncompression is selected by kernel configuration.
From arch/arm/Kconfig.debug:
menu "Kernel hacking"
...
config DEBUG_LL
bool "Kernel low-level debugging functions (read help!)"
depends on DEBUG_KERNEL
help
Say Y here to include definitions of printascii, printch, printhex
in the kernel. This is helpful if you are debugging code that
executes before the console is initialized.
Note that selecting this option will limit the kernel to a single
UART definition, as specified below. Attempting to boot the kernel
image on a different platform *will not work*, so this option should
not be enabled for kernels that are intended to be portable.
...
prompt "Kernel low-level debugging port"
...
config DEBUG_ZYNQ_UART0
bool "Kernel low-level debugging on Xilinx Zynq using UART0"
depends on ARCH_ZYNQ
help
Say Y here if you want the debug print routines to direct
their output to UART0 on the Zynq platform.
config DEBUG_ZYNQ_UART1
bool "Kernel low-level debugging on Xilinx Zynq using UART1"
depends on ARCH_ZYNQ
help
Say Y here if you want the debug print routines to direct
their output to UART1 on the Zynq platform.
If you expect a verbose uncompression, then you need to select CONFIG_DEBUG_KERNEL, CONFIG_DEBUG_LL, and an appropriate serial port.
ADDENDUM (response to comment)
Which one is better way to compress the kernel. zImage or gzipping arch/arm/Image ... are they same ... ???
What metric are you going to use to measure "better"?
In the end, the result is the same: a compressed kernel Image.
How many of these image files to you have to save?
How crucial is saving space and load times (if any) versus self-extraction?
In mkimage if I specified -C "gzip", I noticed that at time of loading image in ram, u-boot uncompresses the gzipped image ... !!!
As I already commented, that is a mislabeling of a zImage file, and therefore wrong. The zImage is self-extracting, and should be labeled as "uncompressed" so that U-Boot does not try to perform uncompression.
Interestingly I cannot duplicate your claim at the shell prompt. A zImage renamed to zImage.gz cannot be gunzip'd:
gzip: zImage.gz: not in gzip format.
More importantly, I cannot replicate the results that you claim you got.
=> bootm 20080000 - 22000000
## Booting kernel from Legacy Image at 20080000 ...
Image Name: Linux kernel
Image Type: ARM Linux Kernel Image (gzip compressed)
Data Size: 5774280 Bytes = 5.5 MiB
Load Address: 20008000
Entry Point: 20008000
Verifying Checksum ... OK
## Flattened Device Tree blob at 22000000
Booting using the fdt blob at 0x22000000
Uncompressing Kernel Image ... Error: Bad gzipped data
gzip compressed: uncompress error -1
Must RESET board to recover
resetting ...
Does u-boot contains external decompresser ... ???
If you had bothered to read the link I had provided previously, the answer would be obvious.
U-Boot can be configured to have gzip, bzip2, lzma, and lzo compression algorithms.
However the Linux kernel supports compressing the Image file using gzip, lzo, lzma, xz, and lz4 compression algorithms, that is, a wider selection of size versus time tradeoffs.
Which one better compression method whether in u-boot or kernel (zImage).
Again, what metric are you going to use to measure "better"?
Of course Wolfgang Denk has his opinion.
Plz explain me with actual example (If any h/w requirement) ... !!!
Example of what?
I've already answered your question, and explained how you can configure your kernel to get the expected message.
The issue was with specifying the compression type "-C" as none.
mkimage -A arm -O linux -T kernel **-C none** -a 0x10000000 -e 0x10000000 -n "Linux kernel" -d arch/arm/boot/zImage uImage
So I tried with vmlinux. and converted it to gzip
mkimage -A arm -O linux -T kernel **-C gzip** -a 0x10008000 -e 0x10008000 -n 'Test' -d vmlinux.bin.gz uImage.
So I have noticed size of both images .
first one is of vmlinux and another of zImage
So please correct me if i am misunderstood .
I have a few Windows 7 machines that I am not able to read their memory dumps. I found something that I suspect may be related, but am not positive:
https://twitter.com/aionescu/status/634028737458114560
I also found this: http://support.microsoft.com/kb/2528507
However, the scenario message regarding wow64exts given in the doc is not seen in any of my dumps. I also cannot apply that hotfix at this time to test it. So I'm just looking for some more information or opinions.
I'm able to open any other OS dump as well as my own system's Windows 7 dump, but there are 2 other machines that run Win 7 and it's telling me I have the wrong kernel symbols.
I have tried clearing out my symbol cache, reinstalled the Windows SDK, and also tried to open the dumps on two other machines with the same result. If it matters, the crash is manually created using the scroll lock method.
Symbol path: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols;
Seeing these errors: followed by "Type referenced: nt!_KPRCB"
Does anyone know about the issue mentioned by Alex in the twitter link and if it's possibly related to what I'm seeing?
Update 2015-10-22:
With the Microsoft patch day (2015-10-13) and KB3088195, symbols are available again.
However, symbols for the broken version have not been provided, so below may still be useful.
Microsoft has already published "good" symbols for ntdll in the past, containing type information like _TEB or _KPRCB. Starting from mid of July 2015, Microsoft has still published symbols for ntdll, but not containing that information.
So it depends on the version of ntdll whether you get type information or not. Old dumps referencing an old version of ntdll will download old PDBs containing type information while new dumps reference new versions of ntdll and WinDbg (or any other debugger) downloads PDBs without type information.
Could Microsoft remove type information of "good" symbols retroactively, thus making them "bad"?
Yes. As described in this answer, there is a tool to remove type information from existing PDBs. Doing that and replacing the PDB would result in such an effect.
Can Microsoft publish the "good" version of those PDBs which are currently "bad"?
That's hard to tell, since we don't know whether Microsoft has kept a copy of the "good" version so they could replace the "bad" version on the symbol server with the "good" one. Rebuilding ntdll from the same source code and thus creating new PDBs sounds possible, but the PDB gets a new time stamp and checksum. This can potentially be corrected manually, especially be Microsoft, since they should have the knowledge about the PDB internal format, but IMHO it's unlikely they'll do that. Things may go wrong and MS will hardly have tests to guarantee the correctness of such a thing.
So what can I do?
IMHO you can do nothing to really correct the situation.
You could assume that the types in ntdll have not changed so much. This would allow you to take an older version of wntdll.pdb and the new version of ntdll.dll and apply ChkMatch -m to it. This will copy the timestamp and checksum from the DLL to the PDB. After you did that (in an empty folder), replace the existing wntdll.pdb in your symbols directory with the hacked one.
WinDbg walkthrough (with output shortened to relevant things). I am using the latest version of wntdll.pdb I could find on my PC.
WARNING: doing the following may fix the type information but will likely destroy the correctness of the callstacks. Since any changes in the implementation (which are likely for security fixes) will change the method offsets.
0:005> dt nt!_PEB
*************************************************************************
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
...
*** Type referenced: nt!_PEB ***
*** ***
*************************************************************************
Symbol nt!_PEB not found.
0:005> lm m ntdll
start end module name
773f0000 77570000 ntdll (pdb symbols) e:\debug\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb
0:005> .shell cmd /c copy C:\Windows\SysWOW64\ntdll.dll e:\debug\temp\ntdllhack\ntdll.dll
1 file(s) copied.
0:005> .shell cmd /c copy "E:\Windows SDk\8.0\Debuggers\x86\sym\wntdll.pdb\B081677DFC724CC4AC53992627BEEA242\wntdll.pdb" e:\debug\temp\ntdllhack\wntdll.pdb
1 file(s) copied.
0:005> .shell cmd /c E:\debug\temp\ntdllhack\chkmatch.exe -m E:\debug\temp\ntdllhack\ntdll.dll E:\debug\temp\ntdllhack\wntdll.pdb
...
Executable: E:\debug\temp\ntdllhack\ntdll.dll
Debug info file: E:\debug\temp\ntdllhack\wntdll.pdb
Executable:
TimeDateStamp: 55a69e20
Debug info: 2 ( CodeView )
TimeStamp: 55a68c18 Characteristics: 0 MajorVer: 0 MinorVer: 0
Size: 35 RVA: 000e63e0 FileOffset: 000d67e0
CodeView format: RSDS
Signature: {fa9c48f9-c11d-4e08-94b8-970decd92c97} Age: 2
PdbFile: wntdll.pdb
Debug info: 10 ( Unknown )
TimeStamp: 55a68c18 Characteristics: 0 MajorVer: 565 MinorVer: 6526
Size: 4 RVA: 000e63dc FileOffset: 000d67dc
Debug information file:
Format: PDB 7.00
Signature: {b081677d-fc72-4cc4-ac53-992627beea24} Age: 4
Writing to the debug information file...
Result: Success.
0:005> .shell cmd /c copy E:\debug\temp\ntdllhack\wntdll.pdb E:\debug\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb
1 file(s) copied.
0:005> .reload
Reloading current modules
.............................
0:005> dt nt!_PEB
ntdll!_PEB
+0x000 InheritedAddressSpace : UChar
+0x001 ReadImageFileExecOptions : UChar
...
0:005> !heap -s
LFH Key : 0x219ab08b
Termination on corruption : DISABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
Virtual block: 00920000 - 00920000 (size 00000000)
Virtual block: 02c60000 - 02c60000 (size 00000000)
Virtual block: 02e10000 - 02e10000 (size 00000000)
...
Note: using ChkMatch like this has the benefit that you do not need to turn on .symopt- 100, since that option would affect all PDB files, and you would not find potential other symbol issues. If you don't mind using .symopt, you could simply copy an old wntdll.PDB over the new one.
The issue is now fixed according to Microsoft and Microsoft told me that you should clear your symbol cache to get the new PDBs, otherwise Windbg would use the old Symbols which miss the information.
I am try to understand the build and booting process of Linux kernel for ARM. I took vanila linux from www.kernel.org and build it after run configuration for AT91SAM9260.
In message when we compile the kernel showed that:
==========================================
LD vmlinux
SORTEX vmlinux
SYSMAP System.map
OBJCOPY arch/arm/boot/Image
Kernel: arch/arm/boot/Image is ready
GZIP arch/arm/boot/compressed/piggy.gzip
AS arch/arm/boot/compressed/piggy.gzip.o
LD arch/arm/boot/compressed/vmlinux
OBJCOPY arch/arm/boot/zImage
Kernel: arch/arm/boot/zImage is ready
UIMAGE arch/arm/boot/uImage
Image Name: Linux-3.9.1+
Created: Sat Nov 23 18:15:58 2013
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 1635544 Bytes = 1597.21 kB = 1.56 MB
Load Address: 20008000
Entry Point: 20008000
Image arch/arm/boot/uImage is ready
==========================================
My questions are:
Image type is uncompressed, this means that we don't compress vmlinux to zImage ?
Load Address: 20008000: this is address of decompressed image = ZRELADDR defined in arch/arm/boot/Makefile ?
This address also is address of ../arm/kernel/head.o ?
It seems that we don't use address KERNEL_PHYS , this method is common way or just for AT91SAM family ?
Basically, our procedures to build and booting are:
a. building kernel steps: vmlinux -> uImage (skip to create zImage).
b. kernel booting steps: DataFlash/NAND --load-->uImage (# 0x22200000) ---decompress--> uncompressed image (# 0x20008000).
In this case, no zImage in booting process although in build message I saw zImage created. I'm wrong ?
4 . How about the address 0xC0008000 which I found in /arch/arm/kernel/vmlinux.lds at line:
. = 0xC0000000 + 0x00008000;
Do we use it ?. I confuse this address with the ZRELADDR.
Regards.
The uImage file has most probably been built using the zImage. It says uncompressed because the uImage itself is not compressed.
The load address can be used by the boot-loader to store data necessary for the early phases of the Linux kernel boot (such as the command line defined in the bootloader.)
You're right about the boot process. But when the zImage is used then the decompression is done by the kernel instead of the bootloader. See decompress_kernel()
The address 0xc0008000 is a virtual address. It maps to the physical address 0x20008000. Virtual addresses can be used only after Linux sets up the memory translation (MMU).
I'm compiling a custom kernel under Ubuntu and I'm running into the problem that my kernel doesn't seem to know where to look for firmware. Under Ubuntu 8.04, firmware is tied to kernel version the same way driver modules are. For example, kernel 2.6.24-24-generic stores its kernel modules in:
/lib/modules/2.6.24-24-generic
and its firmware in:
/lib/firmware/2.6.24-24-generic
When I compile the 2.6.24-24-generic Ubuntu kernel according the "Alternate Build Method: The Old-Fashioned Debian Way" I get the appropriate modules directory and all my devices work except those requiring firmware such as my Intel wireless card (ipw2200 module).
The kernel log shows for example that when ipw2200 tries to load the firmware the kernel subsystem controlling the loading of firmware is unable to locate it:
ipw2200: Detected Intel PRO/Wireless 2200BG Network Connection
ipw2200: ipw2200-bss.fw request_firmware failed: Reason -2
errno-base.h defines this as:
#define ENOENT 2 /* No such file or directory */
(The function returning ENOENT puts a minus in front of it.)
I tried creating a symlink in /lib/firmware where my kernel's name pointed to the 2.6.24-24-generic directory, however this resulted in the same error. This firmware is non-GPL, provided by Intel and packed by Ubuntu. I don't believe it has any actual tie to a particular kernel version. cmp shows that the versions in the various directories are identical.
So how does the kernel know where to look for firmware?
Update
I found this solution to the exact problem I'm having, however it no longer works as Ubuntu has eliminated /etc/hotplug.d and no longer stores its firmware in /usr/lib/hotplug/firmware.
Update2
Some more research turned up some more answers. Up until version 92 of udev, the program firmware_helper was the way firmware got loaded. Starting with udev 93 this program was replaced with a script named firmware.sh providing identical functionality as far as I can tell. Both of these hardcode the firmware path to /lib/firmware. Ubuntu still seems to be using the /lib/udev/firmware_helper binary.
The name of the firmware file is passed to firmware_helper in the environment variable $FIRMWARE which is concatenated to the path /lib/firmware and used to load the firmware.
The actual request to load the firmware is made by the driver (ipw2200 in my case) via the system call:
request_firmware(..., "ipw2200-bss.fw", ...);
Now somewhere in between the driver calling request_firmware and firmware_helper looking at the $FIRMWARE environment variable, the kernel package name is getting prepended to the firmware name.
So who's doing it?
From the kernel's perspective, see /usr/src/linux/Documentation/firmware_class/README:
kernel(driver): calls request_firmware(&fw_entry, $FIRMWARE, device)
userspace:
- /sys/class/firmware/xxx/{loading,data} appear.
- hotplug gets called with a firmware identifier in $FIRMWARE
and the usual hotplug environment.
- hotplug: echo 1 > /sys/class/firmware/xxx/loading
kernel: Discard any previous partial load.
userspace:
- hotplug: cat appropriate_firmware_image > \
/sys/class/firmware/xxx/data
kernel: grows a buffer in PAGE_SIZE increments to hold the image as it
comes in.
userspace:
- hotplug: echo 0 > /sys/class/firmware/xxx/loading
kernel: request_firmware() returns and the driver has the firmware
image in fw_entry->{data,size}. If something went wrong
request_firmware() returns non-zero and fw_entry is set to
NULL.
kernel(driver): Driver code calls release_firmware(fw_entry) releasing
the firmware image and any related resource.
The kernel doesn't actually load any firmware at all. It simply informs userspace, "I want a firmware by the name of xxx", and waits for userspace to pipe the firmware image back to the kernel.
Now, on Ubuntu 8.04,
$ grep firmware /etc/udev/rules.d/80-program.rules
# Load firmware on demand
SUBSYSTEM=="firmware", ACTION=="add", RUN+="firmware_helper"
so as you've discovered, udev is configured to run firmware_helper when the kernel asks for firmware.
$ apt-get source udev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Need to get 312kB of source archives.
Get:1 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (dsc) [716B]
Get:2 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (tar) [245kB]
Get:3 http://us.archive.ubuntu.com hardy-security/main udev 117-8ubuntu0.2 (diff) [65.7kB]
Fetched 312kB in 1s (223kB/s)
gpg: Signature made Tue 14 Apr 2009 05:31:34 PM EDT using DSA key ID 17063E6D
gpg: Can't check signature: public key not found
dpkg-source: extracting udev in udev-117
dpkg-source: unpacking udev_117.orig.tar.gz
dpkg-source: applying ./udev_117-8ubuntu0.2.diff.gz
$ cd udev-117/
$ cat debian/patches/80-extras-firmware.patch
If you read the source, you'll find that Ubuntu wrote a firmware_helper which is hard-coded to first look for /lib/modules/$(uname -r)/$FIRMWARE, then /lib/modules/$FIRMWARE, and no other locations. Translating it to sh, it does approximately this:
echo -n 1 > /sys/$DEVPATH/loading
cat /lib/firmware/$(uname -r)/$FIRMWARE > /sys/$DEVPATH/data \
|| cat /lib/firmware/$FIRMWARE > /sys/$DEVPATH/data
if [ $? = 0 ]; then
echo -n 1 > /sys/$DEVPATH/loading
echo -n -1 > /sys/$DEVPATH/loading
fi
which is exactly the format the kernel expects.
To make a long story short: Ubuntu's udev package has customizations that always look in /lib/firmware/$(uname -r) first. This policy is being handled in userspace.
Wow this is very useful information and it led me to the solution for my problem when making a custom USB kernel module for a device requiring firmware.
Basically, every Ubuntu brings a new rehash of hal,sysfs,devfs,udev,and so on...and things just change. In fact I read they stopped using hal.
So let's reverse engineer this yet again so it's pertinent to the latest [Ubuntu] systems.
On Ubuntu Lucid (the latest at time of writing), /lib/udev/rules.d/50-firmware.rules is used. This file calls the binary /lib/udev/firmware, where magic happens.
Listing: /lib/udev/rules.d/50-firmware.rules
# firmware-class requests, copies files into the kernel
SUBSYSTEM=="firmware", ACTION=="add", RUN+="firmware --firmware=$env{FIRMWARE} --devpath=$env{DEVPATH}"
The magic should be something along these lines (source: Linux Device Drivers, 3rd Ed., Ch. 14: The Linux Device Model):
echo 1 to loading
copy firmware to data
on failure, echo -1 to loading and halt firmware loading process
echo 0 to loading (signal the kernel)
then, a specific kernel module receives the data and pushes it to the device
If you look at Lucid's source page for udev, in udev-151/extras/firmware/firmware.c, the source for that firmware /lib/udev/firmware binary, that's exactly what goes on.
Excerpt: Lucid source, udev-151/extras/firmware/firmware.c
util_strscpyl(datapath, sizeof(datapath), udev_get_sys_path(udev), devpath, "/data", NULL);
if (!copy_firmware(udev, fwpath, datapath, statbuf.st_size)) {
err(udev, "error sending firmware '%s' to device\n", firmware);
set_loading(udev, loadpath, "-1");
rc = 4;
goto exit;
};
set_loading(udev, loadpath, "0");
Additionally, many devices use an Intel HEX format (textish files containing checksum and other stuff) (wiki it i have no reputation and no ability to link). The kernel program ihex2fw (called from Makefile in kernel_source/lib/firmware on .HEX files) converts these HEX files to an arbitrary-designed binary format that the Linux kernel then picks up with request_ihex_firmware, because they thought reading text files in the kernel was silly (it would slow things down).
On current Linux systems, this is handled via udev and the firmware.agent.
Linux 3.5.7 Gentoo, I have the same issue.
SOLVED:
emerge ipw2200-firmware
Then go to /usr/src/linux
make menucofig
on device driver, remove all wirless drivers don't needed, set Intell 2200 as module and recompile.
make
make modules_install
cp arch/x86/boot/bzImage /boot/kernel-yourdefault