A hard disk has 4 primary (MBR) partitions, all formatted as NTFS. Only one of them contains a bootable operating system (Windows XP, Windows Vista, Windows 7, Windows 8 or Windows 10). How does my bootloader program figure out which filesystem is bootable? Is it possible just by reading the boot sector (i.e. first 512 bytes) of the filesystem?
The active bit in the partition table has been lost.
Checking that byte 0 is 0xeb or 0xe9 and byte 510 is 0x55 and byte 511 is 0xAA is not enough, because even non-bootable NTFS filesystems created by the mkfs.ntfs tool on Linux pass this test, and the expected and required output for this case is non-bootable.
If my program is able to list the files in the root directory of the NTFS filesystem, which files or directories should I be looking for (NTLDR)?
If all my program has is the first 40960 bytes of the filesystem, can it still decide if the partition is bootable? (Preferably with as simple logic as possible.)
Is this correct: if files \BOOTMGR or \NTLDR exist on the NTFS filesystem, then it's (probably) bootable.
According to my best understanding, the simplest way to detect whether an NTFS filesystem contains a bootable Windows is checking that any of the files BOOTMGR or NTLDR exists in the root directory, because one of these files will be loaded by the boot code.
The NTFS boot sector (i.e. first 512 bytes of the filesystem) doesn't contain definitive information about bootability, because it can be exactly the same for bootable and nonbootable filesystems.
Some more info about Windows booting (with the role of the files BOOTMGR and NTLDR):
https://sites.google.com/site/h2obsession/ibm-pc-at/windows/boot-process/phase-5-ntldr-or-bootmgr
http://thestarman.pcministry.com/asm/mbr/BOOTMGR_Loader_Sectors.htm
It's also worth looking at the source code of os-prober. In os-probes/mounted/x86/20microsoft it's indeed looking for files BOOTMGR and NTLDR (both lowercase). It also has some additional checks, like for BOOTMGR it checks the file boot/bcd and for NTLDR it checks the file ntdetect.com and boot.ini.
Related
I think FS stands for filesystem, but I don't know what BLK stands for. Not only that, but what are the meanings behind the pci hierarchy parameters. i.e. When I see HD(1,MBR,0x0003B) what does "1","MBR", and what looks to be an address, stand for?
Here's the mapping table I'm looking at in UEFI shell:
Mapping table
FS0: Alias(s):HD21a0e0b:;BLK1:
PciRoot(0x0)/Pci(0x1D,0x0)/USB(0x0,0x0)/USB(0x4,0x0)/HD(1,MBR,0x0003B)
FS1: Alias(s):HD23a0a1:;BLK4:
PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0x0,0x0)/HD(1,MBR,0x00000000,0x3F)
BLK3: Alias(s):
PciRoot(0x0)/Pci(0x1F,0x2)/Sata(0x0,0x0,0x0)
BLK0: Alias(s):
PciRoot(0x0)/Pci(0x1D,0x0)/USB(0x0,0x0)/USB(0x4,0x0)
BLK2: Alias(s):
PciRoot(0x0)/Pci(0x1D,0x0)/USB(0x0,0x0)/USB(0x4,0x0)/HD(2,MBR,0x0003B)
I'm guessing BLK's are available ports and FS's are physical things that are plugged into those ports. It looks like once somethign is plugged into a BLK, it becomes an FS, but still retains its BLK value. i.g. FS0=BLK1
According to archwiki:
fsX means filesystem
blkX means block device or data storage device
MBR should mean Master Boot Record
HD should mean Hard Drive
1 might mean Primary, 2 Secondary Partition
That hex number after MBR could be the device signature or disk identifier. Or maybe an offset of that device to important information.
Links that might help further:
RHEL 5 Installation Guide EFI Shell Guide
Red Hat 7.1 Itanium EFI Shell Guide
HP Knowledge Base: "UEFI Shell 'fs' devices gone after restore from image backup"
OpenVMS: Firmware upgrades from a USB stick (on UEFI)
SourceForge EFI Shell Development Documentation
I'm trying to write a test pattern to every sector of a formatted USB drive. There is one logical drive (e.g. h:). This volume is FAT-formatted and contains data to be overwritten. Also, I want to overwrite the whole physical drive. The program is running with elevated user rights.
First I did the following:
// from the drive letter "h:" I get the physical disk number using
// IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS => "\\.\PhysicalDrive2"
hDevice = ::CreateFile( "\\.\PhysicalDrive2", GENERIC_READ|GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL );
// get the number of available sectors with IOCTL_DISK_GET_DRIVE_GEOMETRY_EX
// => ulNumberOfSectors
// now I try to write some sectors, e.g. 2 (I want to use a higher value):
WriteFile( hDevice, abBuffer, 2*512, &byteswritten, NULL );
The call to WriteFile fails with ERROR_ACCESS_DENIED.
If I write one sector, it works.
When I overwrite the first sector and plug the device out and in again, Windows wants to format it. In this situation my code with 2048 sectors at once works without ERROR_ACCESS_DENIED.
I also unmounted the volume as described in CodeProject: WriteFile on Physical Drives with Windows 7 but this didn't change anything. Obviously the volume is unmounted because it's no longer visible in Windows Explorer.
I want to write more than a single sector due to perfomance reasons. I'm also afraid that other problems in the field might occur because I don't fully understand ths problem.
Any suggestions?
I didn't have problems with different WriteFile() sizes, but I did solve the
WriteFile(): Access is denied <ERROR_ACCESS_DENIED/5> to
'\.\physicaldriveX
devices (usually USB HDD/SSD) in Windows 7 running as Administrator (elevated rights) as follows:
Computer Management -> Disk Management:
Volume (H: in your case) -> right-click -> Delete Volume
Disk (Disk 2 in your case) -> right-click -> Off-line
Disk (Disk 2 in your case) -> right-click -> On-line
After that, I'm able to write to '\.\physicaldriveX' with no problem.
I think the Win7 locks (unlike previous Windows releases) the physical device as long as there is any file system on the device to avoid consistency problems.
You cannot directly access sectors of a drive which are owned by a mounted filesystem.
See Changes to the file system and to the storage stack to restrict direct disk access and direct volume access
The documentation for FSCTL_DISMOUNT_VOLUME describes the following sequence for overwriting a filesystem:
Open a volume.
Lock the volume.
Format the volume.
Dismount the volume.
Unlock the volume.
Close the volume handle.
Your pattern-writing operation would be in step 3 instead of formatting.
Another method is to use clean to delete all the partitions (and ALL DATA) on the disk:
C:\> diskpart
Diskpart> list disk
Diskpart> select disk N (where N is your disk number)
Diskpart> clean
Diskpart> exit
I'm going to write and test a bootloader. In order to do this, I am planning to copy the bootloader onto a floppy image file and mount it in a VM.
However, I'm not sure where to put the bootloader's machine code. Does it just get dumped into the first few bytes of the file?
The boot sector of the floppy was the first sector. If you're talking about a raw floppy image (1440K), it should be the first 512 bytes of the image file.
From memory, this gets loaded by the BIOS into 7c00:0000 (real mode) and then jumps to that address.
The DOS boot floppies had a 3-byte JMP instruction there to jump over the Disk Parameter Block (DPB), which detailed the attributes of the disk. But, if you're in total control of the disk and your boot code, I don't think you need to follow that convention. I don't recall any BIOS' checking what was loaded for validity (though admittedly it was a long time ago).
its been a VERY long time but if i recall in DOS it was stored in the MBR. i believe its still the same today
http://en.wikipedia.org/wiki/Master_boot_record
I needed to recover the partition table I deleted accidentally. I used an application named TestDisk. Its simply mind blowing. I reads each cylinder from the disk. I've seen similar such applications which work with MBR & partitioning.
I'm curious.
How do they read
clusters/cylinders/sectors from the
disk? Is there some kind of API for this?
Is it again OS dependent? If so whats the way to for Linux & for windows?
EDIT:
Well, I'm not just curious I want a hands on experience. I want to write a simple application which displays each LBA.
Cylinders and sectors (wiki explanation) are largely obsoleted by the newer LBA (logical block addressing) scheme for addressing drives.
If you're curious about the history, use the Wikipedia article as a starting point. If you're just wondering how it works now, code is expected to simply use the LBA address (which works largely the same way as a file does - a linear array of bytes arranged in blocks)
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
For *nix, there have been already answers (/dev directory); for Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To actually perform reads or writes you have then to use the DeviceIoControl function.
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
I'm the OP. I'm combining Eric Seppanen's & Matteo Italia's answers to make it complete.
*NIX Platforms:
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
Windows Platform
For Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To perform reads or writes simply call ReadFile and WriteFile (buffer must be aligned on sector size).
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
Alternatively you can also you DeviceIoControl function which sends a control code directly to a specified device driver, causing the corresponding device to perform the corresponding operation.
On linux, as root, you can save your MBR like this (Assuming you drive is /dev/sda):
dd if=/dev/sda of=mbr bs=512 count=1
If you wanted to read 1Mb from you drive, starting at the 10th MB:
dd if=/dev/sda of=1Mb bs=1Mb count=1 skip=10
I'm trying to figure out the available disk space programmatically in windows. For this, I need to first get a list of the available drives, then check which of those are local drives and then query the available bytes on each local drive.
I'm a bit stuck on the first part, where the API presents two functions:
GetLogicalDrives (http://msdn.microsoft.com/en-us/library/aa364972(VS.85).aspx) which gives you a DWORD with the bits set (bit 0 if drive A is present, bit 1 if drive B etc)
GetLogicalDriveStrings (http://msdn.microsoft.com/en-us/library/aa364975(VS.85).aspx) which gives you the actual strings.
Now, although I'll be using strings later on, I'd prefer using the first option for querying. However, on my system a DWORD is typedef-ed to "unsigned long", which is 4 bytes, whereas drive letters only range A-Z (26 - i think - characters). Obviously, one can define more than 26 drives on their system (however unlikely they are to do so) - so I was wondering if there was any convention for those drives. Can someone point me to a resource on this?
Thanks.
DWORD is always 4 bytes, regardless of the system (it's a Win32 type).
The maximum for drive letters in Windows is 26. Because English alphabet has only 26 letters :). However, Windows allows two ways to mount a volume:
to a drive letter
to a directory (on an NTFS volume).
You can mount one volume to multiple locations (but no more than one drive letter, IIRC). A GUI for this task is presented by Control Panel -> Administrative Tools -> Computer Management -> Disk Management.
If you want to have more than 26 drives with the additional drives being redirects to already active drives and are okay with them not working properly in most programs, then you can assign more with the following method (be warned they won't even show up in the file explorer):
subst ♪: C:\Temp\
cd /D ♪:\
and to delete them (also they aren't preserved through restarts):
subst /D ♪:
You can enumerate all volumes and their mount points as described in this article.
You could use WMI. The following WMI query should list all drives:
SELECT * FROM Win32_DiskDrive
It it not sufficient to enumerate MS-DOS drives (there can be at most 26 of them, by the way, although each can be bound twice, once globally and once locally in your session), a volume can, for example, be mounted to a directory. What you want is probably to enumerate all volumes in the system, using FindFirstVolume et al. Take a look at the associated MSDN example.