I'm trying to figure out the available disk space programmatically in windows. For this, I need to first get a list of the available drives, then check which of those are local drives and then query the available bytes on each local drive.
I'm a bit stuck on the first part, where the API presents two functions:
GetLogicalDrives (http://msdn.microsoft.com/en-us/library/aa364972(VS.85).aspx) which gives you a DWORD with the bits set (bit 0 if drive A is present, bit 1 if drive B etc)
GetLogicalDriveStrings (http://msdn.microsoft.com/en-us/library/aa364975(VS.85).aspx) which gives you the actual strings.
Now, although I'll be using strings later on, I'd prefer using the first option for querying. However, on my system a DWORD is typedef-ed to "unsigned long", which is 4 bytes, whereas drive letters only range A-Z (26 - i think - characters). Obviously, one can define more than 26 drives on their system (however unlikely they are to do so) - so I was wondering if there was any convention for those drives. Can someone point me to a resource on this?
Thanks.
DWORD is always 4 bytes, regardless of the system (it's a Win32 type).
The maximum for drive letters in Windows is 26. Because English alphabet has only 26 letters :). However, Windows allows two ways to mount a volume:
to a drive letter
to a directory (on an NTFS volume).
You can mount one volume to multiple locations (but no more than one drive letter, IIRC). A GUI for this task is presented by Control Panel -> Administrative Tools -> Computer Management -> Disk Management.
If you want to have more than 26 drives with the additional drives being redirects to already active drives and are okay with them not working properly in most programs, then you can assign more with the following method (be warned they won't even show up in the file explorer):
subst ♪: C:\Temp\
cd /D ♪:\
and to delete them (also they aren't preserved through restarts):
subst /D ♪:
You can enumerate all volumes and their mount points as described in this article.
You could use WMI. The following WMI query should list all drives:
SELECT * FROM Win32_DiskDrive
It it not sufficient to enumerate MS-DOS drives (there can be at most 26 of them, by the way, although each can be bound twice, once globally and once locally in your session), a volume can, for example, be mounted to a directory. What you want is probably to enumerate all volumes in the system, using FindFirstVolume et al. Take a look at the associated MSDN example.
Related
How to uniquely identify computer (mainboard) using C#(.Net/Mono, local application)?
Edition. We can identify mainboard in .Net using something like this (see Get Unique System Identifiers in C#):
using System.Management;
...
ManagementObjectSearcher searcher = new ManagementObjectSearcher("select * from Win32_MotherboardDevice");
...
But unfortunately Mono does not support System.Management. How to do it under Mono for Linux? - I don't know :(
Write a function that takes a few unique hardware parameters as input and generates a hash out of them.
For example, Windows activation looks at the following hardware characteristics:
Display Adapter
SCSI Adapter
IDE Adapter (effectively the motherboard)
Network Adapter (NIC) and its MAC Address
RAM Amount Range (i.e., 0-64mb, 64-128mb, etc.)
Processor Type
Processor Serial Number
Hard Drive Device
Hard Drive Volume Serial Number (VSN)
CD-ROM / CD-RW / DVD-ROM
You can pick up a few of them to generate your unique computer identifier.
Please see: Get Unique System Identifiers in C#
You realistically have MotherboardID, CPUID, Disk Serial and MAC address, from experience none of them are 100%.
Our stats show
Disk serial Is missing 0.1 %
MAC Is missing 1.3 %
Motherboard ID Is missing 30 %
CPUID Is missing 99 %
0.04% of machines tested yielded no information, we couldn't even read the computer name. It maybe that these were some kind of virtual PC, HyperV or VMWare instance, or maybe just very locked down? In any case your design has to be able to cope with these cases.
Disk serial is the most reliable, but easy to change, mac can be changed and depending on the filtering applied when reading it can change if device drivers are added (hyperv, wireshark etc).
Motherboard and CPUID sometimes return values that are invalid "NONE", "AAAA..", "XXXX..." etc.
You should also note that these functions can be very slow to call (they may take a few seconds even on a fast PC), so it may be worth kicking them off on a background thread as early as possible, you ideally don't want to be blocking on them.
Try this:
http://carso-owen.blogspot.com/2007/02/how-to-get-my-motherboard-serial-number.html
Personally though, I'd go with hard drive serial number. If a mainboard dies and is replaced, that PC isn't valid any more. If the HDD drive is replaced, it doesn't matter too much because the software was on it.
Of course, on the other hand, if the HDD is just moved elsewhere, the information goes with it, so you might want to look at a combination of serial numbers, depending what you want it for.
How about the MAC address of the network card?
I am trying to determine if a file is on a local drive. I found GetDriveType() WINAPI which retrieves the drive type. However reading the description of the return values it seems, and thats how I understand it, that it retrieves a flash drive as FIXED which is not what I want.
Its working fine on local drives:
bool IsDriveRemovableOrRemote(CString driveRoot)
{
UINT driveType = GetDriveType(driveRoot);
return (DRIVE_REMOVABLE == driveType || DRIVE_CDROM == driveType || DRIVE_NO_ROOT_DIR == driveType || DRIVE_REMOTE == driveType);
}
I don't have a flash/external drive to test ATM but I would like if someone can tell me if my interpretation is correct? and if so, what better alternative should I use?
Please bear in mind that I only have the path of the file.
You should read the doco more closely. While a Flash drive is considered a fixed device, there's a note in that linked page:
To determine whether a drive is a USB-type drive, call SetupDiGetDeviceRegistryProperty and specify the SPDRP_REMOVAL_POLICY property.
The process seems a little messy if all you start with is the path but you can start reading the doco here. It looks like you may need to enumerate the devices until you find one matching your drive.
To avoid doing this to all your requests, I would do a two-stage check. If your current method says it's not fixed, treat it as non-local.
If it says it is fixed, then you can enumerate the devices using my suggested method to be certain.
Alternatively, you can enumerate all fixed non-USB drives the first time you need to, and then just cache the information. I'm pretty certain that the list of these drives won't change while the system is running - drives that get added and deleted are, by definition, removable.
You can try using DeviceIoControl and query for the BusType = BusTypeUsb by passing IOCTL_STORAGE_QUERY_PROPERTY as its second parameter. Read Determining USB by Thomas Lee at the bottom of page.
I'm trying to write a test pattern to every sector of a formatted USB drive. There is one logical drive (e.g. h:). This volume is FAT-formatted and contains data to be overwritten. Also, I want to overwrite the whole physical drive. The program is running with elevated user rights.
First I did the following:
// from the drive letter "h:" I get the physical disk number using
// IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS => "\\.\PhysicalDrive2"
hDevice = ::CreateFile( "\\.\PhysicalDrive2", GENERIC_READ|GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL );
// get the number of available sectors with IOCTL_DISK_GET_DRIVE_GEOMETRY_EX
// => ulNumberOfSectors
// now I try to write some sectors, e.g. 2 (I want to use a higher value):
WriteFile( hDevice, abBuffer, 2*512, &byteswritten, NULL );
The call to WriteFile fails with ERROR_ACCESS_DENIED.
If I write one sector, it works.
When I overwrite the first sector and plug the device out and in again, Windows wants to format it. In this situation my code with 2048 sectors at once works without ERROR_ACCESS_DENIED.
I also unmounted the volume as described in CodeProject: WriteFile on Physical Drives with Windows 7 but this didn't change anything. Obviously the volume is unmounted because it's no longer visible in Windows Explorer.
I want to write more than a single sector due to perfomance reasons. I'm also afraid that other problems in the field might occur because I don't fully understand ths problem.
Any suggestions?
I didn't have problems with different WriteFile() sizes, but I did solve the
WriteFile(): Access is denied <ERROR_ACCESS_DENIED/5> to
'\.\physicaldriveX
devices (usually USB HDD/SSD) in Windows 7 running as Administrator (elevated rights) as follows:
Computer Management -> Disk Management:
Volume (H: in your case) -> right-click -> Delete Volume
Disk (Disk 2 in your case) -> right-click -> Off-line
Disk (Disk 2 in your case) -> right-click -> On-line
After that, I'm able to write to '\.\physicaldriveX' with no problem.
I think the Win7 locks (unlike previous Windows releases) the physical device as long as there is any file system on the device to avoid consistency problems.
You cannot directly access sectors of a drive which are owned by a mounted filesystem.
See Changes to the file system and to the storage stack to restrict direct disk access and direct volume access
The documentation for FSCTL_DISMOUNT_VOLUME describes the following sequence for overwriting a filesystem:
Open a volume.
Lock the volume.
Format the volume.
Dismount the volume.
Unlock the volume.
Close the volume handle.
Your pattern-writing operation would be in step 3 instead of formatting.
Another method is to use clean to delete all the partitions (and ALL DATA) on the disk:
C:\> diskpart
Diskpart> list disk
Diskpart> select disk N (where N is your disk number)
Diskpart> clean
Diskpart> exit
I needed to recover the partition table I deleted accidentally. I used an application named TestDisk. Its simply mind blowing. I reads each cylinder from the disk. I've seen similar such applications which work with MBR & partitioning.
I'm curious.
How do they read
clusters/cylinders/sectors from the
disk? Is there some kind of API for this?
Is it again OS dependent? If so whats the way to for Linux & for windows?
EDIT:
Well, I'm not just curious I want a hands on experience. I want to write a simple application which displays each LBA.
Cylinders and sectors (wiki explanation) are largely obsoleted by the newer LBA (logical block addressing) scheme for addressing drives.
If you're curious about the history, use the Wikipedia article as a starting point. If you're just wondering how it works now, code is expected to simply use the LBA address (which works largely the same way as a file does - a linear array of bytes arranged in blocks)
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
For *nix, there have been already answers (/dev directory); for Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To actually perform reads or writes you have then to use the DeviceIoControl function.
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
I'm the OP. I'm combining Eric Seppanen's & Matteo Italia's answers to make it complete.
*NIX Platforms:
It's easy due to the magic of *nix special device files. You can open and read /dev/sda the same way you'd read any other file.
Just use open, lseek, read, write (or pread, pwrite). If you want to make sure you're physically fetching data from a drive and not from kernel buffers you can open with the flag O_DIRECT (though you must perform aligned reads/writes of 512 byte chunks for this to work).
Windows Platform
For Windows, there are the special objects \\.\PhisicalDriveX, with X as the number of the drive, which can be opened using the normal CreateFile API. To perform reads or writes simply call ReadFile and WriteFile (buffer must be aligned on sector size).
More info can be found in "Physical Disks and Volumes" section of the CreateFile API documentation.
Alternatively you can also you DeviceIoControl function which sends a control code directly to a specified device driver, causing the corresponding device to perform the corresponding operation.
On linux, as root, you can save your MBR like this (Assuming you drive is /dev/sda):
dd if=/dev/sda of=mbr bs=512 count=1
If you wanted to read 1Mb from you drive, starting at the 10th MB:
dd if=/dev/sda of=1Mb bs=1Mb count=1 skip=10
I have come up against this problem a few times at inopportune moments:
Trying to work on open source Java projects with deep paths
Storing deep Fitnesse wiki trees in source control
An error trying to use Bazaar to import my source control tree
Why does this limit exist?
Why hasn't it been removed yet?
How do you cope with the path limit?
And no, switching to Linux or Mac OS X is not a valid answer to this question ;)
Quoting this article https://learn.microsoft.com/en-us/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation
Maximum Path Length Limitation
In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is "D:\some 256-character path string<NUL>" where "<NUL>" represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)
Now we see that it is 1+2+256+1 or [drive][:\][path][null] = 260. One could assume that 256 is a reasonable fixed string length from the DOS days. And going back to the DOS APIs we realize that the system tracked the current path per drive, and we have 26 (32 with symbols) maximum drives (and current directories).
The INT 0x21 AH=0x47 says “This function returns the path description without the drive letter and the initial backslash.” So we see that the system stores the CWD as a pair (drive, path) and you ask for the path by specifying the drive (1=A, 2=B, …), if you specify a 0 then it assumes the path for the drive returned by INT 0x21 AH=0x15 AL=0x19. So now we know why it is 260 and not 256, because those 4 bytes are not stored in the path string.
Why a 256 byte path string, because 640K is enough RAM.
This is not strictly true as the NTFS filesystem supports paths up to 32k characters. You can use the win32 api and "\\?\" prefix the path to use greater than 260 characters.
A detailed explanation of long path from the .Net BCL team blog.
A small excerpt highlights the issue with long paths
Another concern is inconsistent behavior that would result by exposing long path support. Long paths with the \\?\ prefix can be used in most of the file-related Windows APIs, but not all Windows APIs. For example, LoadLibrary, which maps a module into the address of the calling process, fails if the file name is longer than MAX_PATH. So this means MoveFile will let you move a DLL to a location such that its path is longer than 260 characters, but when you try to load the DLL, it would fail. There are similar examples throughout the Windows APIs; some workarounds exist, but they are on a case-by-case basis.
The question is why does the limitation still exist. Surely modern Windows can increase the side of MAX_PATH to allow longer paths. Why has the limitation not been removed?
The reason it cannot be removed is that Windows promised it would never change.
Through API contract, Windows has guaranteed all applications that the standard file APIs will never return a path longer than 260 characters.
Consider the following correct code:
WIN32_FIND_DATA findData;
FindFirstFile("C:\Contoso\*", ref findData);
Windows guaranteed my program that it would populate my WIN32_FIND_DATA structure:
WIN32_FIND_DATA {
DWORD dwFileAttributes;
FILETIME ftCreationTime;
FILETIME ftLastAccessTime;
FILETIME ftLastWriteTime;
//...
TCHAR cFileName[MAX_PATH];
//..
}
My application didn't declare the value of the constant MAX_PATH, the Windows API did. My application used that defined value.
My structure is correctly defined, and only allocates 592 bytes total. That means that i am only able to receive a filename that is less than 260 characters. Windows promised me that if i wrote my application correctly, my application would continue to work in the future.
If Windows were to allow filenames longer than 260 characters then my existing application (which used the correct API correctly) would fail.
For anyone calling for Microsoft to change the MAX_PATH constant, they first need to ensure that no existing application fails. For example, i still own and use a Windows application that was written to run on Windows 3.11. It still runs on 64-bit Windows 10. That is what backwards compatibility gets you.
Microsoft did create a way to use the full 32,768 path names; but they had to create a new API contract to do it. For one, you should use the Shell API to enumerate files (as not all files exist on a hard drive or network share).
But they also have to not break existing user applications. The vast majority of applications do not use the shell api for file work. Everyone just calls FindFirstFile/FindNextFile and calls it a day.
From Windows 10. you can remove the limitation by modifying a registry key.
Tip Starting in Windows 10, version 1607, MAX_PATH limitations have been removed from common Win32 file and directory functions. However, you must opt-in to the new behavior.
A registry key allows you to enable or disable the new long path behavior. To enable long path behavior set the registry key at HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD). The key's value will be cached by the system (per process) after the first call to an affected Win32 file or directory function (list follows). The registry key will not be reloaded during the lifetime of the process. In order for all apps on the system to recognize the value of the key, a reboot might be required because some processes may have started before the key was set.
The registry key can also be controlled via Group Policy at Computer Configuration > Administrative Templates > System > Filesystem > Enable NTFS long paths.
You can also enable the new long path behavior per app via the manifest:
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
<ws2:longPathAware>true</ws2:longPathAware>
</windowsSettings>
</application>
You can mount a folder as a drive. From the command line, if you have a path C:\path\to\long\folder you can map it to drive letter X: using:
subst x: \path\to\long\folder
One way to cope with the path limit is to shorten path entries with symbolic links.
For example:
create a C:\p directory to keep short links to long paths
mklink /J C:\p\foo C:\Some\Crazy\Long\Path\foo
add C:\p\foo to your path instead of the long path
You can enable long path names using PowerShell:
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled -Type DWord -Value 1
Another version is to use a Group Policy in Computer Configuration/Administrative Templates/System/Filesystem:
As to why this still exists - MS doesn't consider it a priority, and values backwards compatibility over advancing their OS (at least in this instance).
A workaround I use is to use the "short names" for the directories in the path, instead of their standard, human-readable versions. So e.g. for C:\Program Files\ I would use C:\PROGRA~1\ You can find the short name equivalents using dir /x.
As to how to cope with the path size limitation on Windows - using 7zip to pack (and unpack) your path-length sensitive files seems like a viable workaround. I've used it to transport several IDE installations (those Eclipse plugin paths, yikes!) and piles of autogenerated documentation and haven't had a single problem so far.
Not really sure how it evades the 260 char limit set by Windows (from a technical PoV), but hey, it works!
More details on their SourceForge page here:
"NTFS can actually support pathnames up to 32,000 characters in
length."
7-zip also support such long names.
But it's disabled in SFX code. Some users don't like long paths, since
they don't understand how to work with them. That is why I have
disabled it in SFX code.
and release notes:
9.32 alpha 2013-12-01
Improved support for file pathnames longer than 260 characters.
4.44 beta 2007-01-20
7-Zip now supports file pathnames longer than 260 characters.
IMPORTANT NOTE: For this to work properly, you'll need to specify the destination path in the 7zip "Extract" dialog directly, rather than dragging & dropping the files into the intended folder. Otherwise the "Temp" folder will be used as an interim cache and you'll bounce into the same 260 char limitation once Windows Explorer starts moving the files to their "final resting place". See the replies to this question for more information.
It does, and it is a default for some reason, but you could easily override it with this registry key:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001
See: https://blogs.msdn.microsoft.com/jeremykuhne/2016/07/30/net-4-6-2-and-long-paths-on-windows-10/
Another way to cope with it is to use Cygwin, depending on what do you want to do with the files (i.e. if Cygwin commands suit your needs)
For example it allows to copy, move or rename files that even Windows Explorer can't. Or of course deal with the contents of them like md5sum, grep, gzip, etc.
Also for programs that you are coding, you could link them to the Cygwin DLL and it would enable them to use long paths (I haven't tested this though)