I have come up against this problem a few times at inopportune moments:
Trying to work on open source Java projects with deep paths
Storing deep Fitnesse wiki trees in source control
An error trying to use Bazaar to import my source control tree
Why does this limit exist?
Why hasn't it been removed yet?
How do you cope with the path limit?
And no, switching to Linux or Mac OS X is not a valid answer to this question ;)
Quoting this article https://learn.microsoft.com/en-us/windows/desktop/FileIO/naming-a-file#maximum-path-length-limitation
Maximum Path Length Limitation
In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is "D:\some 256-character path string<NUL>" where "<NUL>" represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)
Now we see that it is 1+2+256+1 or [drive][:\][path][null] = 260. One could assume that 256 is a reasonable fixed string length from the DOS days. And going back to the DOS APIs we realize that the system tracked the current path per drive, and we have 26 (32 with symbols) maximum drives (and current directories).
The INT 0x21 AH=0x47 says “This function returns the path description without the drive letter and the initial backslash.” So we see that the system stores the CWD as a pair (drive, path) and you ask for the path by specifying the drive (1=A, 2=B, …), if you specify a 0 then it assumes the path for the drive returned by INT 0x21 AH=0x15 AL=0x19. So now we know why it is 260 and not 256, because those 4 bytes are not stored in the path string.
Why a 256 byte path string, because 640K is enough RAM.
This is not strictly true as the NTFS filesystem supports paths up to 32k characters. You can use the win32 api and "\\?\" prefix the path to use greater than 260 characters.
A detailed explanation of long path from the .Net BCL team blog.
A small excerpt highlights the issue with long paths
Another concern is inconsistent behavior that would result by exposing long path support. Long paths with the \\?\ prefix can be used in most of the file-related Windows APIs, but not all Windows APIs. For example, LoadLibrary, which maps a module into the address of the calling process, fails if the file name is longer than MAX_PATH. So this means MoveFile will let you move a DLL to a location such that its path is longer than 260 characters, but when you try to load the DLL, it would fail. There are similar examples throughout the Windows APIs; some workarounds exist, but they are on a case-by-case basis.
The question is why does the limitation still exist. Surely modern Windows can increase the side of MAX_PATH to allow longer paths. Why has the limitation not been removed?
The reason it cannot be removed is that Windows promised it would never change.
Through API contract, Windows has guaranteed all applications that the standard file APIs will never return a path longer than 260 characters.
Consider the following correct code:
WIN32_FIND_DATA findData;
FindFirstFile("C:\Contoso\*", ref findData);
Windows guaranteed my program that it would populate my WIN32_FIND_DATA structure:
WIN32_FIND_DATA {
DWORD dwFileAttributes;
FILETIME ftCreationTime;
FILETIME ftLastAccessTime;
FILETIME ftLastWriteTime;
//...
TCHAR cFileName[MAX_PATH];
//..
}
My application didn't declare the value of the constant MAX_PATH, the Windows API did. My application used that defined value.
My structure is correctly defined, and only allocates 592 bytes total. That means that i am only able to receive a filename that is less than 260 characters. Windows promised me that if i wrote my application correctly, my application would continue to work in the future.
If Windows were to allow filenames longer than 260 characters then my existing application (which used the correct API correctly) would fail.
For anyone calling for Microsoft to change the MAX_PATH constant, they first need to ensure that no existing application fails. For example, i still own and use a Windows application that was written to run on Windows 3.11. It still runs on 64-bit Windows 10. That is what backwards compatibility gets you.
Microsoft did create a way to use the full 32,768 path names; but they had to create a new API contract to do it. For one, you should use the Shell API to enumerate files (as not all files exist on a hard drive or network share).
But they also have to not break existing user applications. The vast majority of applications do not use the shell api for file work. Everyone just calls FindFirstFile/FindNextFile and calls it a day.
From Windows 10. you can remove the limitation by modifying a registry key.
Tip Starting in Windows 10, version 1607, MAX_PATH limitations have been removed from common Win32 file and directory functions. However, you must opt-in to the new behavior.
A registry key allows you to enable or disable the new long path behavior. To enable long path behavior set the registry key at HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD). The key's value will be cached by the system (per process) after the first call to an affected Win32 file or directory function (list follows). The registry key will not be reloaded during the lifetime of the process. In order for all apps on the system to recognize the value of the key, a reboot might be required because some processes may have started before the key was set.
The registry key can also be controlled via Group Policy at Computer Configuration > Administrative Templates > System > Filesystem > Enable NTFS long paths.
You can also enable the new long path behavior per app via the manifest:
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
<ws2:longPathAware>true</ws2:longPathAware>
</windowsSettings>
</application>
You can mount a folder as a drive. From the command line, if you have a path C:\path\to\long\folder you can map it to drive letter X: using:
subst x: \path\to\long\folder
One way to cope with the path limit is to shorten path entries with symbolic links.
For example:
create a C:\p directory to keep short links to long paths
mklink /J C:\p\foo C:\Some\Crazy\Long\Path\foo
add C:\p\foo to your path instead of the long path
You can enable long path names using PowerShell:
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name LongPathsEnabled -Type DWord -Value 1
Another version is to use a Group Policy in Computer Configuration/Administrative Templates/System/Filesystem:
As to why this still exists - MS doesn't consider it a priority, and values backwards compatibility over advancing their OS (at least in this instance).
A workaround I use is to use the "short names" for the directories in the path, instead of their standard, human-readable versions. So e.g. for C:\Program Files\ I would use C:\PROGRA~1\ You can find the short name equivalents using dir /x.
As to how to cope with the path size limitation on Windows - using 7zip to pack (and unpack) your path-length sensitive files seems like a viable workaround. I've used it to transport several IDE installations (those Eclipse plugin paths, yikes!) and piles of autogenerated documentation and haven't had a single problem so far.
Not really sure how it evades the 260 char limit set by Windows (from a technical PoV), but hey, it works!
More details on their SourceForge page here:
"NTFS can actually support pathnames up to 32,000 characters in
length."
7-zip also support such long names.
But it's disabled in SFX code. Some users don't like long paths, since
they don't understand how to work with them. That is why I have
disabled it in SFX code.
and release notes:
9.32 alpha 2013-12-01
Improved support for file pathnames longer than 260 characters.
4.44 beta 2007-01-20
7-Zip now supports file pathnames longer than 260 characters.
IMPORTANT NOTE: For this to work properly, you'll need to specify the destination path in the 7zip "Extract" dialog directly, rather than dragging & dropping the files into the intended folder. Otherwise the "Temp" folder will be used as an interim cache and you'll bounce into the same 260 char limitation once Windows Explorer starts moving the files to their "final resting place". See the replies to this question for more information.
It does, and it is a default for some reason, but you could easily override it with this registry key:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001
See: https://blogs.msdn.microsoft.com/jeremykuhne/2016/07/30/net-4-6-2-and-long-paths-on-windows-10/
Another way to cope with it is to use Cygwin, depending on what do you want to do with the files (i.e. if Cygwin commands suit your needs)
For example it allows to copy, move or rename files that even Windows Explorer can't. Or of course deal with the contents of them like md5sum, grep, gzip, etc.
Also for programs that you are coding, you could link them to the Cygwin DLL and it would enable them to use long paths (I haven't tested this though)
Related
It appears that both GlobalMemoryStatusEx and GetPerformanceInfo Windows APIs report the current size of the page file, but the page file can be configured (per disk) to grow up to a certain limit.
Is there an API to retrieve that limit, so that I can calculate the maximum possible commit size for the whole system (provided page file settings otherwise remain unchanged)?
Seems to be possible via WMI (COM and/or command-line):
wmic pagefile list /format:list
wmic pagefileset list /format:list
But can also be done in a platform-dependent* way by reading/parsing this registry setting:
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PagingFiles
As described here, this format is REG_MULTI_SZ (one line per page file):
<page-file> [<initial-size> <max-size>]
For example:
c:\pagefile.sys 1024 8192
In case of a system-managed page file the value would be:
?:\pagefile.sys
In case of system-managed the size is limited by the free space on the system drive (not entirely true for Hyper-V VMs).
* The registry setting appears to work at least in Windows XP, 7, 2012 and 10 (32/64-bit, PC).
I am trying to determine if a file is on a local drive. I found GetDriveType() WINAPI which retrieves the drive type. However reading the description of the return values it seems, and thats how I understand it, that it retrieves a flash drive as FIXED which is not what I want.
Its working fine on local drives:
bool IsDriveRemovableOrRemote(CString driveRoot)
{
UINT driveType = GetDriveType(driveRoot);
return (DRIVE_REMOVABLE == driveType || DRIVE_CDROM == driveType || DRIVE_NO_ROOT_DIR == driveType || DRIVE_REMOTE == driveType);
}
I don't have a flash/external drive to test ATM but I would like if someone can tell me if my interpretation is correct? and if so, what better alternative should I use?
Please bear in mind that I only have the path of the file.
You should read the doco more closely. While a Flash drive is considered a fixed device, there's a note in that linked page:
To determine whether a drive is a USB-type drive, call SetupDiGetDeviceRegistryProperty and specify the SPDRP_REMOVAL_POLICY property.
The process seems a little messy if all you start with is the path but you can start reading the doco here. It looks like you may need to enumerate the devices until you find one matching your drive.
To avoid doing this to all your requests, I would do a two-stage check. If your current method says it's not fixed, treat it as non-local.
If it says it is fixed, then you can enumerate the devices using my suggested method to be certain.
Alternatively, you can enumerate all fixed non-USB drives the first time you need to, and then just cache the information. I'm pretty certain that the list of these drives won't change while the system is running - drives that get added and deleted are, by definition, removable.
You can try using DeviceIoControl and query for the BusType = BusTypeUsb by passing IOCTL_STORAGE_QUERY_PROPERTY as its second parameter. Read Determining USB by Thomas Lee at the bottom of page.
Can anyone confirm that the maximum length of a key container name in Windows is 260 characters. I expect this to be the size because the value is stored as a file, but I have not found the specific answer as of yet.
In the meantime I'll see what trial and error I can do.
Updated:
I did trial and error with aspnet_reiis.exe and discovered that 260 is indeed the maximum length. 261 and above causes key creation failure.
I don't think there is one, or at least you shouldn't rely on there being one. From the documentation:
The key container name. This is a null-terminated string that identifies the key container to the CSP. This name is independent of the method used to store the keys. Some CSPs store their key containers internally (in hardware), some use the system registry, and others use the file system. When dwFlags is set to CRYPT_VERIFYCONTEXT, pszContainer must be set to NULL.
Piratically speaking you would probably be safe with 256 as that is the maximum length of both a file name and a registry key.
I'm trying to figure out the available disk space programmatically in windows. For this, I need to first get a list of the available drives, then check which of those are local drives and then query the available bytes on each local drive.
I'm a bit stuck on the first part, where the API presents two functions:
GetLogicalDrives (http://msdn.microsoft.com/en-us/library/aa364972(VS.85).aspx) which gives you a DWORD with the bits set (bit 0 if drive A is present, bit 1 if drive B etc)
GetLogicalDriveStrings (http://msdn.microsoft.com/en-us/library/aa364975(VS.85).aspx) which gives you the actual strings.
Now, although I'll be using strings later on, I'd prefer using the first option for querying. However, on my system a DWORD is typedef-ed to "unsigned long", which is 4 bytes, whereas drive letters only range A-Z (26 - i think - characters). Obviously, one can define more than 26 drives on their system (however unlikely they are to do so) - so I was wondering if there was any convention for those drives. Can someone point me to a resource on this?
Thanks.
DWORD is always 4 bytes, regardless of the system (it's a Win32 type).
The maximum for drive letters in Windows is 26. Because English alphabet has only 26 letters :). However, Windows allows two ways to mount a volume:
to a drive letter
to a directory (on an NTFS volume).
You can mount one volume to multiple locations (but no more than one drive letter, IIRC). A GUI for this task is presented by Control Panel -> Administrative Tools -> Computer Management -> Disk Management.
If you want to have more than 26 drives with the additional drives being redirects to already active drives and are okay with them not working properly in most programs, then you can assign more with the following method (be warned they won't even show up in the file explorer):
subst ♪: C:\Temp\
cd /D ♪:\
and to delete them (also they aren't preserved through restarts):
subst /D ♪:
You can enumerate all volumes and their mount points as described in this article.
You could use WMI. The following WMI query should list all drives:
SELECT * FROM Win32_DiskDrive
It it not sufficient to enumerate MS-DOS drives (there can be at most 26 of them, by the way, although each can be bound twice, once globally and once locally in your session), a volume can, for example, be mounted to a directory. What you want is probably to enumerate all volumes in the system, using FindFirstVolume et al. Take a look at the associated MSDN example.
IOKit and the DiskArbitration framework can tell me a lot of things about mounted volumes on a mac, but they don't seem to be able to differentiate between HFS+ and HFS Standard volumes.
The IOKit/DA keys Content, DAVolumeKind and DAMediaContent are always Apple_HFS and hfs for both HFS Standard and HFS+ volumes.
diskutil and DiskUtility.app can tell the difference, but I they don't seem to have been open sourced by Apple.
p.s. statfs (2) does not differentiate
There are two ways to do this:
Use getattrlist() to retrieve the ATTR_VOL_SIGNATURE attribute for the mount path of the volume.
Use the Carbon call FSGetVolumeInfo() and look in the signature field of the returned struct.
The signature of a volume is a 16 bit value, usually interpreted as two ASCII characters. The signature for HFS is 'BD', HFS+ is 'H+', and case sensitive HFS+ is 'HX'.
The man page for getattrlist says the field is a u_int32, but the equivalent field in the FSVolumeInfo struct is only 16 bits, so I'm not sure which 16 bits of the 32 are filled in with the signature when using getattrlist, you'll probably have to just experiment a bit if you want to go the non-Carbon route.
getattrlist man page
HFS Plus Volume Format reference
FSGetVolumeInfo
In addition to Carbon FSGetVolumeInfo() which returns a FSVolumeInfo containing signature and filesystemID fields, there is the Cocoa -getFileSystemInfoForPath: method of class NSWorkspace which returns a string representation of filesystem type: e.g., hfs for HFS+ and msdos for DOS FAT.
The other problem you can run into should you ever try your hand at reading partition maps directly is that, historically, HFS+ volumes were nested inside HFS wrappers. This was done so that anyone trying to use an HFS+ disk with an older OS would see a single file on the drive explaining where all the rest of their data was.