I found a reference to a file in a log that had the following format:
\\?\C:\Path\path\file.log
I cannot find a reference to what the sequence of \?\ means. I believe the part between the backslashes refers to a hostname.
For instance, on my Windows computer, the following works just fine:
dir \\?\C:\
and also, just fine with same result:
dir \\.\C:\
Questions:
Is there a reference to what the question mark means in this particular path format?
What might generate a file path in such a format?
A long read, but worth reading if you are in this domain: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
Extract:
The Windows API has many functions that also have Unicode versions to
permit an extended-length path for a maximum total path length of
32,767 characters. This type of path is composed of components
separated by backslashes, each up to the value returned in the
lpMaximumComponentLength parameter of the GetVolumeInformation
function (this value is commonly 255 characters). To specify an
extended-length path, use the "\\?\" prefix. For example,
"\\?\D:\very long path".
and:
The "\\?\" prefix can also be used with paths constructed according to
the universal naming convention (UNC). To specify such a path using
UNC, use the "\\?\UNC\" prefix. For example, "\\?\UNC\server\share",
where "server" is the name of the computer and "share" is the name of
the shared folder. These prefixes are not used as part of the path
itself. They indicate that the path should be passed to the system
with minimal modification, which means that you cannot use forward
slashes to represent path separators, or a period to represent the
current directory, or double dots to represent the parent directory.
Because you cannot use the "\\?\" prefix with a relative path,
relative paths are always limited to a total of MAX_PATH characters.
The Windows API parses input strings for file I/O. Among other things, it translates / to \ as part of converting the name to an NT-style name, or interpreting the . and .. pseudo directories. With few exceptions, the Windows API also limits path names to 260 characters.
The documented purpose of the \\?\ prefix is:
For file I/O, the "\\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system.
Among other things, this allows using otherwise reserved symbols in path names (such as . or ..). Opting out of any translations, the system no longer has to maintain an internal buffer, and the arbitrary limit of 260 characters can also be lifted (as long as the underlying filesystem supports it). Note, that this is not the purpose of the \\?\ prefix, rather than a corollary, even if the prefix is primarily used for its corollary.
Related
What is the difference between paths prefixed with \??\ and those prefixed with \\?\
At Windows 7 CMD-Line
DIR gives: \??\Volume{00000000-0000-0000-0000-000000000000}\
WMIC VOLUME LIST gives \\?\Volume{00000000-0000-0000-0000-000000000000}\
Thanks
Additional Information:
I created a directory on my system drive where i placed all my mount points into. So I did a MD C:\HDDs first, then a
MD C:\HDD\Drive1, MD C:\HDD\Drive2 ...for all my drives. After mounting the drives on those empty directories, I can see the GUID by switching to that Directory with CD /D C:\HDDs and issueing a DIR command. Maybe I have to issue a DIR /ah to Show hidden stuff, just in case the mount points are hidden directories ...
In NT, "\??\" is a path prefix that stands for the object directory that's reserved for a user's devices, or more specifically, device aliases. A device alias is implemented in the object namespace as a symbolic link that typically resolves to a device object in the "\Device" directory. Sometimes in the documentation these device aliases are referred to as "junctions" in the object namespace, which is not to be confused with "directory junctions" (or mount points) in a file system.
Using the "\??\" prefix instructs the object manager to search in the caller's local device directory, "\Sessions\0\DosDevices\[Logon Authentication ID]", which is coupled to (i.e. shadows) the global device directory, "\Global??". For efficiency, both of these directories are cached by an access token's associated logon-session record and also by each process object. The SYSTEM logon (ID 0x3E7) uses "\Global??" as its local device directory. Note that the local directory has a "Global" link to allow accessing global devices when a local device shadows the global one (e.g. "\\?\Global\Z:"), or to allow a device driver to create a global device when not executing in a SYSTEM thread. NT originally used a single "\DosDevices" directory, regardless of the caller. With the introduction of Terminal Services and fast-user switching in NT 5, they had to generalize it to the present system of local and global devices. Nowadays, for backwards compatibility, "\DosDevices" is a link to "\??".
Translating DOS paths to native NT paths is implemented by NT's user-mode runtime library (i.e. the Rtl prefixed functions that are exported by "ntdll.dll").
The straight-forward case is a path that's prefixed by either "\\.\" or "\\?\". This is a local device path, not a UNC path. (Strictly speaking it's in the form of a UNC path, but "." and "?" are reserved device domains.) For this case, the prefix is simply replaced by NT "\??\". The difference between the two WINAPI device-path prefixes is that a "\\?\" path (all backslashes, no forward slashes) is a so-called "extended" path, which bypasses all normalization, whereas a "\\.\" path gets normalized.
Device path normalization resolves "." and ".." components, replaces forward slashes with backslashes, and strips trailing spaces and dots from the final path component. Because forward slashes are translated to backslashes, the prefix of a normalized device path can be "//./" or "//?/" or any combination of slashes and backslashes, except for exactly "\\?\". Note that if the process doesn't support long paths, normalized paths are limited to less than MAX_PATH (260) characters. (Long-path support can be enabled in Windows 10 through a combination of registry and application manifest settings; consult the relevant documentation.) GetFullPathNameW handles both prefixes equivalently and even normalizes an extended path that begins with "\\?\".
UNC paths are also unsurprising. The runtime library simply replaces the leading "\\" in the normalized path with an explicit reference to the "UNC" device, i.e. "\??\UNC\" (e.g. "\\server\share" -> "\??\UNC\server\share"). Note that "\Global??\UNC" is a symbolic link to "\Device\Mup", the Multiple UNC Provider device, which is responsible for mapping the "server\share" to the correct UNC provider (e.g. to the LanmanWorkstation redirector for an SMB share).
DOS drive paths (i.e. those beginning with an "[A-Z]:" drive) are interesting in a couple of cases. The first is that the runtime library supports per-drive working directories using conventionally 'hidden' environment variables such as "=C:". For example, "C:System32" resolves to "C:\Windows\System32" if the "=C:" environment variable is set to "C:\Windows". Also, if the last component of the path is a reserved DOS device name, including if the name has trailing colons, spaces, dots, and even a file extension, the path gets translated to a device path (e.g. "C:\Windows\nul: .txt" -> "\??\nul"). (DOS devices are also reserved in the final component of relative paths that have no drive.) Otherwise, the runtime library simply prepends "\??\" to the normalized path (e.g. "C:/Windows" -> "\??\C:\Windows").
A DOS drive such as "C:" (i.e. "\Global??\C:") is an alias for an NT volume device (i.e. an object symbolic link). The NT device name is not persistent and is typically enumerated, so the final target depends on the relative order in which volumes are added, and it may even change if a volume is removed and subsequently restored. For example, the final NT path for "E:\Temp" on a removable drive may start out as "\Device\HarddiskVolume8\Temp" and then, after removing and reinserting it, the new final path is "\Device\HarddiskVolume10\Temp". The mount-point manager implements persistence using the unique ID of a volume, which it associates with a volume GUID name (e.g. "Volume{00000000-0000-0000-0000-000000000000}") and optionally (usually) a DOS drive letter. The GUID name is used to implement volume mount points in file systems that support junctions (i.e. IO_REPARSE_TAG_MOUNT_POINT reparse points), such as NTFS and ReFS.
I had to help someone delete a folder that had weird characters in it that caused the path to be re-interepeted as a different path:
c:\test. -> c:\test
It took me a while to recall the \\?\ construct, since I have no idea what it's called or how to search for it. Once I remembered it, though, it was easy:
\\?\c:\test. -> c:\test.
What is the name of this construct, that I (and others) may search for it?
I don't think it has an official name in widespread use, so I doubt that you'll get very far in any searches. It is described here: https://msdn.microsoft.com/en-gb/library/windows/desktop/aa365247.aspx#maxpath
The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".
For this usage it might be called the extended-length path prefix. However the prefix serves other purposes, most specifically suppressing user mode path canonicalisation, the purpose that you were availing yourself of.
As you can see from comments to this answer, there are lots of varied opinions on the most suitable name. I think that we can all agree that there is no single officially used name for this thing!
I'm reading this. Here I've found some code lines, for example: wsprintf(szDrive, "\\\\.\\%c:", *lpszSrc); I want to ask, what does this string give?
I tried to look for information but all that I've found is:
In the ANSI version of this function, the name is limited to MAX_PATH
characters. To extend this limit to 32,767 wide characters, call the
Unicode version of the function and prepend "\\?\" to the path. For
more information, see Naming Files, Paths, and Namespaces.
and this do not answer into my question, so asking here. As I think it should be connected with windows specific or NTFS but not sure about that.
The %c is the single character format specifier for wsprintf.
The code is used to generate path names of this form:
\\.\C:
This is the path to a physical volume. You use such a path when performing file operations directly on a volume, bypassing the file system. So you'd use such a path when implementing raw disk copy, for example. The documentation for CreateFile has more detail.
This all ties in with the fact that the code you found this in performs a raw disk copy.
I have several level of directories in the folder path. when the path exceeds 256, then I could not create a sub-folder or file from it. Is there any chance to build paths more than this length.
Can anyone help me out.
In fact the limit on path strings is 260 characters. The underlying OS, these days, can support much longer path names, up to 32,767 characters. In order to name a path with a long name you need to use the magic \\?\ prefix, and use the Unicode version of the API.
However, many tools don't support such long names. A classic example of such a tool is Explorer which won't let you create objects with names longer than 260 characters. Because of this I strongly advise you to avoid creating such long names—doing so will save you much heartache in the long run.
This should get you started: http://msdn.microsoft.com/en-us/library/aa365247%28v=vs.85%29.aspx#maxpath
Sadly it's an issue that I don't think will be going away any time soon, so you'd do well to familiarize yourself with that stuff.
As an aside, if you have access to robocopy (comes packaged with Windows Vista and 7, but is also available for XP), which supports long paths, you could create your files/subfolders in a higher-up folder and then use robocopy to move the subfolder to its desired location deeper in the folder tree.
According to the documentation here http://msdn.microsoft.com/en-us/library/Aa365247, the maximum length is actually about 32,000, but most windows APIs still limit you to MAX_PATH which is 260. There are some unicode APIs that let you go beyond the 260 limit.
See here, http://msdn.microsoft.com/en-us/library/aa363856.
In the ANSI version of this function, the name is limited to MAX_PATH characters. To extend this limit to 32,767 wide characters, call the Unicode version of the function and prepend \\?\ to the path. For more information, see Naming a File.
This is an addendum to the answers above. I extracted only a summary to what I think is relevant, from Microsoft's official documentation:
Maximum Path Length Limitation
In the Windows API (with some exceptions), the maximum length for a path is MAX_PATH,
which is defined as 260 characters. A local path is structured in the following order:
drive letter, colon, backslash, name components separated by backslashes, and a terminating null character.
Example: "D:\some 256-character-path-string" -> 256
Using long paths
The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters.
To specify an extended-length path, use the "\?" prefix. For example, "\?\D:\very long path".
Relative paths
Relative paths are always limited to a total of MAX_PATH characters.
Enable Long Paths in Win10
Starting in Windows 10.1607, MAX_PATH limitations have been removed from common Win32 file and directory functions.
However, you must opt-in to the new behavior.
From Microsoft documentation:
https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file?redirectedfrom=MSDN#maximum-path-length-limitation
Warning for Delphi users:
There is a problem in IOUtils. It cannot be used in conjunction with Max_Path. It uses InternalCheckDirPathParam all over the place!
Details: TDirectory.GetDirectoryRoot does not handle correctly paths of Max_Path characters
After years of programming it's still some of the simple things that keep tripping me up.
Is there a commonly agreed definition of filename ?
Even the wikipedia article confuses the two interpretations.
It starts by defining it as 'a special kind of string used to uniquely identify a file stored on the file system of a computer'. That seems clear enough, and suggests that a filename is a fully qualified filename, specifying the complete path to the file.
However, it then goes on to:
talk about basename and extension (so basename would contain an absolute path ?)
says that the length of a filename in DOS is limited to 8.3
says that a filename without a path part is assumed to be a file in the current working directory (so the filename does not uniquely identify a file)
So, simple questions:
what is a correct definition of 'filename' (include references)
how should I unambiguously name variables for:
a path to a file (which can be absolute/full or relative)
a path to a resource that can be a file/directory/socket
No references, just vernacular from experience. When I'm being specific I tend to use:
path or filespec (or file specification): all of the characters needed to identify a file on a filesystem. The path may be absolute (starting from the root, or topmost, directory) or relative (starting from the currently active directory).
filename: the characters needed to identify a file within the current directory.
extension: characters at the end of the filename that typically identify the type of the file. By convention, the extension usually starts with a dot ("."), and a filename may contain more than one extension.
basename: the filename up to (but not including) the dot that begins the first extension.
Javadoc for File.getName() method
file·name also file name
(fīl'nām') Pronunciation Key n. A
name given to a computer file to
distinguish it from other files, often
containing an extension that
classifies it by type.
# Dictionary.com
It states that a filename is used to name a file, (just like you name a person). And that it's used to distinguish it from other files. This does not tell you it includes a path, or other file-system imposed attributes.
This definition does say that often a filename has an extension. But this definition is very careful... (Which I think is a good thing)
So.. before you start thinking about paths and such, you have to set your scope. Are you in a unix world? Ar you in a dos/windows world?
Again no references, but the file name specification depends on the operating system or to be more accurate the file system. Lets start with early versions of DOS (Disk Operating System). File names were 8 character names containing numbers, letters, dashes, and underscores. They were followed by a three, two, one, or even zero character extension used to identify the file type. A dot separated the name from the extension. The name had to be unique in the directory.
You could extend the name by adding a directory name, or series of directory names. a slash character separated the directory names from each other and from the file name. This was usually referred to as the path name. The path was relative to current directory.
Finally in DOS you could include the drive name. Usually a single letter followed by a : and a slash (some systems two slashes). Adding the drive to the path made it an absolute path instead of relative.
Today most of us use long file names which do not follow the old 8 character dot three character pattern. Still many files systems keep such as name and use the long name simply as a pointer to old style identifier.