Where is the use of "\\?\" defined? - windows

This command is to delete all files and sub-folders in a folder
rd /s "\\?\D:\TestFolder
This command snippet got from a youtube video right here
Could someone explain what this, \\?\, does?

It's the prefix to bypass Windows path normalization. With it you'll be able to access paths that are not valid in Win32 namespace like names ending with . or spaces: D:\TestFolder\folder ending with space \file name ending with dot., or files with path longer than MAX_PATH (260 characters in older Windows)
For file I/O, the "\\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system. For example, if the file system supports large paths and file names, you can exceed the MAX_PATH limits that are otherwise enforced by the Windows APIs. For more information about the normal maximum path limitation, see the previous section Maximum Path Length Limitation.
Naming Files, Paths, and Namespaces - Win32 File Namespaces
See
Dots at the end of file name?
How to copy files that have too long of a filepath in Windows?

Related

Why does cmd DIR search include files that start with the 3 searched letters but then differ?

If in a directory I have two files, test.pro and test.properties, and I run dir /s *.pro to find all files with the .pro extension, it lists both files. However, when I run dir /s *.pr, it lists neither. Why does searching for a three-letter file extension list files with extensions that start with the three letters? How can I search for ONLY .pro files using dir?
The command DIR lets the file system search for file system entries (file names, directory names, reparse points (links)) matching the wildcard pattern *.pro in long name or in short 8.3 name on short name management also enabled for the file system of current drive.
The short file name of test.pro is TEST.PRO.
The short file name of test.properties is TEST~1.PRO.
Therefore the wildcard pattern *.pro matches test.pro and TEST~1.PRO displayed with long file name test.properties.
The wildcard pattern *.pr does not match with the two file names because of the file extension is pro respectively properties and not just pr. The wildcard pattern *.pr? would also find the file names test.pro and TEST~1.PRO displayed with long file name test.properties.
There can be used %SystemRoot%\System32\where.exe /R . *.pro to find recursively in current directory and all its subdirectories files with file extension .pro not matching file names with a longer file extension beginning with pro because of WHERE applies the wildcard pattern only on long file names.
DIR and also FOR use the Windows file I/O function which directly searches with the wildcard pattern *.pro for suitable file system entries. WHERE uses the Windows file I/O function to search for * to get a list of long names of all file system entries and then applies the wildcard pattern itself on each string returned by the file system. For that reason the usage of the wildcard pattern *.pro returns a positive result only on test.pro and not on test.properties on using WHERE.

Windows directory that will never contain non-ASCII characters for temp file?

Using MinGW 7.3.0 on Windows, Hunspell can't load the dictionary files from locations that have non-ASCII characters because of Windows limitations. I've tried everything[1] and I'm now resorting to copying the file to a path without ASCII characters before giving it to Hunspell. What is a good location to copy it to?
[1]
Windows requires wchar_t support for std::iostream.open() to work right, which MinGW does not implement
std::filesystem can solve this, but only available in GCC 8
Hunspell insists on loading files on its own, it is not possible to pass the read files as strings to it
The "natural" fit would be the use the user's choosen temporary directory (or subdirectory thereof) (see %temp% or GetTempPath()). However, that defaults to something that contains the user name (which can contain "non-ASCII" characters; e.g. c:\users\Ø¥Ć¼\AppData\LocalLow\Temp) or something arbitrary (regarding character set) all together.
So you're most likely best off to choose some directory that
a) does not contain off-limits characters from the get go. For example, a directory underneath C:\ProgramData that you choose yourself (e.g. the application name) that you know does not contain non-ASCII characters.
b) let the user decide where to put these files and make sure it is not permissible to enter a path that contains only allowed characters.
c) Pass the "short path name" to Hunspell, which should not contain non-ASCII characters for compatibility with FAT file system traits. For example, the short path name for c:\temp\Ø¥Ć¼ is c:\temp\571D~1.
You can see the short names for directories using cmd.exe /c dir /x:
C:\temp>dir /x
...
19.07.2019 15:30 <DIR> .
19.07.2019 15:30 <DIR> ..
19.07.2019 15:30 <DIR> 571D~1 Ø¥Ć¼
How you can invoke the GetShortPathName Win32 API from MinGW I don't know, but I would assume that it is possible.
Also make sure to review the MSDN page for the above function for traitoffs, e.g. short names are not supported everywhere (e.g. SMB + see comments below).
From this bug tracker:
In WIN32 environment, use UTF-8 encoded paths started with the long
path prefix \\?\ to handle system-independent character encoding
and very long path names (without the long path prefix Hunspell will
use fopen() with system-dependent character encoding instead of
_wfopen()).
So the actual solution seems to be:
Call GetFullPathNameW to normalize the path. Required because paths with long path prefix \\?\ are passed to the NT API unchanged.
Prepend L"\\\\?\\" to the normalized path (backslashes doubled because of C string literal requirements).
For a UNC path, you have to use the "UNC" device directly (i. e. L"\\\\server\\share" → L"\\\\?\\UNC\\server\\share" (thanks eryksun)
Encode the path in UTF-8, e. g. using WideCharToMultiByte() with CP_UTF8.
Pass the final UTF-8 encoded path to Hunspell.
It looks like C:\Windows\Temp is still a valid path you can write to yourself.

Haskell Directory creates invalid symlink on Windows

This year System.Directory was updated to include createFileLink and createDirectoryLink actions, and for me on Windows 10 both work fine for relative paths.
When I use either on an absolute path (of about 50 character length, so I suppose in unicode it exceeds 260) it prepends \\?\ (i.e. "\\\\?\\") to the paths, which can be seen from DIR as follows
<SYMLINKD> source [\\?\T:\Code\hLink\binaries\dest]
<SYMLINK> source.txt [\\?\T:\Code\hLink\binaries\dest\source.txt]
The directory link works fine, but the file link doesn't do anything, it doesn't even say that the target file is missing.
When I create a file link using MKLINK without \\?\ in the absolute path it works fine as well, and when I create either link using MKLINKwith \\?\ it has the same result.
Is this a Windows problem? Can I make Haskell use short path format instead? (Using Win10 so apparently I can enable long paths via registry)
Should the Windows api be passing the \\?\ header to symlinks at all?
References:
MaxPath and the meaning of \\?\, plus disabling path limitations on Win10
https://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx#maxpath
Changelog reporting the addition of \\?\ to win32 calls https://hackage.haskell.org/package/directory-1.3.1.1/changelog

What is the longest directory and file name?

I need to optimize my Java application that works with directories and files. I use String[] buffer; to keep file or dir name. So what's the longest possible file or dir name allowed in Windows 8.1?
Filename with path to the file is limited to 260 characters.

Visual Studio 2008 References too long?

I have recently re-organised our source control and found one of our solutions no longer builds. The only error that we get is:
Error 65 Unknown build error, 'The
specified path, file name, or both are
too long. The fully qualified file
name must be less than 260 characters,
and the directory name must be less
than 248 characters.'
I went through each reference in visual studio and the longest complete path (path AND filename) of each reference was 161 characters.
My solution folder structure is thus:
C:\projects_svn\ABC\branches\01.02.03\ABC\ABC
SUITE\ABC.DEF.GHIJKLM.NOP\
Any help would be greatly appreciated, I do not feel this folder structure is too long, given the project size, and organises things much better.
Here are some technical details which may help (if you need more let me know):
Visual Studio 2008 SP 2 running on Windows XP or Windows 7. Using Subversion as SCM. Coding in C#/WPF.
Thanks
I think what's likely happening here is that one of your projects is using a relative hint path that is going very far back up the tree and then back down the directory structure. For example
c:\foo\bar\baz\..\..\..\some\other\dir\foo.dll
Even though the path to the actual file is less than 256 the relative goop makes it much longer.
The best way to find these is to search all of your .csproj / .vbproj files for the string ..\..\. See if that turns up any results.
You also need to consider the "backdrop" files for SVN.
While I might have a short file name like this:
c:\myfolder1\myfolder2\MyFile.txt
There is probably lurking somewhere a longer file name version like this:
c:\myfolder1\myfolder2.svn\text-base\MyFile.txt.svn-base
And that backdrop file is the one that gets the "way too long" error.
Here is what I get via CCNET (calling a MSBUILD file) using SVN source control.
Specific names removed to protect the innocent. (And my job!)
Please note that this "name massaging" results in shorter paths that probably would not generate the errors. Aka, don't count the number of characters in my massaged examples.
But the error messages are what I was getting.
Removing directory "C:\CCNETEnvironment\MyFolder2\MyProject\working\checkout".
C:\src\MyFolder1\MyProject\My_MSBuild.xml(173,5): error MSB3231: Unable to remove directory "prop-base". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
C:\src\MyFolder1\MyProject\My_MSBuild.xml(173,5): error MSB3231: Unable to remove directory "text-base". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
C:\src\MyFolder1\MyProject\My_MSBuild.xml(173,5): error MSB3231: Unable to remove directory "prop-base". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
C:\src\MyFolder1\MyProject\My_MSBuild.xml(173,5): error MSB3231: Unable to remove directory "text-base". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
C:\src\MyFolder1\MyProject\My_MSBuild.xml(173,5): error MSB3231: Unable to remove directory "C:\CCNETEnvironment\MyFolder2\MyProject\working\checkout". Could not find a part of the path 'MyFile.txt.svn-base'.
Done building target "Clean" in project "My_MSBuild.xml" -- FAILED.

Resources