The MSDN document Naming Files, Paths, and Namespaces talks about the \\?\ prefix. To quote:
For file I/O, the "\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system.
Experimentation showed me that the \??\ prefix has the same effect, both disabling path parsing (.. handling) and enabling paths longer than MAX_PATH.
The MSDN refers to \\? as the "Win32 file namespace", so does it known purely by the Win32 usermode API and translated to \?? in the NT namespace? And anyway, through Winobj I see GLOBAL?? in the NT namespace, not ??.
The answer to your question is, yes there is a difference between passing \\?\ and \??\ to user mode functions.
Internally, NT always represents paths with the \??\ prefix. Normally, when you call a user mode function (e.g. CreateDirectoryW) with a normal path like C:\foo, the user mode functions call an internal function named RtlDosPathNameToNtPathName_U which converts this to an NT-style path prefixed with \??\. This conversion is done with a fixed size static buffer, which is where the famous MAX_PATH limitation comes from.
When you call a user mode function specifying the \\?\ prefix (note, only one ?), RtlDosPathNameToNtPathName_U is not called. Instead, the second back-slash is turned into a ? character and the path is used verbatim. This is what the docs mean when they talk about \\?\ turning off the "...automatic expansion of the path string."
However, when you call a user mode function with the \??\ prefix, which remember is an internal NT prefix, this expansion is still done.
The user mode functions specifically look for \\?\ to disable the automatic expansion process, and since you're not providing it, your path is treated as a non-prefixed path and fed to RtlDosPathNameToNtPathName_U. This function is smart enough not to add an extra \??\ prefix to the start of the path, however the fixed size static buffer is still used.
This is the key difference. When you pass \??\ as a prefix your paths are still subject to the MAX_PATH length limit.
The following example program demonstrates this. The TestNestedDir function simply attempts to create (and then delete) a path greater than MAX_PATH characters in length, one level at a time. The results you'll see if you run this code are:
CreateDir, no prefix = 0
CreateDir, prefix \\?\ = 1
CreateDir, prefix \??\ = 0
Only the creation done with the \\?\ prefix is successful.
#include <stdio.h>
#include <tchar.h>
#include <string>
#include <assert.h>
#include <Windows.h>
const wchar_t* pszLongPath =
L"C:\\"
L"12345678901234567890123456789012345678901234567890\\"
L"12345678901234567890123456789012345678901234567890\\"
L"12345678901234567890123456789012345678901234567890\\"
L"12345678901234567890123456789012345678901234567890\\"
L"12345678901234567890123456789012345678901234567890\\"
L"12345678901234567890123456789012345678901234567890";
bool TestCreateNestedDir(LPCWSTR pszPath)
{
std::wstring strPath = pszPath;
std::wstring::size_type pos = 0, first = std::wstring::npos;
bool fDirs = false, fResult = false;
// step through each level in the path, but only try to start creating directories
// after seeing a : character
while ((pos = strPath.find_first_of(L'\\', pos)) != std::wstring::npos)
{
if (fDirs)
{
// get a substring for this level of the path
std::wstring strSub = strPath.substr(0, pos);
// check if the level already exists for some reason
DWORD dwAttr = ::GetFileAttributesW(strSub.c_str());
if (dwAttr != -1 && (dwAttr & FILE_ATTRIBUTE_DIRECTORY))
{
++pos;
continue;
}
// try to make the dir. if it exists, remember the first one we successfully made for later cleanup
if (!::CreateDirectoryW(strSub.c_str(), nullptr))
break;
if (first == std::wstring::npos) first = pos;
}
else
if (pos > 0 && strPath[pos - 1] == L':')
fDirs = true;
++pos;
}
if (pos == std::wstring::npos)
{
// try to create the last level of the path (we assume this one doesn't exist)
if (::CreateDirectoryW(pszPath, nullptr))
{
fResult = true;
::RemoveDirectoryW(pszPath);
}
}
else
--pos;
// now delete any dirs we successfully made
while ((pos = strPath.find_last_of(L'\\', pos)) != std::wstring::npos)
{
::RemoveDirectoryW(strPath.substr(0, pos).c_str());
if (pos == first) break;
--pos;
}
return fResult;
}
int _tmain(int argc, _TCHAR* argv[])
{
assert(wcslen(pszLongPath) > MAX_PATH);
printf("CreateDir, no prefix = %ld\n", TestCreateNestedDir(pszLongPath));
std::wstring strPrefix = L"\\\\?\\" + std::wstring(pszLongPath);
printf("CreateDir, prefix \\\\?\\ = %ld\n", TestCreateNestedDir(strPrefix.c_str()));
strPrefix[1] = L'?';
printf("CreateDir, prefix \\??\\ = %ld\n", TestCreateNestedDir(strPrefix.c_str()));
return 0;
}
I am trying to open file which reside in a directory.
It is always return error code 13 if length is above or equal 127.
char name[200]="E:\\suri_temp\\abc85\\tool\\src1111\\turi_temp\\abc85\\tool\\src1111\\puri_temp\\abc85\\tool\\src\\nuri_temp\\abc85\\to\\abcd.tmp\\suri1111.log";
int len = strlen(name); //len=127
HFILE handle ;
WORD temp;
OFSTRUCT ofstruct;
if( (handle = OpenFile(name, &ofstruct, OF_EXIST)) == HFILE_ERROR )
{
temp = GetLastError(); // if length 127 or above(it comes here temp = 13)
}
else
_lclose(handle); // if length is below 127 it comes here
Anyone faced this problem?
You are just finding out the hard way what's well documented in the MSDN article for OpenFile:
Note This function has limited capabilities and is not recommended. For new application development, use the CreateFile function.
The OFSTRUCT structure contains a path string member with a length that is limited to OFS_MAXPATHNAME characters, which is 128 characters. Because of this, you cannot use the OpenFile function to open a file with a path length that exceeds 128 characters. The CreateFile function does not have this path length limitation.
As indicated, use CreateFile() instead.
Using
OpenFile(name, &ofstruct, OF_EXIST))
is ok in debug mode, but fails in release mode.
Use
if(GetFileAttributes("filename") == -1)
I am trying to delete all the temporary files created by my application during uninstall. I use the following code:
bool DeleteFileNow( QString filenameStr )
{
wchar_t* filename;
filenameStr.toWCharArray(filename);
QFileInfo info(filenameStr);
// don't do anything if the file doesn't exist!
if (!info.exists())
return false;
// determine the path in which to store the temp filename
wchar_t* path;
info.absolutePath().toWCharArray(path);
TRACE( "Generating temporary name" );
// generate a guaranteed to be unique temporary filename to house the pending delete
wchar_t tempname[MAX_PATH];
if (!GetTempFileNameW(path, L".xX", 0, tempname))
return false;
TRACE( "Moving real file name to dummy" );
// move the real file to the dummy filename
if (!MoveFileExW(filename, tempname, MOVEFILE_REPLACE_EXISTING))
{
// clean up the temp file
DeleteFileW(tempname);
return false;
}
TRACE( "Queueing the OS" );
// queue the deletion (the OS will delete it when all handles (ours or other processes) close)
return DeleteFileW(tempname) != FALSE;
}
My application is crashing. I think its due to some missing windows dll for the operations performed. Is there any other way to perform the same operation using Qt alone?
Roku have already told your problem in manipulating with QString and wchar_t*.
See the documentation: QString Class Reference, method toWCharArray:
int QString::toWCharArray ( wchar_t * array ) const
Fills the array with the data contained in this QString object. The array is encoded in utf16 on platforms where wchar_t is 2 bytes wide (e.g. windows) and in ucs4 on platforms where wchar_t is 4 bytes wide (most Unix systems).
array has to be allocated by the caller and contain enough space to hold the complete string (allocating the array with the same length as the string is always sufficient).
returns the actual length of the string in array.
If you are simply looking for a way to remove a file using Qt, use QFile::remove:
QFile file(fileNameStr);
file.remove(); // Returns a bool; true if successful
If you want Qt to manage the entire life cycle of a temporary file for you, take a look at QTemporaryFile:
QTemporaryFile tempFile(fileName);
if (tempFile.open())
{
// Do stuff with file here
}
// When tempFile falls out of scope, it is automatically deleted.
I am trying to understand WideCharToMultiByte and I was wondering when lpUsedDefaultChar would be set to be TRUE.
Here is a sample: What should be lpszW inorder for the flag to be set to be true?
lpszW = L”__WHAT SHOULD_BE_HERE__”;
int c = ??;
BOOL fUsedDefaultChar = false;
DWORD dwSize = WideCharToMultiByte(CP_ACP, 0, lpszW, c, myOutStr ,myOutLen, NULL, &fUsedDefaultChar);
http://msdn.microsoft.com/en-us/library/dd374130(VS.85).aspx
Any books/tutorials for understanding Unicode/UTF stuff would be great.
Thanks!
Anything that is not present in the current codepage will map to ? (by default) and UsedDefaultChar will be != FALSE.
Windows-1252 is probably the most common codepage and most of those characters map to the same value in unicode.
Take Ω (ohm) for example, it is probably not present in whatever your current codepage is and therefore will not map to a valid narrow character:
BOOL fUsedDefaultChar=FALSE;
DWORD dwSize;
char myOutStr[MAX_PATH];
WCHAR lpszW[10]=L"Hello";
*lpszW=0x2126; //ohm sign, you could also use the \u2126 syntax if your compiler supports it.
dwSize = WideCharToMultiByte(CP_ACP, 0, lpszW, -1, myOutStr ,MAX_PATH, NULL, &fUsedDefaultChar);
printf("%d %s\n",fUsedDefaultChar,myOutStr); //This prints "1 ?ello" on my system
The MSDN documentation is very clear about when lpUsedDefaultChar is set to TRUE:
lpDefaultChar [in] Optional. Pointer
to the character to use if a character
cannot be represented in the specified
code page. The application sets this
parameter to NULL if the function is
to use a system default value. To
obtain the system default character,
the application can call the GetCPInfo
or GetCPInfoEx function.
lpUsedDefaultChar [out] Optional.
Pointer to a flag that indicates if
the function has used a default
character in the conversion. The flag
is set to TRUE if one or more
characters in the source string cannot
be represented in the specified code
page. Otherwise, the flag is set to
FALSE. This parameter can be set to
NULL.
That does not leave much room for misunderstanding, in my opinion.
The GetModuleFileName() takes a buffer and size of buffer as input; however its return value can only tell us how many characters is has copied, and if the size is not enough (ERROR_INSUFFICIENT_BUFFER).
How do I determine the real required buffer size to hold entire file name for GetModuleFileName()?
Most people use MAX_PATH but I remember the path can exceed that (260 by default definition)...
(The trick of using zero as size of buffer does not work for this API - I've already tried before)
The usual recipe is to call it setting the size to zero and it is guaranteed to fail and provide the size needed to allocate sufficient buffer. Allocate a buffer (don't forget room for nul-termination) and call it a second time.
In a lot of cases MAX_PATH is sufficient because many of the file systems restrict the total length of a path name. However, it is possible to construct legal and useful file names that exceed MAX_PATH, so it is probably good advice to query for the required buffer.
Don't forget to eventually return the buffer from the allocator that provided it.
Edit: Francis points out in a comment that the usual recipe doesn't work for GetModuleFileName(). Unfortunately, Francis is absolutely right on that point, and my only excuse is that I didn't go look it up to verify before providing a "usual" solution.
I don't know what the author of that API was thinking, except that it is possible that when it was introduced, MAX_PATH really was the largest possible path, making the correct recipe easy. Simply do all file name manipulation in a buffer of length no less than MAX_PATH characters.
Oh, yeah, don't forget that path names since 1995 or so allow Unicode characters. Because Unicode takes more room, any path name can be preceeded by \\?\ to explicitly request that the MAX_PATH restriction on its byte length be dropped for that name. This complicates the question.
MSDN has this to say about path length in the article titled File Names, Paths, and Namespaces:
Maximum Path Length
In the Windows API (with some
exceptions discussed in the following
paragraphs), the maximum length for a
path is MAX_PATH, which is defined as
260 characters. A local path is
structured in the following order:
drive letter, colon, backslash,
components separated by backslashes,
and a terminating null character. For
example, the maximum path on drive D
is "D:\<some 256 character path
string><NUL>" where "<NUL>" represents
the invisible terminating null
character for the current system
codepage. (The characters < > are used
here for visual clarity and cannot be
part of a valid path string.)
Note File I/O functions in the
Windows API convert "/" to "\" as part
of converting the name to an NT-style
name, except when using the "\\?\"
prefix as detailed in the following
sections.
The Windows API has many functions
that also have Unicode versions to
permit an extended-length path for a
maximum total path length of 32,767
characters. This type of path is
composed of components separated by
backslashes, each up to the value
returned in the
lpMaximumComponentLength parameter of
the GetVolumeInformation function. To
specify an extended-length path, use
the "\\?\" prefix. For example,
"\\?\D:\<very long path>". (The
characters < > are used here for
visual clarity and cannot be part of a
valid path string.)
Note The maximum path of 32,767
characters is approximate, because the
"\\?\" prefix may be expanded to a
longer string by the system at run
time, and this expansion applies to
the total length.
The "\\?\" prefix can also be used
with paths constructed according to
the universal naming convention (UNC).
To specify such a path using UNC, use
the "\\?\UNC\" prefix. For example,
"\\?\UNC\server\share", where "server"
is the name of the machine and "share"
is the name of the shared folder.
These prefixes are not used as part of
the path itself. They indicate that
the path should be passed to the
system with minimal modification,
which means that you cannot use
forward slashes to represent path
separators, or a period to represent
the current directory. Also, you
cannot use the "\\?\" prefix with a
relative path, therefore relative
paths are limited to MAX_PATH
characters as previously stated for
paths not using the "\\?\" prefix.
When using an API to create a
directory, the specified path cannot
be so long that you cannot append an
8.3 file name (that is, the directory name cannot exceed MAX_PATH minus 12).
The shell and the file system have
different requirements. It is possible
to create a path with the Windows API
that the shell user interface might
not be able to handle.
So an easy answer would be to allocate a buffer of size MAX_PATH, retrieve the name and check for errors. If it fit, you are done. Otherwise, if it begins with "\\?\", get a buffer of size 64KB or so (the phrase "maximum path of 32,767 characters is approximate" above is a tad troubling here so I'm leaving some details for further study) and try again.
Overflowing MAX_PATH but not beginning with "\\?\" appears to be a "can't happen" case. Again, what to do then is a detail you'll have to deal with.
There may also be some confusion over what the path length limit is for a network name which begins "\\Server\Share\", not to mention names from the kernel object name space which begin with "\\.\". The above article does not say, and I'm not certain about whether this API could return such a path.
Implement some reasonable strategy for growing the buffer like start with MAX_PATH, then make each successive size 1,5 times (or 2 times for less iterations) bigger then the previous one. Iterate until the function succeeds.
Using
extern char* _pgmptr
might work.
From the documentation of GetModuleFileName:
The global variable _pgmptr is automatically initialized to the full path of the executable file, and can be used to retrieve the full path name of an executable file.
But if I read about _pgmptr:
When a program is not run from the command line, _pgmptr might be initialized to the program name (the file's base name without the file name extension) or to a file name, relative path, or full path.
Anyone who knows how _pgmptr is initialized? If SO had support for follow-up questions I would posted this question as a follow up.
While the API is proof of bad design, the solution is actually very simple. Simple, yet sad it has to be this way, for it's somewhat of a performance hog as it might require multiple memory allocations. Here is some keypoints to the solution:
You can't really rely on the return value between different Windows-versions as it can have different semantics on different Windows-versions (XP for example).
If the supplied buffer is too small to hold the string, the return value is the amount of characters including the 0-terminator.
If the supplied buffer is large enough to hold the string, the return value is the amount of characters excluding the 0-terminator.
This means that if the returned value exactly equals the buffer size, you still don't know whether it succeeded or not. There might be more data. Or not. In the end you can only be certain of success if the buffer length is actually greater than required. Sadly...
So, the solution is to start off with a small buffer. We then call GetModuleFileName passing the exact buffer length (in TCHARs) and comparing the return result with it. If the return result is less than our buffer length, it succeeded. If the return result is greater than or equal to our buffer length, we have to try again with a larger buffer. Rinse and repeat until done. When done we make a string copy (strdup/wcsdup/tcsdup) of the buffer, clean up, and return the string copy. This string will have the right allocation size rather than the likely overhead from our temporary buffer. Note that the caller is responsible for freeing the returned string (strdup/wcsdup/tcsdup mallocs memory).
See below for an implementation and usage code example. I have been using this code for over a decade now, including in enterprise document management software where there can be a lot of quite long paths. The code can ofcourse be optimized in various ways, for example by first loading the returned string into a local buffer (TCHAR buf[256]). If that buffer is too small you can then start the dynamic allocation loop. Other optimizations are possible but that's beyond the scope here.
Implementation and usage example:
/* Ensure Win32 API Unicode setting is in sync with CRT Unicode setting */
#if defined(_UNICODE) && !defined(UNICODE)
# define UNICODE
#elif defined(UNICODE) && !defined(_UNICODE)
# define _UNICODE
#endif
#include <stdio.h> /* not needed for our function, just for printf */
#include <tchar.h>
#include <windows.h>
LPCTSTR GetMainModulePath(void)
{
TCHAR* buf = NULL;
DWORD bufLen = 256;
DWORD retLen;
while (32768 >= bufLen)
{
if (!(buf = (TCHAR*)malloc(sizeof(TCHAR) * (size_t)bufLen))
{
/* Insufficient memory */
return NULL;
}
if (!(retLen = GetModuleFileName(NULL, buf, bufLen)))
{
/* GetModuleFileName failed */
free(buf);
return NULL;
}
else if (bufLen > retLen)
{
/* Success */
LPCTSTR result = _tcsdup(buf); /* Caller should free returned pointer */
free(buf);
return result;
}
free(buf);
bufLen <<= 1;
}
/* Path too long */
return NULL;
}
int main(int argc, char* argv[])
{
LPCTSTR path;
if (!(path = GetMainModulePath()))
{
/* Insufficient memory or path too long */
return 0;
}
_tprintf("%s\n", path);
free(path); /* GetMainModulePath malloced memory using _tcsdup */
return 0;
}
Having said all that, I like to point out you need to be very aware of various other caveats with GetModuleFileName(Ex). There are varying issues between 32/64-bit/WOW64. Also the output is not necessarily a full, long path, but could very well be a short-filename or be subject to path aliasing. I expect when you use such a function that the goal is to provide the caller with a useable, reliable full, long path, therefor I suggest to indeed ensure to return a useable, reliable, full, long absolute path, in such a way that it is portable between various Windows-versions and architectures (again 32/64-bit/WOW64). How to do that efficiently is beyond the scope here.
While this is one of the worst Win32 APIs in existance, I wish you alot of coding joy nonetheless.
My example is a concrete implementation of the "if at first you don't succeed, double the length of the buffer" approach. It retrieves the path of the executable that is running, using a string (actually a wstring, since I want to be able to handle Unicode) as the buffer. To determine when it has successfully retrieved the full path, it checks the value returned from GetModuleFileNameW against the value returned by wstring::length(), then uses that value to resize the final string in order to strip the extra null characters. If it fails, it returns an empty string.
inline std::wstring getPathToExecutableW()
{
static const size_t INITIAL_BUFFER_SIZE = MAX_PATH;
static const size_t MAX_ITERATIONS = 7;
std::wstring ret;
DWORD bufferSize = INITIAL_BUFFER_SIZE;
for (size_t iterations = 0; iterations < MAX_ITERATIONS; ++iterations)
{
ret.resize(bufferSize);
DWORD charsReturned = GetModuleFileNameW(NULL, &ret[0], bufferSize);
if (charsReturned < ret.length())
{
ret.resize(charsReturned);
return ret;
}
else
{
bufferSize *= 2;
}
}
return L"";
}
Here is a another solution with std::wstring:
DWORD getCurrentProcessBinaryFile(std::wstring& outPath)
{
// #see https://msdn.microsoft.com/en-us/magazine/mt238407.aspx
DWORD dwError = 0;
DWORD dwResult = 0;
DWORD dwSize = MAX_PATH;
SetLastError(0);
while (dwSize <= 32768) {
outPath.resize(dwSize);
dwResult = GetModuleFileName(0, &outPath[0], dwSize);
dwError = GetLastError();
/* if function has failed there is nothing we can do */
if (0 == dwResult) {
return dwError;
}
/* check if buffer was too small and string was truncated */
if (ERROR_INSUFFICIENT_BUFFER == dwError) {
dwSize *= 2;
dwError = 0;
continue;
}
/* finally we received the result string */
outPath.resize(dwResult);
return 0;
}
return ERROR_BUFFER_OVERFLOW;
}
Windows cannot handle properly paths longer than 260 characters, so just use MAX_PATH.
You cannot run a program having path longer than MAX_PATH.
My approach to this is to use argv, assuming you only want to get the filename of the running program. When you try to get the filename from a different module, the only secure way to do this without any other tricks is described already, an implementation can be found here.
// assume argv is there and a char** array
int nAllocCharCount = 1024;
int nBufSize = argv[0][0] ? strlen((char *) argv[0]) : nAllocCharCount;
TCHAR * pszCompleteFilePath = new TCHAR[nBufSize+1];
nBufSize = GetModuleFileName(NULL, (TCHAR*)pszCompleteFilePath, nBufSize);
if (!argv[0][0])
{
// resize memory until enough is available
while (GetLastError() == ERROR_INSUFFICIENT_BUFFER)
{
delete[] pszCompleteFilePath;
nBufSize += nAllocCharCount;
pszCompleteFilePath = new TCHAR[nBufSize+1];
nBufSize = GetModuleFileName(NULL, (TCHAR*)pszCompleteFilePath, nBufSize);
}
TCHAR * pTmp = pszCompleteFilePath;
pszCompleteFilePath = new TCHAR[nBufSize+1];
memcpy_s((void*)pszCompleteFilePath, nBufSize*sizeof(TCHAR), pTmp, nBufSize*sizeof(TCHAR));
delete[] pTmp;
pTmp = NULL;
}
pszCompleteFilePath[nBufSize] = '\0';
// do work here
// variable 'pszCompleteFilePath' contains always the complete path now
// cleanup
delete[] pszCompleteFilePath;
pszCompleteFilePath = NULL;
I had no case where argv didn't contain the file path (Win32 and Win32-console application), yet. But just in case there is a fallback to a solution that has been described above. Seems a bit ugly to me, but still gets the job done.