I just stumbled on this article on MSDN that says a path can be 259 characters + NUL termination, but if you prefix it with "\\?\" WinAPI allows you to use the
maximum total path length of 32,767 characters.
Eager to see it working I tried using that prefix from Explorer (On XP SP3) but it doesn't work at all (on any path). If you put \\?\C:\Path\to\an\existing.file on explorer's bar, it will give the "file not found" error.
So I'm confused. Can I code something for (non-ancient) Windows that makes full use of the mentioned path size on NTFS? Why Explorer doesn't use it?
There is a SET of API calls that are work with extened-paths and some that do not. The MSDN usually mentions this.
Not that if you just type that path into windows explorerunder xp this does not work, because the extened path syntax is just an escape sequence for the WIn32 API and not for windows explorer. Now, In Win7 this does work because many people expected this to work.
Also for long paths, it does help if you change the working directory or open up explorer with a sub-directory as a root.
Before someone tells me to RTFM...
Note that these examples are intended for use with the Windows API functions and do not all necessarily work with Windows shell applications such as Windows Explorer.
[...]
For file I/O, the "\\?\" prefix to a path string tells the Windows APIs to disable all string parsing and to send the string that follows it straight to the file system. For example, if the file system supports large paths and file names, you can exceed the MAX_PATH limits that are otherwise enforced by the Windows APIs.
On a secondary note, this makes me wonder about the possibilities of hiding files (or finding such files) from explorer by using illegal file names.
Are you asking why all components in Windows do not support it, or are you asking whether it's legal to use these long paths?
You can definitely safely use them, but you may irritate someone who wants to use tools like Explorer to browse them. We see paths like this all the time in the wild. Sometimes people are pretty surprised when they can't use MY_FAVORITE_TOOL to delete it...
Related
The title pretty much says it, on Windows, can I somehow simulate multiple working directories in a multithreaded application by using something similar to openat?
See also this question.
Yes there is, and it's called NtCreateFile() (https://msdn.microsoft.com/en-us/library/bb432380(v=vs.85).aspx) :)
openat() takes an open fd to a base directory from which path operations start. Similarly, you can supply a HANDLE to the ObjectAttributes.RootDirectory of NtCreateFile() to have whatever that directory's path is to be used as a base for relative path operations.
If going direct to the NT kernel API is too much for you and you want to remain within Win32, you can retrieve the current path of the base directory HANDLE easily enough (see https://msdn.microsoft.com/en-us/library/windows/desktop/aa366789(v=vs.85).aspx). If you open the directory without FILE_SHARE_DELETE permissions, nobody will be able to rename it and therefore the path retrieved will never move so long as you keep the HANDLE open.
You then stitch together the retrieved path with the relative path using normal string concatenation.
The NT kernel API approach is the only solution on Windows which allows the base directory HANDLE to be arbitrarily renamed by third party processes and that not cause races in your code. i.e. it's the only equivalent to POSIX openat(). I agree it's very unfortunate that Win32 doesn't make available this facility - in fact, Win32 doesn't make available atomic renames which is a very useful POSIX facility indeed to let you avoid lock files and is something the NT kernel API provides too.
I store the MRU of logins to my application in a file called login.ini and I save it in widnows application folders.
I noticed that on some systems — I don't know why; I cannot find a common cause — the user cannot create the file, whereas it creates all other files in the same folder.
The only reason I can think of is that some antivirus/windows setting/... doesn't allow this particular user to create the file on this system.
I solved the problem by renaming the file and it seems ok, but I'd like to be sure. Does anyone know more?
Note for bounty:
This is a related question I asked that details a little more what I am doing.
A little Google-fu turns up that other Windows developers have sucessfully created login.ini for their programs, and others use it in a third-party Windows login management program, so I would expect that its "reservedness" is partially dependent on its location in the file system (i.e. in the system files). However, I don't think the name "login.ini" is a system-wide reserved name, no.
I think you're right - certain antivirus programs MAY be messing with the creation of that file, as it is a fairly likely candidate imho for a virus filename. It looks as if it may already have been used for that purpose somewhere (apparently outside of the US), tho don't quote me on that.
So, if a different name works for you, I'd go with that. :)
Anti-virus is a definite possibility for messing with your file. Stuff like that happened all the time to me when I was using Norton.
'login.ini' is not a system-wide reserved name, it would only mess things up with the OS if you had it in the (assuming your drive is C:) C:\WINDOWS or C:\WINDOWS\System32 directories.
If you just have the file in an application files directory (like C:\Program Files or C:\All Users\Application Settings and such) it shouldn't interfere with the system.
If you determine that anti-virus is a definite problem, you could change the name to something like loginData and maybe make up a new file extension if you want to (assuming you are just going to read the file from a program, where the extension doesn't matter. otherwise stick to a recognized file extension)
I am in the middle of writing a tool that finds lost files of an iTunes library, for both Mac and Windows. On the Mac, I can quickly find files by naming using the wonderful "CatalogSearch" function.
On Windows, however, there seems to be no OS API for searching by file name (or is there?).
After some googling, I learned that there are tools (like TFind, Everything) that read the NTFS directory directly and scan it to find files by name.
I would like to do the same, but without having to start from scratch (although I've written quite a few disk tools in the past, I've never had the energy to dig into NTFS).
I wonder if there are ready-made libs around, possibly as a .dll, that would give me this search feature: Pass in a file name, get back its path.
Alternatively, what about the Windows indexing service? At least when I tried this on a recently installed XP Home system, the Search operation under the Start menu would actually scan all directories, which suggests that it has no complete database. As I'm not a Windows user at all, I wonder why this isn't working.
In the end, the complete solution I need is: I have a list of file names to find, and I need code that searches the entire disk (or uses a DB for it) to get me all results in one go. E.g, the search should not start a new full scan for every file I'm looking up. That's why I think the MFT way would be optimal, as it could quickly iterate over all names, comparing each to my list.
The best way to solve your problem seems to be by using the Windows Change Journal.
Problem: If it is not enabled for a volume or the volume is a non-NTFS you need a fallback (or enable the Change Journal if it is NTFS). You need administrator rights as well to access the Change Journal.
You get the files by using the FSCTL_ENUM_USN_DATA and DeviceIOControll with LowUsn=0. This directly accesses the MFT and writes all filenames into the supplied buffer. Because it sequentially acesses the MFT it is faster than the FindFirstFile API.
Linux has a feature called namespaces, which let you give a different "view" of the filesystem to different processes. In Windows terms, this would be useful for example if you had a legacy program "floyd" that always loaded its configuration from C:\floyd\floyd.ini. If Windows had namespaces, you could write a wrapper script which would create a namespace in which to run floyd, making it so when Alice ran the script, floyd would start up in an environment where C:\floyd existed but actually pointed to C:\Users\Alice\Floyd.
Now you may be thinking, "OK, just use soft or hard links and make C:\floyd an alias for C:\Users\Alice." But with namespaces, Bob can also run the startup script, but his instance of floyd (on the same computer, running at the same time) will see C:\floyd with the contents of, say, C:\Users\Bob\Program Settings\Floyd Config (or any other path we like).
You can do this on Linux with namespaces. Is there something similar or analogous on Windows? It's fine if it requires writing a C program, and it's OK if it only works on recent versions of Windows.
NTFS hard links are really a simple case of reparse points. Reparse points are typed, and can include more advanced behavior. For instance, they're also used for "offline storage" (transparent migration of files to and from secondary storage). You can therefore also use reparse points to implement per-user symbolic links, by creating a new reparse type.
The reparse point type even has an explicit "Name surrogate" bit, which (if set) indicates that reparse points of those types are some kind of symbolic link.
You can even have multiple reparse points in a path. Therefore files inside your symbolic namepace can still be migrated to secondary storage - you'd just have two reparse points in the path.
I think Virtual Store does this automatically for legacy programs that try to write to nonstandard directories. So the legacy program writes to a user- and program-specific directory instead to C:\floyd.
This sounds like Windows Vista's file system virtualization. For example, it can silently redirect c:\Program Files\Floyd to c:\Users\<username>\AppData\Local\VirtualStore\Program Files\Floyd. However, file system virtualization isn't nearly as configurable as Linux namespaces. From what I can tell from reading, file system virtualization should apply any time a 32-bit interactive process opens for writing a file, folder, or registry key that's only writable by administrators. (So you typically end up with some read-only files under c:\Program Files and some per-user writable files under c:\Users\<username>\AppData\Local\VirtalStore.)
An application virtualization product can probably also do this, although those are often more complicated and more expensive.
You can use hardlinks for that, but only with NTFS. http://en.wikipedia.org/wiki/Hard_link
I think windows doesn't have virtual FS view per process.
The most relevant thing is probably special environment folders, such as %temp%, %appdata%, %localappdata%. Not that they're equivalent, but they fulfil the same purpose.
You can define your own environment variables then use '%myspecialplace%\myfile.txt' to access them.
As a horrid kludge (and here I book my passage to programming hell) could you use a NamedPipe for C:\Floyd that mapped the IO operations onto a file specific to the current process user?
I know it's not pretty and I don't know enough about NamedPipes (FIFOs in other dialects) on Windows to know how feasible this is.
Dan
There are several things that come to my mind.
First of all you can create a file system filter driver (or use a ready driver, such as our CallbackFilter product) that would redirect all file system calls, coming from the application, to other location. This is close to virtualization that you mentioned but this won't change the list of drive letters though. Such approach is both powerful and non-trivial, so see the other option first of all.
And the other option is:
there exist several products (Thinstall, Molebox if memory serves) that "wrap" the application redirecting it's file I/O to some other location. There was also some SDK to do the same, but I don't remember it's name at all.
e.g. http://www.msigeek.com/4819/file-re-direction-using-correctfilepaths-shim-to-fix-broken-applications
But I think it's not configurable per user, although the target can vary per-user based on environment variable substitution.
Most programs, though, store configuration in the registry, in which case RegOverridePredefKey would do the trick.
There's a shortage of good solutions for this. For simplicity, I can't improve on using NTFS soft links (junctions) for this - as you correctly point out, this creates issues if you want per-user configuration. As MSalters correctly says, all NTFS soft and hard links are just special cases of reparse points, so you could do something more general by impplementing a new reparse type, if you don't mind some work digging into NTFS..
(Junction is a pretty useful tool when experimenting with NTFS soft links: http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx )
You could just take a direct approach - give each user (or your program initialization if you only care about one piece of software) a logon script that sets up the appropriate junction into their user directory (and make sure you clean it up afterwards). But it's clumsy.
In general the right Windows approach is to put things into the folders pointed at by the %localappdata% (from Vista on) and, more generally, %userprofile% system variables.
Win file system virtualization is intended to ensure this in the cases where it applies.
I have a VB6 desktop application that is deployed on well over 1200 desktops. The devices throughout are a mix of Windows XP SP2 and SP3 systems. All but one of these PCs (XP SP2) is able to successfully decipher the DOS 8.3 path (ie C:\PROGRA~1\DATFOL~1\Config\) that is used in an .ini file related to this application. This particular PC errors out with a message: "Run-time error '76': Path not found".
The string is obtained from the .ini file using the
GetPrivateProfileString function. (The string is not hard-coded into the application - it is obtained from an ini file).
Since there is only one machine having the problem, I'm looking towards some configuration value on that device as being the root cause. I looked at the NtfsDisable8dot3NameCreation setting in the registry to see if this might cause the issue, but I have been unable to reproduce the problem on any other machine when changing this setting.
Anybody have any thoughts or perhaps another direction I could take?
Don't use hard-coded paths or short filenames. The Program Files folder might not be on the C: drive, might not be named Program Files, and even if it is, might not have a short filename of PROGRA~1 (and the same for DATAFOL~1). Write the install path to an INI file or the registry during installation and read+use that in your program.
If someone was gimping around and made a temp/backup/testing \DataFolder_Temp, deleted the original then renamed, the short path would be DATAFOL~2.
Delete the directory and recreate it.
check the PC. The PROGRA~1 or DATFOL~1 might actually be ~2 instead. Put the 8.3 name used in your code into explorer and see what IT tells you.