Windows has several "virtual folders" and "special folders" (Wikipedia's names for them) that change the way the file system works. For example, the Desktop folder acts like the root of the file system hierarchy in some circumstances, even though C:/ is the drive, also called the partition, that the entire file system is on, and therefore logically Desktop should be under C:/. Screenshots: There are also the Library folders. Also, with msysGit and Cygwin on your computer, the program's folder then magically becomes the root of the file system.
I'm sure there are instances on other OS's, but I only really work with Windows, although it seems Linux is much less abstracted.
How do applications do this? Is it just a really complicated shell script, or something that only affects certain programs (Wikipedia says Desktop is only the root folder in Windows Explorer, for example)? Is it similar to how on Linux all folders are "mounted", even the root folder? Even if the OS developers don't usually give away their secrets, I would still like to know the basic method, if possible.
So, wikipedia's entry for special folder says that:
The Desktop virtual folder is the root of the Windows Shell namespace,
which contains other virtual folders.
This is not the same as saying it's "the root of the file system hierarchy". Windows has a set of set of virtual folders. All of these are nested under Desktop.
For Win7 and up, the default filesystem layout is actually:
Desktop -> C:\Users\(username)\Desktop
My Documents -> C:\Users\(username)\Documents
Downloads -> C:\Users\(username)\Downloads
These aliases mean something to the explorer shell, but not at the filesystem level (open cmd.exe and see what I mean).
At the filesystem level, and to an application, the filesystem path is what is going to matter. It takes extra API calls (the use of SHGetSpecialFolderPath()) to figure out where on the filesystem that special folder lives; the app only cares about where in the filesystem it needs to target.
As for Cygwin, I strongly suspect that it's doing something totally different. Unix has a special call, chroot(), that really does modify the root for the currently running shell. This means that applications running under that shell have absolutely no knowledge of the directory structure below the chroot()'d path, and this is generally done for security reasons. In fact, this is usually referred to as a chroot jail, because it isn't an alias or a virtual directory, and can't be escaped from (unless there's a bug in the kernel).
Related
So I have quite a few "Grab'n'Go" Windows games and programs that I've stuffed onto an external harddrive, and it is a little daunting to have all their saves end up on my C: drive. What I'm looking to do, and I hope this can be done on Windows XP at the least (if not 7 or 10), is to create a BAT file to trick each program to save their files somewhere else but C:\Documents\User\My Documents, and preferably a different folder for each program (Dungeon Siege to G:\DS_saves, RCT3 to H:\RCT3, etc).
Is such a thing possible? And is it a different command per Windows version?
You can't "trick" games while they're running, since you have no access to their code or the running threads.
However, you can accomplish what you're trying to do by simply right-clicking the Documents folder, go to Properties, then Location and change the target folder.
This will change the directory for Documents in your system, and your games/apps will use the new folder even if it's on a different drive.
This can be done for all system default directories, and it's very useful for SSD-based machines where space is essential.
Actual question
How can I "redirect" (symbolic links?) or temporarily change the values of %HOMEDRIVE% and %HOMEPATH% (maybe based on two batch scripts that change the values when I start working at the machine and reset everything after I'm done)?
Background
A lot of Windows programs pick up either system variable %USERPROFILE% or a combination of %HOMEDRIVE% and %HOMEPATH in order to figure out where a user's "home" directory is that they need to use (e.g. C:\Users\JohnDoe). Being a fan of the philosophy of portable apps (see at the bottom what I mean by that), I'm wondering if I could somehow make my home directory portable, i.e. have it point to a directory on a portable device. Or, in other words, I would like to make it "dynamically changeable".
What I tried
I've read a bit about Set and Setx and tried this:
Finding out what the current variable values are:
C:\Users\JohnDoe> Set HOME
HOMEDRIVE=C:
HOMEPATH=\Users\<username>
Set them:
C:\Users\JohnDoe> Setx HOMEDRIVE "d:"
C:\Users\JohnDoe> Setx HOMEPATH "\home"
That creates HOMEDIR and HOMEPATH as user-specific environment variables (see Control Panel >> System >> Environment Variables). But the "global ones" did not change and e.g. RStudio will not recognize the user-specific variables (still writes to C:\Users\JohnDoes\[...]).
C:\Users\JohnDoe> Set HOME
HOMEDRIVE=C:
HOMEPATH=\Users\<username>
Portable Apps
For example, see PortableApps.com or simply think "software that simply has been installed to a portable device instead of C:\Program Files\<app-name>". The latter works a treat for me for programs that don't really care about %HOMEDRIVE% and %HOMEPATH% (e.g. Eclipse or R).
Some others (e.g. Git) just need to be tweaked a little bit (e.g. see this post).
Yet some others (e.g. RStudio) run in fact smoothly off the portable device, but write stuff to the user's "home" directory. That's the kind of software that I would like to tell to use a directory on my portable device instead.
While you can change the environment variables this wont work for all applications, as there is also a Shell folder API in Windows which gives various user specific locations.
So it is better to use the actual Windows method to change the profile. And no this will not be possible to be a mobile device as it expects a stable and available location for the registry database of the user.
I think when setting up windows (and with sysprep) one can specify a location for the }users} folder (different drive). The quickest thing to move are the special library folders, you can change their location in the Explorer Properties view.
I have a big collection of folders for projects I'm working on. I've been trying to find a better way to sort them all for a long time and I want to write an app that creates groups based on whatever criteria I say, such as "folders from 2011" or "folders containing a x type of file" etc.
This is fairly straightforward, and wouldn't present much of a problem to code using its own UI in winForms or WPF or something. But I think it would be far better if I could make these folders appear to be part of the filesystem, so other apps (like existing file explorers) can see them.
Is this possible? Would it cause problems I haven't considered? How do I go about doing it if it is possible?
One way I thought of doing it would be to have the app monitor the filesystem and create folder shortcuts every time there's a change, but I'm curious about whether its possible to actually present a fake filesystem to explorer through a 'gateway' folder
EDIT: Ok it's obviously possible since http://www.virtualfolder.net/ can do it, and now that I think of it so can TrueCrypt, although it would be nice if it didn't have to appear as a separate drive. So the question becomes, how do I implement it?
You can create a Shell Namespace Extension that gathers the file information you want and displays it within Windows Explorer any way you wish. You can choose where your extension is located, whether as its own top-level node, a child of another system virtual folder/extension, or as a child of a file system folder.
Writing a SNE is not trivial, but it is a lot easier then writing a lower-level file system driver, and it does not require special driver-oriented compilers. Any compiler that supports developing COM objects will work.
This is accomplished using filesystem drivers or filesystem filter drivers. First let you create a virtual filesystem and mount it to a drive letter and also to a folder on NTFS drive (folder must exist but its contents are "replaced" with a virtual filesystem directory tree). Filesystem filter drivers let you introduce virtual files and folders in existing folders without replacing them.
VirtualFolder uses filesystem driver as it creates a drive letter.
Both types of drivers are written in C and work in kernel-mode. Writing them requires deep knowledge of Windows internals and experience with driver development (since filesystem drivers are one of the most complicated driver types).
We offer several products related to virtual storage. One of them, Callback File System, is a filesystem driver. It calls your user-mode code to perform actual filesystem functions. Another product, CallbackFilter, is an FS filter driver (and it also calls your user-mode code). However, current version of CallbackFilter doesn't let you introduce virtual files and folders (this would be implemented in the next release).
There's also Pismo File Mount product available, they use filter driver techniques. You can check with them if what you need can be accomplished.
From what I gather you are looking for a way to present the results of predefined file queries to appear as though they are located at a specific location in the file system. If that is correct you may want to look into Hard Links and Junctions. There are limits on what you can do with these file system services. However it is really straight forward to implement.
I store the MRU of logins to my application in a file called login.ini and I save it in widnows application folders.
I noticed that on some systems — I don't know why; I cannot find a common cause — the user cannot create the file, whereas it creates all other files in the same folder.
The only reason I can think of is that some antivirus/windows setting/... doesn't allow this particular user to create the file on this system.
I solved the problem by renaming the file and it seems ok, but I'd like to be sure. Does anyone know more?
Note for bounty:
This is a related question I asked that details a little more what I am doing.
A little Google-fu turns up that other Windows developers have sucessfully created login.ini for their programs, and others use it in a third-party Windows login management program, so I would expect that its "reservedness" is partially dependent on its location in the file system (i.e. in the system files). However, I don't think the name "login.ini" is a system-wide reserved name, no.
I think you're right - certain antivirus programs MAY be messing with the creation of that file, as it is a fairly likely candidate imho for a virus filename. It looks as if it may already have been used for that purpose somewhere (apparently outside of the US), tho don't quote me on that.
So, if a different name works for you, I'd go with that. :)
Anti-virus is a definite possibility for messing with your file. Stuff like that happened all the time to me when I was using Norton.
'login.ini' is not a system-wide reserved name, it would only mess things up with the OS if you had it in the (assuming your drive is C:) C:\WINDOWS or C:\WINDOWS\System32 directories.
If you just have the file in an application files directory (like C:\Program Files or C:\All Users\Application Settings and such) it shouldn't interfere with the system.
If you determine that anti-virus is a definite problem, you could change the name to something like loginData and maybe make up a new file extension if you want to (assuming you are just going to read the file from a program, where the extension doesn't matter. otherwise stick to a recognized file extension)
Linux has a feature called namespaces, which let you give a different "view" of the filesystem to different processes. In Windows terms, this would be useful for example if you had a legacy program "floyd" that always loaded its configuration from C:\floyd\floyd.ini. If Windows had namespaces, you could write a wrapper script which would create a namespace in which to run floyd, making it so when Alice ran the script, floyd would start up in an environment where C:\floyd existed but actually pointed to C:\Users\Alice\Floyd.
Now you may be thinking, "OK, just use soft or hard links and make C:\floyd an alias for C:\Users\Alice." But with namespaces, Bob can also run the startup script, but his instance of floyd (on the same computer, running at the same time) will see C:\floyd with the contents of, say, C:\Users\Bob\Program Settings\Floyd Config (or any other path we like).
You can do this on Linux with namespaces. Is there something similar or analogous on Windows? It's fine if it requires writing a C program, and it's OK if it only works on recent versions of Windows.
NTFS hard links are really a simple case of reparse points. Reparse points are typed, and can include more advanced behavior. For instance, they're also used for "offline storage" (transparent migration of files to and from secondary storage). You can therefore also use reparse points to implement per-user symbolic links, by creating a new reparse type.
The reparse point type even has an explicit "Name surrogate" bit, which (if set) indicates that reparse points of those types are some kind of symbolic link.
You can even have multiple reparse points in a path. Therefore files inside your symbolic namepace can still be migrated to secondary storage - you'd just have two reparse points in the path.
I think Virtual Store does this automatically for legacy programs that try to write to nonstandard directories. So the legacy program writes to a user- and program-specific directory instead to C:\floyd.
This sounds like Windows Vista's file system virtualization. For example, it can silently redirect c:\Program Files\Floyd to c:\Users\<username>\AppData\Local\VirtualStore\Program Files\Floyd. However, file system virtualization isn't nearly as configurable as Linux namespaces. From what I can tell from reading, file system virtualization should apply any time a 32-bit interactive process opens for writing a file, folder, or registry key that's only writable by administrators. (So you typically end up with some read-only files under c:\Program Files and some per-user writable files under c:\Users\<username>\AppData\Local\VirtalStore.)
An application virtualization product can probably also do this, although those are often more complicated and more expensive.
You can use hardlinks for that, but only with NTFS. http://en.wikipedia.org/wiki/Hard_link
I think windows doesn't have virtual FS view per process.
The most relevant thing is probably special environment folders, such as %temp%, %appdata%, %localappdata%. Not that they're equivalent, but they fulfil the same purpose.
You can define your own environment variables then use '%myspecialplace%\myfile.txt' to access them.
As a horrid kludge (and here I book my passage to programming hell) could you use a NamedPipe for C:\Floyd that mapped the IO operations onto a file specific to the current process user?
I know it's not pretty and I don't know enough about NamedPipes (FIFOs in other dialects) on Windows to know how feasible this is.
Dan
There are several things that come to my mind.
First of all you can create a file system filter driver (or use a ready driver, such as our CallbackFilter product) that would redirect all file system calls, coming from the application, to other location. This is close to virtualization that you mentioned but this won't change the list of drive letters though. Such approach is both powerful and non-trivial, so see the other option first of all.
And the other option is:
there exist several products (Thinstall, Molebox if memory serves) that "wrap" the application redirecting it's file I/O to some other location. There was also some SDK to do the same, but I don't remember it's name at all.
e.g. http://www.msigeek.com/4819/file-re-direction-using-correctfilepaths-shim-to-fix-broken-applications
But I think it's not configurable per user, although the target can vary per-user based on environment variable substitution.
Most programs, though, store configuration in the registry, in which case RegOverridePredefKey would do the trick.
There's a shortage of good solutions for this. For simplicity, I can't improve on using NTFS soft links (junctions) for this - as you correctly point out, this creates issues if you want per-user configuration. As MSalters correctly says, all NTFS soft and hard links are just special cases of reparse points, so you could do something more general by impplementing a new reparse type, if you don't mind some work digging into NTFS..
(Junction is a pretty useful tool when experimenting with NTFS soft links: http://technet.microsoft.com/en-us/sysinternals/bb896768.aspx )
You could just take a direct approach - give each user (or your program initialization if you only care about one piece of software) a logon script that sets up the appropriate junction into their user directory (and make sure you clean it up afterwards). But it's clumsy.
In general the right Windows approach is to put things into the folders pointed at by the %localappdata% (from Vista on) and, more generally, %userprofile% system variables.
Win file system virtualization is intended to ensure this in the cases where it applies.