Getting full file attributes for files managed by Microsoft OneDrive - winapi

Microsoft OneDrive allows file to be stored either locally, remotely or in the both ways. This is determined by new file attributes which appeared in the Windows 10:
FILE_ATTRIBUTE_PINNED 0x00080000
FILE_ATTRIBUTE_UNPINNED 0x00100000
FILE_ATTRIBUTE_RECALL_ON_OPEN 0x00040000
FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS 0x00400000
as well as some file attributes inherited from previous versions of Windows:
FILE_ATTRIBUTE_SPARSE_FILE 0x00000200
FILE_ATTRIBUTE_REPARSE_POINT 0x00000400
FILE_ATTRIBUTE_OFFLINE 0x00001000
Problem is that I cannot find a way to retrieve these new file attributes via Win32 API or NT Native API. I've tried so far:
GetFileAttributes()
FindFirstFile()
NtQueryAttributesFile()
For OneDrive file which was set to be stored always remotely, all these methods return 0x00500020 whilst true attributes are 0x00501620 (REPARSE_POINT, SPARSE_FILE and OFFLINE are masked out). True file attributes can be retrieved using the following PowerShell command:
[Convert]::ToString( (Get-ItemProperty -Path 'C:\Users\username\OneDrive\test.txt').Attributes.Value__, 16 )
attrib.exe system command is also able to display some of these new OneDrive-related file attributes (O for offline, U for unpinned, P for pinned).
Is there a way to retrieve these file attributes in my software? Maybe I need to add something to the manifest?

From MSDN RtlSetThreadPlaceholderCompatibilityMode
When placeholders are exposed, characteristics such as the presence of
a reparse point, the sparse bit, and the offline bit are plainly
visible through directory enumeration and other types of file
information queries. When placeholders are disguised, these details
are completely hidden, making the file look like a normal file.
Most Windows applications see exposed placeholders by default. For
compatibility reasons, Windows may decide that certain applications
see disguised placeholders by default.
I am guessing that Windows has placed your test program in some sort of compatibility mode and is therefore filtering the attributes.
RtlSetThreadPlaceholderCompatibilityMode

Related

How to detect OneDrive online-only files

Starting from Windows 10 Fall Creators Update (version 16299.15) and OneDrive build 17.3.7064.1005 the On-Demand Files are available for users (https://support.office.com/en-us/article/learn-about-onedrive-files-on-demand-0e6860d3-d9f3-4971-b321-7092438fb38e)
Any OneDrive file now can have one of the following type: online-only, locally available, and always available.
Using WinAPI how can I know that the file (e.g. "C:\Users\Username\OneDrive\Getting started with OneDrive.pdf") is online-only file?
After years, I'm still using FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS attribute described here to determine if a file or a directory is completely present locally or not.
Microsoft docs says the following for FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS:
When this attribute is set, it means that the file or directory is not fully present locally. For a file that means that not all of its data is on local storage (e.g. it may be sparse with some data still in remote storage). For a directory it means that some of the directory contents are being virtualized from another location. Reading the file / enumerating the directory will be more expensive than normal, e.g. it will cause at least some of the file/directory content to be fetched from a remote store. Only kernel-mode callers can set this bit.
There are some advantages of FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS:
It can be used for both files and directories.
It can be set in kernel mode only, so there is no chance for anyone to set the attribute arbitrary.
And as it described in this answer, there are still some interesting undocumented attributes which can provide additional information about cloud files.
Note: I didn't accept Jonathan Potter's answer because I mentioned FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS attribute in comments and started using it a year earlier than he updated his answer.
To check for "online only" all you need is to call GetFileAttributes() and see if the FILE_ATTRIBUTE_OFFLINE attribute is set.
In fact this isn't new for OneDrive, that attribute has existed for a long time.
There are other OneDrive attributes available via the shell (although the property you need is PKEY_StorageProviderState rather than PKEY_FilePlaceholderStatus) but "online only" is easy to check for.
Edit: Another filesystem attribute, FILE_ATTRIBUTE_PINNED is new for Windows 10, and is used by OneDrive to indicate a file that's "always available".
Edit: As of 2019 it appears that OneDrive now uses FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS rather than FILE_ATTRIBUTE_OFFLINE, as suggested below.
Edit: PKEY_StorageProviderState was broken in Windows 10 1903, and still not fixed in 1909. It returns 4 ("uploading") for all files in any apps other than Explorer.
Take a look at the PKEY_FilePlaceholderStatus property for the file (at the shell level, not the file-system level). This blog post has a example program you can test. This question also hints to some undocumented properties you might want to take a look at.
Microsoft has a UWP example on MSDN.

Global folder in Windows 7

I want to allow all users to read and write in a file, through my application. The file is common to all users. Where should I put that file in Windows 7?
You can use the Known Folders.
If users don't need to see the files the you can use:
FOLDERID_ProgramData - default location will be "c:\ProgramData"
If the user needs to be able to browse and find the file then the "Public" user can be used. Either FOLDERID_Public or FOLDERID_PublicDocuments. That will default to "c:\users\Public*"
You can check the list of all the standard Known Folders to see if there is anything that would work better for you. This page has the same list with friendly descriptions on how they should be used.
Use the function SHGetKnownFolderPath to get the path on the current system. Don't assume the default values are valid.
If you need this to run on Windows XP then use the previous function SHGetFolderPath and equivalent CSIDLs.

How do I set file permissions for non-admin users in VB6?

I have an old update program written in vb6, which runs as admin. However, because it runs as admin, all the files it downloads and saves are read-only to other users. Even files in public places like the shared application data folder (which is where I'm saving the files in question).
I'm lucky I found this before the 'vista-compatible' release. Vista hides the problem by redirecting non-admin writes and future reads to a sortof 'virtual' folder. But the next update may replace the file, and the non-admin program will still go to the virtual folder and use the old file.
As the admin user, how do I allow other users full control of files I write in vb6?
The way I do this is to make it an Installer responsibility.
Use VSI 1.1 to create an Installer MSI for your application. Create an application data folder under CommonAppDataFolder.
As a post-build step run a script to perform the following:
Set the MSI database for a per-machine installation: Property table, row with ALLUSERS set to 1.
In the Directory table, locate the entry for CommonAppDataFolder and obtain its directory Index. Use this Index to query the Directory table for an entry where CommonAppDataFolder is the parent and obtain its Index (this is your app data subfolder).
Look in the File table to obtain the component Index of your program.
Create the CreateFolder table in the database if it isn't present. Add a row to CreateFolder for the desired application subdirectory by its Index, tying it to your program's component Index.
Create the LockPermissions table if it isn't present. Insert a new LockPermissions row for your application data subdirectory, giving it FILE_ALL_ACCESS for Everyone.
That's about it.
You can do it this way, or use VSI 1.1 and then edit the MSI using Orca, or probably by using a 3rd party MSI authoring tool these entries will be settable via its GUI and can be saved in the Installer project. I just use a small WSH script I run after each VSI 1.1 build.
AFAIK this is the recommended way of accomplishing such things according to Windows application guidelines. If your needs are fancier you might use multiple subdirectories or sub-subdirectories some allowing full access, some read-only, etc.
Your program can locate the folder using Shell Automation objects or by calling Shell32 as a standard DLL (using Declare Function or a TLB).
It's not necessarily who writes the file, but where they write it to. The program files folder and it's sub folders are read-only to all standard users by default. Try using the All Users Application Data folder instead.
This is a little tricky for VB6, since it was not at all designed with Vista in mind. Some of the relevant folders were re-named and there's no way I know to get vb6 to give you the exact folder you want short of using a "Declare Function" alias to call directly into the windows API.
So the easiest reliable way I know to find a suitable location is to use the %ALLUSERSPROFILE% environment variable. That returns "C:\Documents and Settings\All Users" by default on XP and "C:\ProgramData" by default on Vista. From there you can look for an "Application Data" folder. It won't be there and you don't need it on Vista, but creating one if it doesn't exist won't hurt anything. That gives you a consistent folder structure on both systems from which you can create a sub folder for your app to use as a work space.
And one final note: this isn't a new change for Vista. Program Files folders have always been read-only to standard users by default. XP worked the same way. It's just that so many people run as administrators in XP you might be able to get away with it.

Can VS_VERSION_INFO be added to non-exe files?

My windows co-workers were asking me if I could modify my non-windows binary files such that when their "Properties" are examined under Windows, they could see a "Version" tab like that which would show for a Visual Studio compiled exe.
Specifically, I have some gzipped binary files and was wondering if I could modify them to satisfy this demand. If there's a better way, that would be fine, too.
Is there a way I could make my binaries appear to be exe files?
I tried simply appending the VS_VERSION_INFO block from notepad.exe to the end of one of my binaries in the hope that Windows scans for the block, but it didn't work.
I tried editing the other information regarding Author, Subject, Revision, etc. That doesn't modify the file, it just creates another data fork(what's the windows term?) for the file in NTFS.
It is not supported by windows, since each file type has their own file format. But that doesn't mean you can't accomplish it. The resources stored inside dlls and exes are part of the file format.
Display to the user:
If you wanted this information to be displayed to the user, this would probably be best accomplished with using a property page shell extension. You would create a similar looking page, but it wouldn't be using the exact same page. There is a really good multi part tutorial on shell extensions, including property pages starting with that link.
Where to actually store the resource:
Instead of appending a block to the file, you could store the resource into a separate alternate data stream on the same file. This would leave the original file stream non corrupted on disk and not cause its primary file size to change.
Alternate data streams allow more than one data stream to be associated with a filename. Each stream is identified by a colon : at the end of the filename and an identifier.
You can create them for example by doing:
notepad test.txt:adsname1
notepad test.txt:adsname2
notepad test.txt
Getting the normal Win32 APIs working:
If you wanted the normal API to work, you'd have to intercept the Win32 APIs: LoadLibraryEx, FindResource, LoadResource and LockResource. This is probably not worth the trouble though since you are already creating your own property page.
Can't think of any way to do this short of a shell extension. The approach I've taken in the past is a separate "census" program that knows how to read version information from any kind of file.
Zip files can be converted into exe files by using a program that turns a zip file into a self-extracting zip (I know that WinZip does this, there are most likely free utilities for this also; here's one that came up on a search but I haven't actually tried it). Once you've got an exe, you should be able to use a tool like Resource Hacker to change the version information.
It won't work. Either Windows would have to know every file format or no file format would be disturbed if version information were appended to it.
No, resource section is only expected inside PE (portable executable; exe, dll, sys).
It is more then just putting the data inside the file, you have a table that points to the data in the file header.
What you can do if you have NTFS drive, is to use NTFS stream to store custom properties this way the contact of the binary file will remain the same, but you will need to use a custom shell extension to show the content of the stream.

Registry vs. INI file for storing user configurable application settings [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I'm a new Windows programmer and I'm not sure where I should store user configurable application settings. I understand the need to provide a user friendly means for the user to change application settings, like an Edit | Settings form or similar. But where should I store the values after the user hits the Apply button on that form?
What are the pros and cons of storing settings in the Windows registry vs. storing them in a local INI file or config file or similar?
Pros of config file:
Easy to do. Don't need to know any Windows API calls. You just need to know the file I/O interface of your programming language.
Portable. If you port your application to another OS, you don't need to change your settings format.
User-editable. The user can edit the config file outside of the program executing.
Pros of registry:
Secure. The user can't accidentally delete the config file or corrupt the data unless he/she knows about regedit. And then the user is just asking for trouble.
I'm no expert Windows programmer, but I'm sure that using the registry makes it easier to do other Windows-specific things (user-specific settings, network administration stuff like group policy, or whatever else).
If you just need a simple way to store config information, I would recommend a config file, using INI or XML as the format. I suggest using the registry only if there is something specific you want to get out of using the registry.
Jeff Atwood has a great article about Windows' registry and why is better to use .INI files instead.
My life would be a heck of a lot easier if per-application settings were stored in a place I could easily see them, manipulate them, and back them up. Like, say... in INI files.
The registry is a single point of failure. That's why every single registry editing tip you'll ever find starts with a big fat screaming disclaimer about how you can break your computer with regedit.
The registry is opaque and binary. As much as I dislike the angle bracket tax, at least XML config files are reasonably human-readable, and they allow as many comments as you see fit.
The registry has to be in sync with the filesystem. Delete an application without "uninstalling" it and you're left with stale registry cruft. Or if an app has a poorly written uninstaller. The filesystem is no longer the statement of record-- it has to be kept in sync with the registry somehow. It's a total violation of the DRY principle.
The registry is monolithic. Let's say you wanted to move an application to a different path on your machine, or even to a different machine altogether. Good luck extracting the relevant settings for that one particular application from the giant registry tarball. A given application typically has dozens of settings strewn all over the registry.
There's one more advantage to using an INI file over the registry which I haven't seen mentioned:
If the user is using some sort of volume/file based encryption, they can get the INI file to be encrypted pretty easily. With the registry it will probably be more problematic.
According to the documentation for GetPrivateProfileString, you should use the registry for storing initialisation information.
However, in so saying, if you still want to use .ini files, and use the standard profile APIs (GetPrivateProfileString, WritePrivateProfileString, and the like) for accessing them, they provide built-in ways to automatically provide "virtual .ini files" backed by the registry. Win-win!
There's a similar question here that covers some of the pros and cons.
I would suggest not using the registry unless your application absolutely needs it. From my understanding, Microsoft is trying to discourage the use of the registry due to the flexibility of settings files. Also, I wouldn't recommend using .ini files, but instead using some of the built-in functionality to .Net for saving user/app settings.
Use of an ini file, in the same directory as the application, makes it possible to back it up with the application. So after you reload your OS, you simply restore the application directory, and you have your configuration the way you want it.
I agree with Daniel. If it's a large application I think I'd do things in the registry. If it's a small application and you want to have aspects of it user-configurable without making a configuration form, go for a quick INI file.
I usually do the parsing like this (if the format in the .ini file is option = value, 1 per line, comments starting with #):
static void Parse()
{
StreamReader tr = new StreamReader("config.ini");
string line;
Dictionary<string, string> config = new Dictionary<string, string>();
while ((line = tr.ReadLine()) != null)
{
// Allow for comments and empty lines.
if (line == "" || line.StartsWith("#"))
continue;
string[] kvPair = line.Split('=');
// Format must be option = value.
if (kvPair.Length != 2)
continue;
// If the option already exists, it's overwritten.
config[kvPair[0].Trim()] = kvPair[1].Trim();
}
}
Edit: Sorry, I thought you had specified the language. The implementation above is in C#.
As Daniel indicated, storing configuration data in the registry gives you the option to use Admin Templates. That is, you can define an Admin Template, use it in a Group Policy and administer the configuration of your application network-wide. Depending on the nature of the application, this can be a big boon.
The existing answers cover a lot of ground but I thought I would mention one other point.
I use the registry to store system-wide settings. That is, when 2 or more programs need the exact same setting. In other words, a setting shared by several programs.
In all other cases I use a local config file that sits either in the same path as the executable or one level down (in a Configuration directory). The reasons are already covered in other answers (portable, can be edited with a text editor etc).
Why put system-wide settings into the registry? Well, I found that if a setting is shared but you use local config files you end up duplicating settings. This may mean you end up needing to change a setting in multiple places.
For example, say Program A and Program B both point to the same database. You can have a "system-wide" registry setting for the connection string. If you want to point to a different database, you can change the connection string in one place, and both programs will now run against the other database.
Note - there is no point in using the registry in this way if two or more programs don't need to use the same values. Such as, Program A and Program B both needing a database connection string that may be the same, but not always. For example, I want Program B to now use a test database but Program A should carry on using a production database.
With the above example, you could have some local configuration override system-wide settings but it may start getting overly complicated for simple tasks.
The registry is optimized for quick access and easy update, and it's the only way to do certain Windows-specific things like associating with an extension. And you can ignore the argument about deleting a single directory to uninstall your program - Windows Vista won't let you modify files in the Program Files directory, so your config will need to go in a different folder anyway.
There's a general guideline for Windows programming - do things the way Microsoft expects you to, and your life will be a lot easier.
That said, I can see the appeal of the INI file, and I wouldn't blame anyone for considering it.
There is one drawback to ini or config files and that is locating them if the user has the option to select where the program is installed.
Other disadvantage of using the registry is that it is a pain if you are working in a mixed environment of 32 and 64 bit applications, as the system call for accessing the registry will randomly(*) add \Wow6432Node\ to your registry path making you crazy while debugging.
(*of course not randomly, but very easy to get lost)
Advantages:
Replacement for a large number of configuration files.
Common administrative functions at a central point.
Almost any data can be saved by applications / drivers.
In contrast to configuration files, code sequences can even be saved.
Access faster than files because the database is indexed.
Access can be logged using the RegMon utility
Disadvantages:
Difficult to work with in the absence of graphical configuration programs.
Direct changes using the registry editor can create inconsistent states produce.
Incomplete uninstallers leave “reminiscences” in the registry Cause problems, e.g. with a new installation.
Installed applications are difficult to export to other PCs.
Chronically poorly documented.
Proprietary structure, therefore not suitable for standard DB access (e.g. SQL)
Computer-specific, therefore not portable to other computers.
Insufficient protection of the registry: depends on the configuration.

Resources