I would like to test some software and would like to make it well-behaved regarding cloud files. For reference functions like RtlIsPartialPlaceholder and RtlIsCloudFilesPlaceholder have been introduced in order to look at the information returned when traversing the folder hierarchy. The above links point to kernel mode documentation, but these functions also exist in user mode (ntdll.dll) and they are really implemented very trivially.
However, in order to test said software I would have to be able to somehow create the states a placeholder for a file on OneDrive can be in.
What functions (registered COM classes would also be fine) can I use to automate creation of a reproducible test set up which I can use to:
create placeholders
only partially hydrate a placeholder
achieve the same for both directories and files
PS: this question was the only one remotely connected to the topic, which I was able to find here on SO.
Standard file/directory operations (create/read/write, etc.) can be achieved transparently using the Win32 API. That's the whole point of this technology.
So, you can create a placeholder using the standard Win32 APIs.
If you create a file or directory in a "sync root" (like in the OneDrive folder hierarchy in the OneDrive case), the associated sync engine process (like OneDrive.exe) will ensure the file or directory is a placeholder. And you can't create a placeholder outside of a sync root hierarchy, AFAIK.
For a file (you can't hydrate or dehydrate a directory), reading/writing correspond to hydration. Note that some sync engines (and/or depending on their configuration) can decide to always fully hydrate a file even if only some bytes were asked by the end-user API (applications).
There are some specific Win32 APIs though for special operations.
You can dehydrate a file using CfDehydratePlaceholder. You can hydrate a file CfHydratePlaceholder. For all the Cloud Filter API, when creating directory handles, don't forget to use the FILE_FLAG_BACKUP_SEMANTICS flags.
Windows Resource Monitor displays (among other things) which files on disk are currently accessed by which processes. And it does that in realtime. How?
I know that it probably uses ETW and that I can generate traces with tools like xperf. But how to get realtime information without having to start, stop and parse a trace file?
I need to programmatically access the data, i.e. from C# or C++.
wOpenTrace/ProcessTrace/StopTrace can get the data in real-time as long as you know the provider GUID. They can run on Win2000 but you need to parse the raw data in your callback functions. To convert raw data into human-readable text, we need the TMF/MOF. Not sure if they are public though.
For Vista/Win7, there is a new set of TDH (Trace Data Helper) APIs (eg: TdhFormatProperty).
Scroll down a little of above links and you can see them. The good thing about TDH is they can parse the data for you (still need to provide TDH the TMF/MOF though).
I tried to write my own .etl to readable .txt program using Open/Process/StopTrace API (because I need to support XP). I found out it's quite difficult. The TMF file is not hard to interpret since it pure text. The hard thing is to decipher more than 50 different undocumented prinf-alike format-specifications' internal structures. So I gave up in the end and stick to the powerful tracefmt.exe provided in Microsoft WDK.
My google search has led me to here: How to create a virtual file?
In that article, people suggested the author to extract the file to a temp directory and work with it from there.
Actually, what I would like to do, is something similar to what the author was originally asking - create a "virtual" file and assign a "path" to it, such that would be understandable by other applications calling standard windows filesystem API. The purpose of this is to prevent copying and unauthorized use.
A concept of what I would like to do:
In our company, we have developed a program - but it's not a standard EXE application, it has an internal format (let's assume the 'application' format's extension is .MDL) and is executed by runtime environment, so the runtime must be installed on the client machine. This concept is similar to Java (although this is not java; just comparing, so that you can better understand). What I want to do now, is to encrypt this .MDL file and include it as a resource in a regular .exe file. This .exe file would check if it's okey to run (check license conditions, keys, authorization etc...) and then, if authenticated, decrypt and extract the .MDL file, exec the runtime and pass the virtual path of the .MDL file as an argument to the runtime.
For obvious reasons, i don't want to write the decrypted .mdl file somewhere on the disk, because somebody could just start the application, wait for the decrypter to write .MDL to temp, copy it from the temp and distribute it.
So, any idea how to do this?
PS: I've got inspired from EA Games' games... when you run them and look into your process table, you see a regular game's exe, and, after certain time, it spawns a new process with name like ~A003.tmp
You should use boxedapp packer. It must help.
I am not aware of any method to have a virtual file associated with a path without using some kind of driver component; that is hard work.
Why don't you have the runtime do the decryption? Then you don't have to worry about decrypting the file on disk. Or you could create an unnamed inheritable section object to which to decrypt the item and pass the handle value to the runtime as a command line parameter.
Whatever you do, people will still be able to bypass it so it seems like a waste of time to me. Every EA game ever made has been cracked; people will either think your product is useful and worth the money or they won't. I don't think it is worth the time or effort to implement schemes more complicated than a very simple registration key check (just to make it look like you did something).
In what way is the Windows registry meant to be used? I know it's alright to store a small amount of user preferences, but is it considered bad practice to store all your users data there? I would think it would depend on the data set, so how about for small amounts of data, say, less than 2KB, in 100 or so different key/value pairs. Is this bad practice? Would a flat file or SQLite db be a better practice?
I'm going to take a contrarian view.
The registry is a fine place to put configuration data of all types. In general it is faster than most configuration files and more reliable (individual operations on the registry are transacted so if your app crashes during a write the registry isn't corrupted - in general that isn't the case with ini files).
Marcelo MD is totally right: Storing things like operation percentage complete in the registry (or any other non volitile storage) is a horrible idea. On the other hand storing data like the most recently used files is just fine - the registry was built for just that kind of problem.
A number of the other commenters on this post talking about the MRU list have discussed the problem of what happens when the MRU list gets out of sync due to application crashes. I'm wondering why storing the MRU list in a flat file in per-user storage is any better?
I'm also not sure what the "security implications" of storing your data in the registry are. The registry is just as secure as the filesystem - the registry and the filesystem use the same ACL mechanism to protect their data.
If you ARE going to store your user data in a file, you should absolutely put your data in %APPDATA%\CompanyName\ApplicationName at least - that way if two different developers create an application with the same name (how many "Media Manager" applications are there out there?) you won't have collisions.
For me, simple user configuration items and user data is better to be stored in either a simple XML configuration file, a SQLLite db, or a MS SQL Server Compact db. The exact storage medium depends on the specifics of the implementation.
I only use the registry for things that I need to set infrequently and that users don't need to be able to change/see. For example, I have stored encrypted license information in the registry before to avoid accidental user removal of the data.
Using the registry to store data has mainly one problem: It's not very user-friendly. Users have virtually no chance of backing up their settings, copying them to another computer, troubleshooting them (or resetting them) if they get corrupted, or generally just see what their software is doing.
My rule of thumb is to use the registry only to communicate with the OS. Filetype associations, uninstaller entries, processes to run at startup, those things obviously have to be in the registry.
But data that is for use in your application only belongs in a file in your App Data folder. (whiever one of the 3+ App Data folders Microsoft currently wants you to use, anyway)
As each user has directory space in Windows already dedicated to storing application user data, I use it to store the user-level data (preferences, for instance) there.
In C#, I would get it by doing something like this:
Environment.GetFolderPath( Environment.SpecialFolder.ApplicationData);
Typically, I'll store SQLite files there or whatever is appropriate for the application.
If your app is going to be deployed "in the enterprise", keep in mind that administrators can tweak the registry using group policy tools. For example, if firefox used the registry for things like the proxy server, it would make deployment a snap because an admin can use the standard tools in active directory to set it up. If you use anything else, I dont think such things can be done very easily.
So don't dismiss the registry all together. If there is a chance an admin might want to standardize parts of your configuration across a network, put the setting in the registry.
I think Microsoft is encouraging use of isolated storage instead of the Windows registry.
Here's an article that explains how to use it in .Net.
You can find those files in Windows XP under Documents & Settings\\Local Settings\ App Data\Isolated Storage. The data is in .dat files
I would differentiate:
On the one hand there is application specific configuration data that is needed for the app to run, e.g. IP addresses to connect to, which folders to use for what sort of files etc, and non trivial per user settings.
Those I put in a config file, ini format for simple stuff, xml if it gets more complex.
On the other hand there is trivial per user settings (best example: window positions and layout). To avoid cluttering the config files (which some users will want to edit themselves, so few and clearly arranged entries are a must), I like to put those in the registry (with conservative defaults being set in the app if no settings in the registry can be found).
I mainly do it like istmatt sais: I store config files inside the %APPDATA% folder. Usually in %APPDATA%\ApplicationName, I don't like the .NET default of APPDATA%\CompanyName\ApplicationName\Version, that level of detail and complexity is counterproductive for most small to medium sized applications.
I disagree with the example of Marcelo MD of not storing recently used files in the registry. IMO this is exactly the volatile sort of user specific information that can be stored there.
(His example of what not to do is very good, though!)
To me it seems easier to think of what you should NOT put there.
e.g: dynamic data, such as an editor's "last file opened" and per project options. It is really annoying when your app loses sync with the registry (file deletion, system crash, etc) and retrieves information that is not valid anymore, possibly deadlocking the user.
At an earlier job I saw a guy that stored a data transfer completness percentage there, Writing the new values at every 10k or so and having the GUI retrieve this value every second so it could show on the titlebar.
I need to get some information that is contained in the MFT on a Windows machine, and I'm hoping that there is some super-secret API for getting this information. I need to be able to get to this information programmatically, and because of legal concerns I might not be able to use the tools provided by the company formally known as sysinternals.
My other option (which I really don't want to have to do) is to get the start sector of the MFT with DeviceIoControl, and manually parse through the information.
Anyway, in particular, what I really need to get out of the Master File Table is the logical sectors used to hold the data that is associated with a file.
There is a documented API for getting info on file positions on disk since Windows 2000. Look for DeviceIoControl function with FSCTL_GET_RETRIEVAL_POINTERS control code on MSDN:
http://msdn.microsoft.com/en-us/library/aa364572(VS.85).aspx
The API has been provided for writing custom disk defragmenters and consists of several other control codes.