This question already has answers here:
Rewrite Registry File in Windows
(3 answers)
Closed 9 years ago.
This is out of curiosity, but I have seen several (and some of them very popular) software called registry defragmenter. While I can see the benefit they offer, but I am very curious on how exactly do you do registry defragmenting? Note that I'm not asking for software name, just a basic description of how it's done programmatically. I understand there is disk defragmenting API from microsoft. Is this that they are using? Or is there "registry defragmenting" api?
While disk defragmenting would be helpful, the more important speed benefit which could be obtained would be arranging the registry nodes so that a typical depth-first search would put the sequentially-accessed nodes in the same registry page.
I'm not aware of any API for that. The algorithm is a straightforward reordering and rewriting operation, complicated by dealing with Windows' concurrent access.
I suspect they're just defragmenting the files used to store registry information. Since the registry files are open during all normal Windows operation, a "normal" file defragmenting tool won't even touch them.
Answer: Most parse the file format directly and manually.
There is another possible way: Using the RegSaveKey and then the RegReplaceKey functions, which are used by the Windows Backup utility.
How do they prevent crashes in a live OS? Simple, they reroute API calls to the Windows Reg* functions and handle them themselves. Caching any changes that need to be written later. It would also be wise to hold an exclusive lock on the hive files.
I trust defragmenters MUCH more than I trust optimizers. Registry Optimizers can set untested or broken keys and enable broken features. With the mass commercialization of them, this is less of a problem. But still, with what I've seen in the past I don't trust them mucking my stable system up in ways that are too hard to pin down.
Related
Preamble:
Recently I came across an interesting story about people who seem to be sending emails with documents that contain child pornography. This is an example (this one is jpeg but im hearing about it being done with PDFs, which generally cant be previewed)
https://www.youtube.com/watch?v=zislzpkpvZc
This can pose a real threat to people in investigative journalism, because even if you delete the file after its been opened in Temp the file may still be recovered by forensics software. Even just having opened the file already puts you in the realm of committing a felony.
This also can pose a real problem to security consultants for a group. Lets say person A emails criminal files, person B is suspicious of email and forwards it to security manager for their program. In order to analyze the file the consultant may have to download it on a harddrive, even if they load it in a VM or Sandbox. Even if they figure out what it is they are still in this legal landmine area that bad timing could land them in jail for 20 years. Thinking about this if the memory was to only enter the RAM then upon a power down all traces of this opened file would disappear.
Question: I have an OK understanding about how computer architecture works, but this problem presented earlier made me start wondering. Is there a limitation, at the OS, hardware, or firmware level, that prevents a program from opening a stream of downloading information directly to the RAM? If not let's say you try to open a pdf, is it possible for the file it's opening to instead be passed to the program as a stream of downloading bytes that could then rewrite/otherwise make retention of the final file on the hdd impossible?
Unfortunately I can only give a Linux/Unix based answer to this, but hopefully it is helpful and extends to Windows too.
There are many ways to pass data between programs without writing to the hard disk, it is usually more of a question of whether the software applications support it (web browser and pdf reader for your example). Streams can be passed via pipes and sockets, but the problem here is that it may be more convenient for the receiving program to seek back in the stream at certain points rather than store all the data in memory. This may be a more efficient use of resources too. Hence many programs do not do this. Indeed a pipe can be made to look like a file, but if the application tries to seek backward, it will cause an error.
If there was more demand for streaming data to applications, it would probably be seen in more cases though as there are no major barriers. Currently it is more common just to store pdfs in a temporary file if they are viewed in a plugin and not downloaded. Video can be different though.
An alternative is to use a RAM drive, it is common for a Linux system to have at least one set up by default (tmpfs), although it seems for Windows that you have to install additional software. Using one of these removes the above limitations and it is fairly easy to set a web browser to use it for temporary files.
I'm wondering how long it takes (in milliseconds) to read a registry value from the Windows registry through standard C# libraries. In this case, I'm reading in some proxy settings.
What order of magnitude value should I expect?
Are there any good benchmark data available?
I'm running WS2k8 R2 amd64. Bonus points: How impactful is the OS sku/version on this measure?
using (RegistryKey registryKey = Registry.CurrentUser.OpenSubKey(#"Software/Copium"))
{
return (string)registryKey.GetValue("BinDir");
}
I cannot quote numbers as I don't know. But having just read 30 pages in the Windows Internals 5 book about the registry the following noteworthy things that I didn't know became clear.
The Registry is transactional and has fail safes to prevent from being corrupted. This can affect performance. Since the transactional level is read committed, reads shouldn't be blocked by writes so they should be performant.
The registry is cached in memory (well frequently used values anyway) so if you access a set of keys often the performance should remain stable after the first hit.
This blog post from Raymond Chen should prove helpful:
The performance cost of reading a registry key
I would add that in general, it is not recommended to store settings in the registry for C# apps, use persisted storage, or a config file or something of that nature. There are problems related to permissions when you deal with the registry. Reading is often not an issue, but if you are going to persist things then it gets hairy, especially with UAC and newer OSs that shadow copy things.
I have been investigating the windows Prefetching system hoping to find a way to speed up the load time of an application I am working on. I found the following link where a developer describes modifications to the prefetcher registry values:
http://dotnet.dzone.com/news/improving-cold-startup
I have made similar modifications locally and found that they do provide faster application loading times. My problem is that I cannot find any documentation on the registry values that were changed and why the new values are better than the old ones.
So my question in short is, does anybody have any further information on the prefetcher registry values given below:
VideoInitTime
EnablePrefetcher
AppLaunchMaxNumPages
AppLaunchMaxNumSections
AppLaunchTimerPeriod
BootMaxNumPages
BootMaxNumSections
BootTimerPeriod
MaxNumActiveTraces
MaxNumSavedTraces
RootDirPath
HostingAppList
You don't say what profiling or other changes you've done, and when people dive in with off-the-wall solutions to perf problems but don't describe how they arrived at the need for them, I'm always a bit doubtful.
Where is your app spending its start-up time? How did you measure that? Can you fix an underlying '300 dlls' problem of the type described in that article?
Messing with OS prefetch policy may being improving your application at the expense of everyone else, which may be the right thing to do (on a single-use industrial control system or something like that), but may be completely antisocial.
"Load less code" is often a good general way to improve application startup time - do you have some very expensive config file storage mechanism, for example (XmlSerializer was notorious for this at one point, for example).
I'm concerned about the dangers of using memory-mapped IO, via CreateFileMapping, on FAT filesystems. The specific scenario is users opening documents directly from USB sticks (yeah, you try and ban them doing this!).
The MSDN Managing Memory-Mapped Files article doesn't say anything about file system constraints.
Update
I didn't have any real reason to be concerned but a vague feeling that I'd read about problems with them at some point (my career spans over 25 years so I have a lot of vague depths in my memory, all the way back to 8-bit micros!). The issue of whether or not they should be supported is pretty important for me to recommend so I wanted to ask if anyone could corroborate my concerns. Thanks for putting my mind at rest.
Memory-mapped files is one of my favorite features. It's absolutely no danger. It's one of the base extremely optimized Windows I/O features. If one starts an EXE or load indirect a DLL it is implemented internally as memory-mapped file mapping.
It is supported on all types of file systems including FAT.
By the way atzz say that memory-mapped files are allowed on network drives. I can add it is not only allowed, but it is strictly recommended to use memory-mapped file also with files from network. In the case the I/O operation will be cached in very good way, which is not done with other (C/C++) I/O.
If you want that the EXE will not crash if you open it from the CD or network one can mark Program Executable with one bit in the header (linker switch /SWAPRUN see http://msdn.microsoft.com/en-us/library/chzz5ts6.aspx). There are no option for documents opened from USB stick.
But what exact problem do the users have? Do they don't use "Safely Remove Hardware" Icon? Then they have to learn to do this exactly like they have to learn to not switch computer power, but shutdown the computer properly.
Could you explain why you find dangers to use memory-mapped files, and in what situations you have problems and is usage of other I/O operation has no such problem?
Yes it does. It even supports mapping of files on CDFS or on network drives. What is the source of your doubts?
I would like to create events for certain resources that are used across various processes and access these events by name. The problem seems to be that the names of the events must be known to all applications referring to them.
Is there maybe a way to get a list of names events in the system?
I am aware that I might use some standard names, but it seems rather inflexible with regard to future extensibility (all application would require a recompile).
I'm afraid, I can't even consider ZwOpenDirectoryObject, because it is described as needing Windows XP or higher, so it is out of question. Thanks for the suggestion though.
I am a little unsure about shared memory, because I haven't tried it so far. Might do some reading in that area I guess. Configuration files and registry are a slight problem, because they do tend to fail with Vista due to access problems. I am a bit afraid, that shared memory will have the same problem.
The idea with ProcessExplorer sounds promising. Does anyone know an API that could be used for listing events for a process? And, does it work without administrative rights?
Thank you for the clarification.
There is not really a master process. It is more of a driver dll that is used from different processes and the events would be used to "lock" resources used by these processes.
I am thinking about setting up a central service that has sufficient access rights even under Vista. It will certainly complicate things, but it might be the only thing left facing the problems with security.
No, there is not any facility to enumerate named events. You could enumerate all objects in the respective object manager directory using ZwOpenDirectoryObject and then filter for events. But this routine is undocumented and therefore should not be used without good reason.
Why not use a separate mechanism to share the event names? You could list them in a configuration file, a registry key or maybe even in shared memory.
Do not mix up the user mode ZwOpenDirectoryObject with the kernel mode ZwOpenDirectoryObject -- the kernel mode API (http://msdn.microsoft.com/en-us/library/ms800966.aspx) indeed seems to available as of XP only, but the user mode version should be available at least since NT 4. Anyway, I would not recommend using ZwOpenDirectoryObject.
Why should configuration files and registry keys fail on Vista? Of course, you have to get the security settings right -- but you would have to do that for your named events as well -- so there should not be a big difference here. Maybe you should tell us some more details about the nature of your processes -- do they all run within the same logon session or do they run as different users even? And is there some master process or who creates the events in the first place?
Frankly, I tend to find the Process Explorer idea to be not a very good one. Despite the fact that you probably will not be able to accomplish that without using undocumented APIs and/or a device driver, I do not think that a process should be spelunking around in the handle table of another process just to find out the names of some kernel objects. And, of course, the same security issues apply again.
ProcessExplorer is able to enumerate all the named events held by some specific process. You could go over the entire process list and do something similar although I have now clue as to what API is used to get the list...