I have implemented a Windows keyboard hook in Rust and would like to pass the data it reads to another process that can actually make use of the data. I tried creating a named pipe and having the callback function write the data to the pipe, but I've found there is a significant amount of overhead in that method. What methods would you suggest for passing data from keyboard hooks to another process?
I appreciate the suggestions. I implemented the mapped memory approach and found that was significantly faster than named pipes, but also more complicated from my perspective. I actually ended up falling back on the Windows messaging system and passed everything using PostMessageW, which was very simple and fast enough. Thanks a lot.
I'm about to choose using FlushFileBuffers after each write in a file or FILE_FLAG_NO_BUFFFERING each time I need open the same file.
But I did not find any performance comparison table about the use of one or another option. Well, except this advice in MSDN:
If an application is performing multiple writes to disk and also needs
to ensure critical data is written to persistent media, the
application should use unbuffered I/O instead of frequently calling
FlushFileBuffers. To open a file for unbuffered I/O, call the
CreateFile function with the FILE_FLAG_NO_BUFFERING and
FILE_FLAG_WRITE_THROUGH flags. This prevents the file contents from
being cached and flushes the metadata to disk with each write.
So I'm assuming that: for an application that will write and read many times it is better not use FlushFileBuffers. But there is any comparison article, blog, forum post about the use of them? And if the file is being closed after the write? My google-fu did not get anything yet.
I write a pdf viewer that uses various libraries written in C. This C code is potentially easy to exploit. And there are just too many lines to check. I will have to assume that this code may contain exploitable bugs.
The thing is that the C code is quite straightforward. A stream of bytes go in at one end, and a bitmap (also a stream of bytes) comes out at the other.
Inspired by google chrome, I am thinking to create a separate process that does the decoding and page rendering. Ideally this should be executed in a process that has absolutely no rights to do anything except reading the one input stream it has, and outputting to a stream of bytes (some uncompresed bitmap) at the other end.
What I think the process should not be able to do is:
any disk access
open sockets
limited amount of memory use
access shared memory with other processes
load other dll's
... anything else?
Is that possible? Is this described somewhere?
If you have the source code - you may check it doesn't do the described things.
Well, limiting available memory is a bit more difficult. You may however use SetProcessWorkingSetSize.
Also after you've built the executable you may check its DLL import table (by dependencies walker) to ensure it doesn't access any file/socket function.
This isn't really possible. Ultimately any potential exploit code will be running with whatever privileges this process runs with. If you run it as a standard user then you will limit the damage that could be done, but your best bet is to just fix the code as much as possible.
The MSDN states in its description of ReadFile() function:
If hFile is opened with FILE_FLAG_OVERLAPPED, the lpOverlapped parameter must point to a valid and unique OVERLAPPED structure, otherwise the function can incorrectly report that the read operation is complete.
I have some applications that are violating the above recommendation and I would like to know the severity of the problem. I mean the program uses named pipe that has been created with FILE_FLAG_OVERLAPPED, but it reads from it using the following call:
ReadFile(handle, &buf, n, &n_read, NULL);
That means it passes NULL as the lpOverlapped parameter. That call should not work correctly in some circumstances according to documentation. I have spent a lot of time trying to reproduce the problem, but I was unable to! I always got all data in right place at right time. I was testing only Named Pipes though.
Would anybody know when can I expect that ReadFile() will incorrectly return and report successful completion even the data are not yet in the buffer? What would have to happen in order to reproduce the problem? Does it happen with files, pipes, sockets, consoles, or other devices? Do I have to use particular version of OS? Or particular version of reading (like register the handle to I/O completion port)? Or particular synchronization of reading and writing processes/threads?
Or when would that fail? It works for me :/
Please help!
With regards, Martin
Internally the system only supports asynchronous I/O. For synchronous I/O the system creates a temporary OVERLAPPED structure with hEvent = NULL;, issues an asynchronous I/O request passing in this temporary, and then waits for completion using GetOverlappedResult( bWait = TRUE ).
Recall that the hEvent of the temporary OVERLAPPED structure is NULL and pay attention to the Remarks section of GetOverlappedResult:
If the hEvent member of the OVERLAPPED structure is NULL, the system uses the state of the hFile handle to signal when the operation has been completed.
A file HANDLE is a waitable object that becomes unsignaled when an I/O operation begins, and signaled when an I/O operation ends.
Now consider a scenario where an asynchronous file HANDLE has a pending I/O request at the time you issue a synchronous I/O request. The system creates an OVERLAPPED structure and waits on the hFile HANDLE for completion. In the meantime the asynchronous I/O completes, thereby signaling the HANDLE causing the synchronous I/O to return prematurely without having actually completed.
Worse though is that by the time the asynchronous I/O that was initiated in response to the synchronous I/O request completes it will update the temporary OVERLAPPED structure that no longer exists. The result is memory corruption.
The full story can be found at The Old New Thing.
Seems like you are in a situation where you are deliberately calling an API in contravention of the documented best practices. In such situations all bets are off. It may work, it may not. If may work on this OS, but not on the next iteration of the OS, or the next service pack of the same OS. What happens when you port to Win64? Will it still work then?
Does calling GetLastError() (or looking at #ERR,hr in the debugger) give any value that is useful in addition to the error code?
I recommend that you call it with a valid OVERLAPPED structure, get it working and remove all doubt (and possibility of random failure). Why have possibly buggy code (and very hard to reproduce bugs) in your software when you can fix the problem easily by using a valid OVERLAPPED structure?
Why ask the question rather than fix the code to call the API as it was intended?
I suspect it always appears to work because, even though this is an asynchronous I/O, it completes very quickly. Depending on how you're testing for success, it's possible the function is incorrectly reporting that the operation completed, but it actually completes before you test the results.
The real test would be to do a read on the pipe before there's data to be read.
But really, you should just fix the code. If your architecture cannot handle asynchronous I/O, then remove the FILE_FLAG_OVERLAPPED from the creation of the named pipe.
When they say
Blockquote
If hFile is opened with FILE_FLAG_OVERLAPPED, the lpOverlapped parameter must point to a valid and unique OVERLAPPED structure, otherwise the function can incorrectly report that the read operation is complete.
they mean that there's nothing in the code preventing it working, but there's also a path through their code that can produce erroneous results. Just because you can't reproduce the problem with your particular hardware does not mean there is no problem.
If you really want to reproduce this problem, leave the code as is and go on with your life. Right about the time you've forgotten all about this problem, strange behavior will surface that will not have any obvious relations to calling ReadFile. You'll spend days pulling your hair out, and the problem will appear to come and go randomly. Eventually you'll find it and kick yourself for not following the instructions. Been there, done that, no fun!
The other way to recreate the problem is to schedule an important demo for your customer. It's sure to fail then!
If you don't want to splatter your code with OVERLAPPED structures and all of the related return value checks, Waits, Events, etc, you can write a wrapper function that takes a handle from which to read, and a timeout. Simply replace your calls to ReadFile with this handy-dandy wrapper.
I am looking for a simple way to read/write a file asynchronously using Win API. What I had is mind is something like the asynchronous winsock API (WSAxxx) completion routines. However the file API doesn't seem to have those. Are they hidden somewhere?
Waiting on the overlapped events in a seperate thread adds thread management overhead, not to mention there either needs to be a thread-per-file, or the 64 objects problem needs to be faced. Completion ports is an overkill. Reading the file synchronously on a seperate thread is irrelevant.
Any suggestions?
CreateFile and ReadFile/WriteFile functions support so-called 'overlapped' mode which is what you need. There' also ReadFileEx/WriteFileEx that work in async mode only.
In short, you need to open file with FILE_FLAG_OVERLAPPED flag and pass OVERLAPPED structure (and callback in case of xxxEx operations) to file access functions.
Here's a sample class for using it.
I know that in .net it's possible. What I don't know is to which win32 functions it maps
As soon as you step into the async territory you should forget the word "easiest"
Seriously, the easiest would be to use .NET's System.IO.FileStream with isAsync=true in constructor and BeginRead/EndRead methods.