Goal
Pass images generated by one process efficiently and at very high speed to another process. The two processes run on the same machine and on the same desktop. The operating system may be WinXP, Vista and Win7.
Detailed description
The first process is solely for controlling the communication with a device which produces the images. These images are about 500x300px in size and may be updated up to several hundred times per second. The second process needs these images to process them. The first process uses a third party API to paint the images from the device to a HDC. This HDC has to be provided by me.
Note: There is already a connection open between the two processes. They are communicating via anonymous pipes and share memory mapped file views.
Thoughts
How would I achieve this goal with as little work as possible? And I mean both work for the computer and me (of course ;)). I am using Delphi, so maybe there is some component available for doing this? I think I could always paint to any image component's HDC, save the content to memory stream, copy the contents via the memory mapped file, unpack it on the other side and paint it there to the destination HDC. I also read about a IPicture interface which can be used to marshal images. I need it as quick as possible, so the less overhead the better. I don't want the machine to be stressed just by copying some images.
What are your ideas? I appreciate every thought on this!
Use a Memory Mapped File.
For a Delphi reference see Memory-mapped Files in Delphi and Shared Memory in Delphi.
For a more versatile approach you can look at using pipes or sending bitmap data via TCP. This would allow you to distribute the image data between nodes more easily, if necessary.
Use shared memory to pass the image data, and something else (named pipes, sockets, ...) to coordinate the handover.
In some cases, you can pass HBITMAP handles across processes. I've seen it done before (yes, on XP/Vista), and was surprised as everyone else on the team when one of my co-workers showed me.
If memory serves me correctly, I believe it will work if the HBITMAP was allocated with one of the GDI function (CreateBitmap, CreateCompatibleBitmap,CreateDIBitmap,etc...) HBIMAP handles created by LoadBitmap will not work as it's just a pointer to an in-proc resource.
That, and I think when you share the HBITMAP across to the other process, don't try to do anything special with it other than normal BitBlt operations.
At least that's what I remember. We got lucky because our graphic libraries were already written to manage all images as HBITMAPs.
YMMV
Ok it seems as if memory mapped files and pipes are the right way to go. That is not too bad because the two processes already share a MMF and two pipes (for bidirectional communication). The only thing left to solve was how to pass the data with as little copy operations as possible.
The design which works quite well looks as follows (sequential flow):
Process 1 (wants image)
give signal to process 2 (via pipe 1) to store image in shared memory
go to sleep and wait for response (blocking read from pipe 2)
Process 2 (provides images)
on signal (via pipe 1) wake up and tell hardware device to paint to HDC 1 (this is backed by shared memory, see below)
give signal to process 1 (via pipe 2)
go to sleep and wait for new job (via pipe 1)
Process 1 (wants image)
on signal (via pipe 2) wake up and paint from shared memory to destination HDC 2
Now for the image transfer via shared memory (my goal was to use not more than one additional copy operation):
Process 2 creates a HBITMAP via CreateDIBSection and provides the handle of the file mapping and the offset of the mapped view. Thus the image data lives in the shared memory. This creates an HBITMAP which is selected into HDC 1 (which is also created by process 2) and which will be used from now on by process 2.
Process 1 uses StretchDIBits with a pointer to the mapped view's memory (as described here). This seems to be the only function for getting bits from memory directly into another HDC (in this case HDC 2). Other functions would copy them first into some intermediary buffer before you could transfer them from there to the final HDC.
So in the end it seems the bits needed to be transferred are about twice as much as in the beginning. But I think this is as good as it gets unless sharing GDI handles between processes would be possible.
Note: I used pipes instead of signals because I need to transfer some additional data, too.
As I can see this, you have two options:
Pass only the image handle / pointer to other process, so both processes work only on one collection of images.
Copy the image content to other process and work on a copy from then on.
Which approach is best depends on your design. Best tool for both approaches would be "memory mapped files", or "named pipes". This are the fastest you can get. Memory mapped files are probaly the fastest form of inter process communication but have the donwside that there is no "client-server" paradigm build into them. So you have to synchronize the acces to MMF yourself. Named pipes on the other hand are almost as fast but have the client-server paradigm build right into them. The difference in speed comes mainly from that.
Now because of the share speed of the updates, the first approach could be better, but then you have to watch out for synchronization between processes, so they do not read / write to single image at the same time. Also some sort of caching or other smart tehniques could be used, so you reduce your traffic to minimum. When facing such high level of communications there is always advisable to look for means of reducing that level if possible.
For a very fast implementation of IPC based on named pipes you can use my IPC implementation. It is message oriented so you do not have to worry about pipe technical details. It also uses thread pool behind the scenes and has mininal additional overhead.
You can stress test it and see for yourself (a typical message takes 0.1 ms for full client-server request-response cycle).
Related
In my practice I faced with the task - visualize some process memory content in real time. The main idea is read arbitrary part of remote process memory, represent it as image, and show in a separate window, then repeat these action with some interval, and in result get dynamic visualization of memory content. For example, it will be useful for view framebuffers/textures that located in process memory. Do exists any tools/software for this purpose? Thanks.
So, I did not find any utilities, and so I created my own tool.
This is mem2pix, program which allows to real-time visualizing some part of remote process memory, supports many pixel format type. Currently works on both Windows and Linux.
As the title says, I want to associate a random bit of data (ULONG) with a running process on the local machine. I want that data persisted with the process it's associated with, not the process thats reading & writing the data. Is this possible in Win32?
Yes but it can be tricky. You can't access an arbitrary memory address of another process and you can't count on shared memory because you want to do it with an arbitrary process.
The tricky way
What you can do is to create a window (with a special and known name) inside the process you want to decorate. See the end of the post for an alternative solution without windows.
First of all you have to get a handle to the process with OpenProcess.
Allocate memory with VirtualAllocEx in the other process to hold a short method that will create a (hidden) window with a special known name.
Copy that function from your own code with WriteProcessMemory.
Execute it with CreateRemoteThread.
Now you need a way to identify and read back this memory from another process other than the one that created that. For this you simply can find the window with that known name and you have your holder for a small chunk of data.
Please note that this technique may be used to inject code in another process so some Antivirus may warn about it.
Final notes
If Address Space Randomization is disabled you may not need to inject code in the process memory, you can call CreateRemoteThread with the address of a Windows kernel function with the same parameters (for example LoadLibrary). You can't do this with native applications (not linked to kernel32.dll).
You can't inject into system processes unless you have debug privileges for your process (with AdjustTokenPrivileges).
As alternative to the fake window you may create a suspended thread with a local variable, a TLS or stack entry used as data chunk. To find this thread you have to give it a name using, for example, this (but it's seldom applicable).
The naive way
A poor man solution (but probably much more easy to implement and somehow even more robust) can be to use ADS to hide a small data file for each process you want to monitor (of course an ADS associated with its image then it's not applicable for services and rundll'ed processes unless you make it much more complicated).
Iterate all processes and for each one create an ADS with a known name (and the process ID).
Inside it you have to store the system startup time and all the data you need.
To read back that informations:
Iterate all processes and check for that ADS, read it and compare the system startup time (if they mismatch then it means you found a widow ADS and it should be deleted.
Of course you have to take care of these widows so periodically you may need to check for them. Of course you can avoid this storing ALL these small chunk of data into a well-known location, your "reader" may check them all each time, deleting files no longer associated to a running process.
How to create a device independent bitmap in windows using win32/mfc that will be shared among all processes running on the machine?
Looking for a best and fastest way of sharing a DIB between all processes on windows XP/7 machine. Processes should be able to lock the contents of bitmap and make drawing on this bitmap and other processes can use this bitmap for reading also.
For e.g. initially this DIB will be created by a main application. When some other process want to draw something on this process, can lock the contents of this DIB and draw on this bitmap. If some other process want to read the contents of this bitmap, can lock the bitmap and read the bitmap.
Please suggest a best way both in time complexity and space complexity manner. Space complexity means, a process which wants to draw contents on this bitmap should not need to copy all contents on local memory and draw, it should directly be able to take this bitmap in device context and draw in the bitmap.
There is, unfortunately*, no supported way to share GDI handles (such as to bitmaps) between processes.
There is a supported way however to get multiple bitmaps (in multiple processes) to share the same storage.
In your primary process, create a memory section using the CreateFileMapping API.
You have several ways to get the section handle to your various processes -
The simplest of which (as pointed out by Hans Passant) is to simply name the section when calling CreateFileMapping... and then using OpenFileMapping.
If you have a main process that launches the other processes ensure that the section is created with the bInheritHandle of the SECURITY_ATTRIBUTES set to TRUE and the handle will be automatically duplicated into any sub processes - its usual to pass the handle value on the command line of the new process.
Otherwise use DuplicateHandle function to copy the handle into other processes - but you will still need some other kind of IPC to get the handle to the process.
However it happens - You can then call CreateDIBSection in each process to create GDI bitmaps that are backed by the same memory section. Note the comments on synchronizing access to the bitmap. If you have multiple processes trying to write to the bitmap you might need to serialize access at that level.
As an ironic note: Because Win32 is based on Win16 there are a lot of Win16 legacy APIs that deal with window messages and the clipboard that do in fact expect HBITMAP's to be usable from multiple processes. Also (as an implementation detail) on Windows NT 5.x and 6.x bitmaps are allocated by kernel mode drivers from a single system wide handle table and are thus technically valid in any process - However, GDI also stores the process ID of each GDI object in that table and GDI API calls explicitly check the process ID and fail out if called on a GDI handle belonging to another process.
I write a pdf viewer that uses various libraries written in C. This C code is potentially easy to exploit. And there are just too many lines to check. I will have to assume that this code may contain exploitable bugs.
The thing is that the C code is quite straightforward. A stream of bytes go in at one end, and a bitmap (also a stream of bytes) comes out at the other.
Inspired by google chrome, I am thinking to create a separate process that does the decoding and page rendering. Ideally this should be executed in a process that has absolutely no rights to do anything except reading the one input stream it has, and outputting to a stream of bytes (some uncompresed bitmap) at the other end.
What I think the process should not be able to do is:
any disk access
open sockets
limited amount of memory use
access shared memory with other processes
load other dll's
... anything else?
Is that possible? Is this described somewhere?
If you have the source code - you may check it doesn't do the described things.
Well, limiting available memory is a bit more difficult. You may however use SetProcessWorkingSetSize.
Also after you've built the executable you may check its DLL import table (by dependencies walker) to ensure it doesn't access any file/socket function.
This isn't really possible. Ultimately any potential exploit code will be running with whatever privileges this process runs with. If you run it as a standard user then you will limit the damage that could be done, but your best bet is to just fix the code as much as possible.
I have a process A that generates HBITMAP GDI objects to be painted on the screen. I have another process B which wants to display the content of images that process A creates.
I plan to do the communications/talking using Point-to-Point message queue or by using other message passing; and use shared memory (along with mutex and or events) to share data.
How do I share image data? I read here that the handles of GDI objects are not guaranteed to be shared amongst processes. Sharing using files is not really an option since the images keep changing (but I still consider it if there is no other way).
Adding more to the ingredients, process A is written using eVC4 (PPC2003 SDK) and process B is written using VS2005 (WM 6 Pro SDK). I have working source code of both applications so I can make some modifications to both but not migrating app A to VS2005 for the time being.
Is COM IImage an option?
I prefer native codes but also consider managed codes.
Use CreateDIBSection with a shared memory object HANDLE as the section handle. The HBITMAP you get back from CreateDIBSection may not be sharable with the other process (I don't know), but if the other process ALSO Creates a DIB Section from the same shared memory object, then the bitmap bits will be shared.