Consider two application.
"A" application receives data from internet like player position and other details
"B" application which also needs the player position but it will be blocked from accessing internet. So the only way is to use SQLite sync player position (these frequently updates in milliseconds).
I can't even use socket or any other plugins too. So do you think SQLite can handle read and write in milliseconds without using CPU heavily ?
If you wish to share the data in anything like real-time then I would use something like inter-process pipes or file mapping (memory) for this.
Writing data to and reading it back from any form of hardware storage will add quite a delay to the data passing, which will only become worse as the hardware data cache is filled.
Hardware is okay for historic data.
Both are supported by Win32 and should be accessible even if you use .NET to produce a UWP application.
See here
I am working on a windows system. I need to create a shared memory for inter process communication to share objects (containing pointers as members). Or some equivalent way for fast transfer of objects from a generator process to a receiver process. the size of the objects are also huge. How do i do that? The porblem is that even if i share the objects I need a way so that the other process gets the access to the locations pointed by the pointers in the objects. And sharing each of those locations for each object is not feasible.
It's difficult to say without more details, but I would consider a memory mapped file. How you create the file depends on whether you need to communicate between sessions or not. You would also need a notification mechanism when new data was posted. You could do that with a registered message, but again that's only possible if your processes are in the same session/desktop.
I can't really be more specific without knowing the details of the requirement.
I am creating a Daemon in Unix that will have exclusive access to a serial port "/dev/tty01". I am planning to create a Master - Slave process paradigm where there is one master (the daemon) and multiple slaves.
I was thinking of having a structure in "Shared memory" where the slaves can access, and there is only one writer to the memory so I most likely wont need a semaphore. The data will be updated fairly slowly, once every minute for example.
I was looking into what would be the best possible way to do this, also if I have a structure in shared memory, how can I guarantee that the structure will be contiguous in memory? It is a requirement I must have.
The master program will have its own internal data structure that is being updated from the serial port, and then it will modify the data and send it out to a global structure that is in shared memory for the clients to use.
I dont have much experience in Unix IPC, but what would be the easiest way to do this? By the way the clients will all be different processes ran by other users locally on the system
It must used shared memory it is a requirement of the project. Also, is it possible to copy one structure into another if the second structure has different data types?
A shared memory segment is a contiguous piece of memory from your process' view.
The calls to create and handle shared memory are rather simple (shmctl/shmat/shmdt)
The layout of the structures in the memory is up to you. Best is a fixed header like a type field and the rest as an union.
For your client processes you could provide a little lib (static or shared) with a set of functions to retrieve data, thereby hiding the shared memory and the structures.
Are you meant to protect against simultanously reads/writes to file mapped memory that is open by multiple processes?
For example if a string in the memory is "hello" and one process writes "hi..." over it, am I correct to say that another process that reads at the same time may get an intermittant value like "hi.lo"?
Basically what I am asking is how do people protect again these sorts of things. Are you meant to use semaphores? Do these work across processes?
Yes, if you need to protect against multiple writers or avoid reading partial updates then a shared Mutex / Semaphore used by each process would work to control access to the shared data.
There is some sample code which does this at the bottom of this MSDN article: Memory-Mapped Files in .NET 4.0
Goal
Pass images generated by one process efficiently and at very high speed to another process. The two processes run on the same machine and on the same desktop. The operating system may be WinXP, Vista and Win7.
Detailed description
The first process is solely for controlling the communication with a device which produces the images. These images are about 500x300px in size and may be updated up to several hundred times per second. The second process needs these images to process them. The first process uses a third party API to paint the images from the device to a HDC. This HDC has to be provided by me.
Note: There is already a connection open between the two processes. They are communicating via anonymous pipes and share memory mapped file views.
Thoughts
How would I achieve this goal with as little work as possible? And I mean both work for the computer and me (of course ;)). I am using Delphi, so maybe there is some component available for doing this? I think I could always paint to any image component's HDC, save the content to memory stream, copy the contents via the memory mapped file, unpack it on the other side and paint it there to the destination HDC. I also read about a IPicture interface which can be used to marshal images. I need it as quick as possible, so the less overhead the better. I don't want the machine to be stressed just by copying some images.
What are your ideas? I appreciate every thought on this!
Use a Memory Mapped File.
For a Delphi reference see Memory-mapped Files in Delphi and Shared Memory in Delphi.
For a more versatile approach you can look at using pipes or sending bitmap data via TCP. This would allow you to distribute the image data between nodes more easily, if necessary.
Use shared memory to pass the image data, and something else (named pipes, sockets, ...) to coordinate the handover.
In some cases, you can pass HBITMAP handles across processes. I've seen it done before (yes, on XP/Vista), and was surprised as everyone else on the team when one of my co-workers showed me.
If memory serves me correctly, I believe it will work if the HBITMAP was allocated with one of the GDI function (CreateBitmap, CreateCompatibleBitmap,CreateDIBitmap,etc...) HBIMAP handles created by LoadBitmap will not work as it's just a pointer to an in-proc resource.
That, and I think when you share the HBITMAP across to the other process, don't try to do anything special with it other than normal BitBlt operations.
At least that's what I remember. We got lucky because our graphic libraries were already written to manage all images as HBITMAPs.
YMMV
Ok it seems as if memory mapped files and pipes are the right way to go. That is not too bad because the two processes already share a MMF and two pipes (for bidirectional communication). The only thing left to solve was how to pass the data with as little copy operations as possible.
The design which works quite well looks as follows (sequential flow):
Process 1 (wants image)
give signal to process 2 (via pipe 1) to store image in shared memory
go to sleep and wait for response (blocking read from pipe 2)
Process 2 (provides images)
on signal (via pipe 1) wake up and tell hardware device to paint to HDC 1 (this is backed by shared memory, see below)
give signal to process 1 (via pipe 2)
go to sleep and wait for new job (via pipe 1)
Process 1 (wants image)
on signal (via pipe 2) wake up and paint from shared memory to destination HDC 2
Now for the image transfer via shared memory (my goal was to use not more than one additional copy operation):
Process 2 creates a HBITMAP via CreateDIBSection and provides the handle of the file mapping and the offset of the mapped view. Thus the image data lives in the shared memory. This creates an HBITMAP which is selected into HDC 1 (which is also created by process 2) and which will be used from now on by process 2.
Process 1 uses StretchDIBits with a pointer to the mapped view's memory (as described here). This seems to be the only function for getting bits from memory directly into another HDC (in this case HDC 2). Other functions would copy them first into some intermediary buffer before you could transfer them from there to the final HDC.
So in the end it seems the bits needed to be transferred are about twice as much as in the beginning. But I think this is as good as it gets unless sharing GDI handles between processes would be possible.
Note: I used pipes instead of signals because I need to transfer some additional data, too.
As I can see this, you have two options:
Pass only the image handle / pointer to other process, so both processes work only on one collection of images.
Copy the image content to other process and work on a copy from then on.
Which approach is best depends on your design. Best tool for both approaches would be "memory mapped files", or "named pipes". This are the fastest you can get. Memory mapped files are probaly the fastest form of inter process communication but have the donwside that there is no "client-server" paradigm build into them. So you have to synchronize the acces to MMF yourself. Named pipes on the other hand are almost as fast but have the client-server paradigm build right into them. The difference in speed comes mainly from that.
Now because of the share speed of the updates, the first approach could be better, but then you have to watch out for synchronization between processes, so they do not read / write to single image at the same time. Also some sort of caching or other smart tehniques could be used, so you reduce your traffic to minimum. When facing such high level of communications there is always advisable to look for means of reducing that level if possible.
For a very fast implementation of IPC based on named pipes you can use my IPC implementation. It is message oriented so you do not have to worry about pipe technical details. It also uses thread pool behind the scenes and has mininal additional overhead.
You can stress test it and see for yourself (a typical message takes 0.1 ms for full client-server request-response cycle).