How can one share depth images between two processes? - image

Question

I have 4 different depth cameras available to me: Kinect, Xtion, PMD nano, Softkinetic DepthSense.
I have the libraries that know how to read all of them: OpenNI, PMD drivers, Softkinetic drivers.
I would ideally like to make a simple grabber for each kind of camera and then just use it as a plugin into any other program i.e. get fast, non redundant access (i.e. not too many memory copies) to the data stream.
One of the problems is that in many cases I dont have the right library in 32 or 64 bit so I cant compile all grabbers in the same project.
What is the best way to achieve this?

Answer 1

I am a researcher so this idea isnt necessarily useful for production code but given this scenario my best solution has been to create a server process for each type of camera. Each server process knows how to load its own type of camera stream and then throws it into a shared memory space that other processes can read from.
It is obviously possible to use different kind of locking mechanisms but I have left the below code without any locks.
The server process will include the following:
#define BOOST_ALL_NO_LIB
#include <boost/interprocess/shared_memory_object.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <boost/interprocess/sync/scoped_lock.hpp>
#include <boost/interprocess/sync/interprocess_mutex.hpp>
using namespace std;
using namespace boost::interprocess;
struct sharedImage
{
enum { width = 320 };
enum { height = 240 };
enum { dataLength = width*height*sizeof(unsigned short) };
sharedImage(){}
interprocess_mutex mutex;
unsigned short data[dataLength];
};
shared_memory_object shm;
sharedImage * sIm;
mapped_region region;
int setupSharedMemory(){
// Clear the object if it exists
shared_memory_object::remove("ImageMem");
shm = shared_memory_object(create_only /*only create*/,"ImageMem" /*name*/,read_write/*read-write mode*/);
printf("Size:%i\n",sizeof(sharedImage));
//Set size
shm.truncate(sizeof(sharedImage));
//Map the whole shared memory in this process
region = mapped_region(shm, read_write);
//Get the address of the mapped region
void * addr = region.get_address();
//Construct the shared structure in the preallocated memory of shm
sIm = new (addr) sharedImage;
return 0;
}
int shutdownSharedMemory(){
shared_memory_object::remove("ImageMem");
return 0;
}
To start it up call setupSharedMemory() and to shut down call shutdownSharedMemory().
All the values are hard coded in this simple example but its easy to imagine making it more flexible.
Now lets assume that you are using SoftKinetic's DepthSense. So then you could write the following callback for the Depth node.
void onNewDepthSample(DepthNode node, DepthNode::NewSampleReceivedData data) {
//scoped_lock<interprocess_mutex> lock(sIm->mutex);
memcpy(sIm->data, data.depthMap, sIm->dataLength);
}
What this does is simply copies the latest depth map into the shared memory space.
You could also add a timestamp and a lock and anything else you need but this basic code works well enough for me so I will leave it as it is.
Now in some other process you can access the data in a very similar fashion.
The code below is what I use to get the live SoftKinetic DepthSense depth stream into Matlab for real time processing. This method has a huge advantage over trying to write my own mex wrapper specifically for SoftKinetic because I can use the same code for all the other cameras if I write servers for them.
#include <math.h>
#include <windows.h>
#include "mex.h"
#define BOOST_ALL_NO_LIB
#include <boost/interprocess/shared_memory_object.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <boost/interprocess/sync/scoped_lock.hpp>
#include <boost/interprocess/sync/interprocess_mutex.hpp>
#include <iostream>
#include <cstdio>
#include <cstdlib>
using namespace boost::interprocess;
struct sharedImage
{
enum { width = 320 };
enum { height = 240 };
enum { dataLength = width*height*sizeof(short) };
sharedImage(): dirty(true){}
interprocess_mutex mutex;
uint8_t data[dataLength];
bool dirty;
};
void getFrame(unsigned short *D)
{
//Open the shared memory object.
shared_memory_object shm(open_only ,"ImageMem", read_write);
//Map the whole shared memory in this process
mapped_region region(shm ,read_write);
//Get the address of the mapped region
void * addr = region.get_address();
//Construct the shared structure in memory
sharedImage * sIm = static_cast<sharedImage*>(addr);
//scoped_lock<interprocess_mutex> lock(sIm->mutex);
memcpy((char*)D, (char*)sIm->data, sIm->dataLength);
}
void mexFunction(int nlhs, mxArray *plhs[ ], int nrhs, const mxArray *prhs[ ])
{
// Build outputs
mwSize dims[2] = {320, 240};
plhs[0] = mxCreateNumericArray(2, dims, mxUINT16_CLASS, mxREAL);
unsigned short *D = (unsigned short*)mxGetData(plhs[0]);
try
{
getFrame(D);
}
catch (interprocess_exception &ex)
{
mexPrintf("getFrame:%s\n", ex.what());
}
}
which on my computer I compile in Matlab with: mex getSKFrame.cpp -IC:\Development\boost_1_48_0
And then finally to use it in Matlab: D = getSKFrame()'; imagesc(D)