New Tango 3D Reconstruction API - google-project-tango

Question

I use the new 3D reconstruction API's (MIRA release). I have a problem when a call the Tango3DR_update function. It returns TANGO_3DR_INVALID code when I set the parameters associated with an image camera (const Tango3DR_ImageBuffer * image * const Tango3DR_Pose image_pose, Tango3DR_CameraCalibration const * calibration). I have checked my parameters, they seem to be correct. When I call this function without image parameters, this to work properly ... Is this a known bug?
thank you in advance for your answers.

Answer 1

TLDR; The support library ImageBufferManager has a bug with strides. Do color_image.stride = image_buffer->width; when creating your Tango3DR_ImageBuffer.
I think there are two things :
Image Format
First, you have to make sure to use the TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP. You can do that by using the ImageBufferManager from the support library.
ImageBufferManager and strides
Second, there is a catch if you use the support library ImageBufferManager though. TangoSupport_getLatestImageBuffer seems to fail to initialize the stride of the returned image (I got 0 and some other very large values) which the 3DR library doesn't like. The original TangoImageBuffer from OnColorAvailable has stride=1280 (=image_width) and forcing that value on the TangoImageBuffer returned
from the ImageBufferManager seems to fix the issue. I believe this is a bug in ImageBufferManager.
This means doing
color_image.stride = image_buffer->width;
instead of
color_image.stride = image_buffer->stride
when creating the Tango3DR_ImageBuffer.
Full code example
I got it working with the following code in my Render method :
TangoImageBuffer* image_buffer;
ret = TangoSupport_getLatestImageBuffer(
image_buffer_manager_, &image_buffer);
if (ret != TANGO_SUCCESS) {
LOG(ERROR) << "Error in TangoSupport_getLatestImageBuffer";
}
...
Tango3DR_ImageBuffer color_image;
color_image.width = image_buffer->width;
color_image.height = image_buffer->height;
// VERY Important - The support library ImageBufferManager seems to have
// a bug where it will always put the stride of the returned buffer
// at 0, which causes 3DR to fail
color_image.stride = image_buffer->width;
color_image.timestamp = image_buffer->timestamp;
color_image.format = (Tango3DR_ImageFormatType)image_buffer->format;
color_image.data = image_buffer->data;
ret = Tango3DR_update(
tango_3dr_context_,
&cloud,
&depth_pose_3dr,
&color_image,
&color_pose_3dr,
&tango_3dr_calibration_,
&updated_indices);
I am using the ImageManager from the support library. So my OnColorAvailable looks like that
void SynchronizationApplication::OnColorAvailable(
const TangoImageBuffer* buffer) {
if (tango_3dr_enabled_ && tango_3dr_use_color_) {
TangoErrorType ret = TangoSupport_updateImageBuffer(
image_buffer_manager_, buffer);
if (ret != TANGO_SUCCESS) {
LOG(ERROR) << "Error in TangoSupport_updatePointCloud";
}
}
}
And the image_buffer_manager_ is initialized as follow (the pixel format might be important).
TangoSupport_createImageBufferManager(
TANGO_HAL_PIXEL_FORMAT_YCrCb_420_SP,
image_width_,
image_height_,
&image_buffer_manager_
);
I am copying the calibration as follow :
void CopyCalibrationTangoTo3DR(const TangoCameraIntrinsics& tango,
Tango3DR_CameraCalibration* out) {
out->calibration_type =
(Tango3DR_TangoCalibrationType)tango.calibration_type;
out->cx = tango.cx;
out->cy = tango.cy;
memcpy(out->distortion, tango.distortion, sizeof(double) * 5);
out->fx = tango.fx;
out->fy = tango.fy;
out->height = tango.height;
out->width = tango.width;
}