I'd like to render the live image data on a GL surface (as shown in various Project Tango samples), and at the same time record (encode) it via a MediaCodec.
(On an Android Lollipop device, I've accomplished that using the camera2 interface and multiple surface targets, which works fine, but thus far Tango is pre-Lollipop...)
From other answers, it appears that you have to use the C API to access the image data.
The C API provides two camera frame functions -- TangoService_connectTextureId() and TangoService_connectOnFrameAvailable(). However, the documentation states "Use either TangoService_connectTextureId() or TangoService_connectOnFrameAvailable() but not both."
Why not both?
How do I best render and retrieve the image data?
The Pythagoras release now allows for simultaneous use of color and color texture callbacks now. That said, you want to use the connectOnFrameAvailable if you want to process the image, you'd end up doing extra unnecessary work if you try and peel it out of the texture.
Related
Dose cobalt support youtube 360 video(Spherical Video)? If yes, how it's been implemented, is there any document for it? Does it need the platform to do extra things to support it?
Almost. There is still some small remaining work that prevents this from being a simple yes, but the vast bulk of the work is done and has been shown to function.
A document will soon be appearing in the source tree explaining all this, but here is a preview...
In order to support spherical video, a platform will have to support decode-to-texture, introduced in Starboard API version 4. Cobalt will choose between punch-out and decode-to-texture when creating an SbPlayer, based on whether a mesh transform has been applied to the video tag. In decode-to-texture mode, every frame, the video texture will then be queried from the player, and rendered into the UI graphics plane with the current transform applied.
I'm trying to do pretty much what TangoARScreen does but with multithreaded rendering on in Unity. I did some experiments and I'm stuck.
I tried several things, such as letting Tango render into the OES texture that would be then blitted into a regular Texture2D in Unity, but OpenGL keeps complaining about invalid display when I try to use it. Probably not even OnTangoCameraTextureAvailable is called in the correct GL context? Hard to say when you have no idea how Tango Core works internally.
Is registering a YUV texture via TangoService_Experimental_connectTextureIdUnity the way to go? I'd have to deal with YUV2RGB conversion I assume. Or should I use OnTangoImageMultithreadedAvailable and deal with the buffer? Render it with a custom share for instance? The documentation is pretty blank in these areas and every experiment means several wasted days at least. Did anyone get this working? Could you point me in the right direction? All I need is live camera image rendered into Unity's camera background.
Frome the April 2017: Gankino release notes: "The C API now supports getting the latest camera image's timestamp outside a GL thread .... Unity multithreaded rendering support will get added in a future release.". So I guess we need to wait a little bit.
Multithreaded rendering still can be used in applications without camera feed (with motion tracking only), choosing "YUV Texture and Raw Bytes" as overlay method in Tango Application Script.
Is it possible to use the Google Tango Java API to track an object by itself (say, in the captured video feed I tap on the object and then the device tracks it continuously) or track an object by color (e.g. If I have a black sphere against a white background)
Unfortunately, the Tango API seems designed only to track the observer (i.e. the Tango tablet/phone), at present. If you'd like to track other objects, I recommend using it with a library like OpenCV:
See:
http://opencv.org/platforms/android.html
http://docs.opencv.org/2.4/doc/tutorials/introduction/android_binary_package/dev_with_OCV_on_Android.html
You'll need some way of detecting or selecting an object, and then some way of tracking it:
http://docs.opencv.org/3.2.0/d5/d54/group__objdetect.html
http://docs.opencv.org/3.2.0/d9/df8/group__tracking.html
I am working on a project, which is Android app that uses camera to capture a photo of some ticket and does OCR recognition for only a part of it. I have no previous experience in image processing, but I know it must be some kind of tricky way, because Android applications have small RAM limits.
I have not enough reputation points to post images so I give URLs to it.
Below, I attach image before any processing:
My aim is to automatically detect these lines of (---) and crop it so that final image look like this one:
What's more - it's important to stay open-source and do it without sending photo to some external image processing service.
You can try using Hough Transform to find the lines. OpenCV has a implementation that is open source and works on Android.
HoughLineP is a very efficient Version of the HoughTransform to find Line Segments.
Olena is definitely the way to go!. It's a generic image processing library, but the interesting part is an module that's called Scribo.
Scribo will do document analysis on the picture to extract text and/or image regions, and optionally send text regions to tesseract for recognition.
Being feasible for Android or not is something that I couldn't tell. I've tried it on OSX and Linux systems and it shows great potential.
I am learning to use OpenGL ES 2.0 by using MoSync to write cross platform C code. I have already managed to draw basic shapes such as a triangle, square and circle so the next stage is to draw some text to the screen. After reading various books, tutorials and forum posts I realise I have to create a texture atlas bitmap.
I have a file with the text I want to use, i.e 0-9 a-z image file. Before I can upload and bind it to a texture object I first need to upload the image to OpenGL. Various tutorials use UIImage or BitmapFactory to upload the image but I cannot use these as MoSync does not contain their header files. Could anyone suggest a way to load my image file to OPenGL?
To use MoSync on the Android platform you are probably going to have to make a native library for MoSync and your OpenGL ES code in C++. Most OpenGL ES projects on Android are done in native code for many reasons which are detailed in this article:
http://software.intel.com/en-us/articles/porting-opengl-games-to-android-on-intel-atom-processors-part-1/
I ended up using maOpenGLTexImage(MAHandle image), which works exactly as glTexImage2D() but it uses an image resource instead and figures out pixel formats etc.