We have a custom camera driver for Arm Based SoC running embedded Linux.
I am using wayland APIs to show the camera frames on the display, for that I am using opencv based color conversion, which is taking too much CPU.(I can not display UYVY directly on the display because of some limitations, so color conversion is required)
cv::cvtColor(myuv, mrgb, cv::COLOR_YUV2BGRA_UYVY);
Now I want to use OpenGL ES APIs for color conversion to display the converted RGB frame on the display
and second task is to get the RGB buffer back to CPU for further processing the camera frame.
I am pretty new with OpenGL ES/EGL/Wayland, Please suggest any good example code(c/c++) based on OpenGL ES, EGL and wayland combination.
Related
When I use AVFoundation to create an 8K (7680 x 4320) MP4 with frames directly drawn onto pixel buffers obtained from the pixel buffer pool, it works with kCVPixelFormatType_32ARGB.
However if I use kCVPixelFormatType_32BGRA, the entire video has a uniform dark green color instead of the actual contents. This problem occurs for resolutions above 8176 x 4088.
What could be causing this problem?
AVAssetWriter.h in SDK 10.15 and in SDK 11.3 says:
The H.264 encoder natively supports ... If you need to work in the RGB domain then kCVPixelFormatType_32BGRA is recommended on iOS and kCVPixelFormatType_32ARGB is recommended on OSX.
AVAssetWriter.h in SDK 12.3 however says:
The H.264 and HEVC encoders natively support ... If you need to work in the RGB domain then kCVPixelFormatType_32BGRA is recommended on iOS and macOS.
AVAssetWriter.h on all three SDKs however also says:
If you are working with high bit depth sources the following yuv pixel formats are recommended when encoding to ProRes: kCVPixelFormatType_4444AYpCbCr16, kCVPixelFormatType_422YpCbCr16, and kCVPixelFormatType_422YpCbCr10. When working in the RGB domain kCVPixelFormatType_64ARGB is recommended.
Whatever be the recommendations, the below prelude states that all of them are just for optimal performance and not for error free encoding!
For optimal performance the format of the pixel buffer should match one of the native formats supported by the selected video encoder. Below are some recommendations:
Now, Keynote movie export with H.264 compression also results in the same problem with the same size limits on my Mid-2012 15-inch Retina MacBook Pro running Catalina (supports upto Keynote 11.1). This problem doesn't occur on a later Mac running Monterey where the latest version 12.2 of Keynote is supported.
I have not included code because Keynote movie export is a simple means to reproduce and understand the problem.
My motivation for asking this question is to obtain clarity on:
What is the right pixel format to use for MP4 creation?
What are safe size limits under which MP4 creation will be problem free?
Although Apple's documentation clearly states they support stereoscopic views on the Mac with the kCGLPFAStereo and NSOpenGLPFAStereo pixel format attributes, I've been unable to get any semblance of a stereoscopic pixel format object.
I've been able to create stereoscopic output recognized by 3DTV displays by manually specifying the format as Side-by-side, but it should be recognized automatically by the hardware over HDMI.
How does one create a stereoscopic display in Cocoa or Core OpenGL with NSOpenGLPFAStereo or kCGLPFAStereo?
I am using the java API with the Tango Peanut phone, and I wanted to know if there is a way to get the raw IR image from the RGB-IR camera that the depth sensor uses. I know that a quarter of the pixels from the RGB-IR camera is IR data. I also know that all of the 4MP RGB-IR image gets put into the superframe and then converted to YUV. However it was unclear on how to decode the IR channel, or if it is even possible at this point. If its lost inside the YUV superframe, is there any other way I can retrieve the raw IR image?
In order to get the depth data:
http://www.pocketmagic.net/wp-content/uploads/2014/05/sbrl.zip
taken from http://palcu.blogspot.ro/2014/04/google-project-tango-hackathon-in.html
You might also wanna try
https://github.com/googlesamples/tango-examples-c/wiki/Depth:-Point-Cloud-Viewer
http://www.pocketmagic.net/2014/05/google-tango-hackathon-in-timisoara/#.VCQ6H1eHuiN
Please keep me posted on success/failure. Are you on a tablet or phone?
We are using a directshow interface to capture images from a video stream. These images are presented in a fixed size window.
Once we have captured an image we store it as a bitmap. Downstream we have the ability to add annotation to the image, for example letters in a fixed size font.
In one of our desktop environments, the annotation has started appearing at half the size that it normally appears at. This implies that the image we are merging the text onto has dimensions that are maybe twice as large.
The system that this happens on is a shared resource as in some unknown individual has installed software on the system that differs from our baseline.
We have two approaches - the 1st is to reimage the system to get our default text size behaviour back. The 2nd is to figure out how directshow manages image dimensions so that we can set the scaling on the image correctly.
A survey of the directshow literature indicates that the above is not a trivial task. The original work was done by another team that did not document what they did. Can anybody point us in the direction of what directshow object we want to deal with to properly size the sampled image?
DirectShow - as a framework - does not deal with resolutions directly. Your video source (such as capture hardware) is capable of providing video feed in certain resolution which you possibly can change. You normally use IAMStreamConfig as described in Configure the Video Output Format in order to choose capture resolution.
Sometimes you cannot affect capture resolution and you need to resample the image in whatever dimensions you captured it. There is no stock filter for this, however Media Foundation provides a suitable Video Resizer DSP which does most of the task. Unfortunately it does not fit DirectShow pipeline smoothly, so you need fitting and/or custom filter for resizing.
When filters connect in DirectShow, they have an AM_MEDIA_TYPE. Here you will find a VIDEOINFOHEADER with a BITMAPINFOHEADER and this header has a biWidth and biHeight.
Try to build the FilterGraph manually (with GraphEdit or GraphStudioNext) and inspect these fields.
I am working on an application and I have a problem I just cant seem to find a solution for. The application is written in vc++. What I need to do is display a YUV video feed with text on top of it.
Right now it works correctly by drawing the text in the OnPaint method using GDI and the video on a DirectDraw overlay. I need to get rid of the overlay because it causes to many problems. It wont work on some video cards, vista, 7, etc.
I cant figure out a way to complete the same thing in a more compatible way. I can draw the video using DirectDraw with a back buffer and copy it to the primary buffer just fine. The issue here is that the text being drawn in GDI flashes because of the amount of times the video is refreshed. I would really like to keep the code to draw the text intact if possible since it works well.
Is there a way to draw the text directly to a DirectDraw buffer or memory buffer or something and then blt it to the back buffer? Should I be looking at another method all together? The two important OS's are XP and 7. If anyone has any ideas just let me know and I will test them out. Thanks.
Try to look into DirectShow and the Ticker sample on microsoft.com:
DirectShow Ticker sample
This sample uses the Video Mixing Renderer to blend video and text. It uses the IVMRMixerBitmap9 interface to blend text onto the bottom portion of the video window.
DirectShow is for building filter graphs for playing back audio or video streams an adding different filters for different effects and manipulation of video and audio samples.
Instead of using the Video Mixing Renderer of DirectShow, you can also use the ISampleGrabber interface. The advantage is, that it is a filter which can be used with other renderers as well, for example when not showing the video on the screen but streaming it over network or dumping it to a file.