Streaming UYVY video using SDL - sdl-2

I am trying a program to capture image data from a camera using v4l2. I have grabbed 4 frames and stored them in a buffer of continuous memory. The data format of the image in the buffer is UYVY. I need help in knowing the steps to be followed to map the image buffer or copy them to a texture so that I can stream them like a video.
I tried converting the UYVY files into BMP format and stream them using SDL_loadBMP() function but the frame rate is very low.
EDIT: Here's the code for SDL:
The "buf" is the image buffer which consists of bmp image data. I have converted the UYVY files to BMP and passed here. I need help in streaming the UYVY directly.
void video_stream(unsigned char* buf){
SDL_Surface *image;
SDL_Surface * screen;
SDL_Renderer *renderer;
SDL_Texture* texture;
SDL_Window *window;
SDL_RWops* rw;
SDL_Rect recta;
SDL_Init(SDL_INIT_EVERYTHING);
recta.x=0;
recta.y=0;
recta.w=s_format.fmt.pix.width;
recta.h=s_format.fmt.pix.height;
window=SDL_CreateWindow("Streaming", 0, 0, s_format.fmt.pix.width, s_format.fmt.pix.height, SDL_WINDOW_RESIZABLE);
rw=SDL_RWFromMem(buf,s_format.fmt.pix.width*s_format.fmt.pix.height*NUMBER_OF_BYTES_PER_PIXEL+OFFSET);
renderer=SDL_CreateRenderer(window,-1,SDL_RENDERER_ACCELERATED);
SDL_RenderSetLogicalSize(renderer,s_format.fmt.pix.width,s_format.fmt.pix.height);
SDL_SetRenderDrawColor(renderer,0,0,0,50);
SDL_RenderClear(renderer);
image=SDL_LoadBMP_RW(rw,1);
texture=SDL_CreateTextureFromSurface(renderer,image);
SDL_RenderCopy(renderer,texture,NULL,&recta);
SDL_RenderPresent(renderer);
SDL_DestroyRenderer(renderer);
SDL_FreeSurface(image);
SDL_DestroyTexture(texture);
SDL_DestroyRenderer(renderer);
SDL_DestroyWindow(window);
SDL_Quit();
}

SDL_CreateTexture() with SDL_PIXELFORMAT_UYVY. Upload new frames using SDL_UpdateTexture(). Render using SDL_RenderCopy.
You can use SDL_TEXTUREACCESS_STREAMING and SDL_LockTexture()/SDL_UnlockTexture() to speed up texture updates.

Related

eglCreateImageKHR is working only with RGBA8888 format. How to send RGB888 data?

I implemented an opengl-es application running on mali-400 gpu. I grab the 1280x960 RGB buffer from camera and render on GPU , I use zero copy operation using EGL_EXT_image_dma_buf_import extension.
my problem is that the Besler camera gives frames in RGB888 format.
but eglCreateImageKHR is working only if I give RGBA8888 format . If I pass RGB888 to eglCreateImageKHR it gives black screen and also egl image returned is 0.
will I have to convert RGB888 to RGBA8888 before passing? what are the other options. below is my code.
EGLint egl_img_attr[] = { EGL_IMAGE_PRESERVED_KHR, EGL_TRUE,
EGL_DMA_BUF_PLANE0_FD_EXT, buffer->dbuf_fd,
EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
EGL_DMA_BUF_PLANE0_PITCH_EXT, TEX_WIDTH * 3,
EGL_WIDTH, TEX_WIDTH,
EGL_HEIGHT, TEX_HEIGHT,
EGL_LINUX_DRM_FOURCC_EXT, DRM_FORMAT_RGBA8888 ,
EGL_NONE, EGL_NONE };
buffer->egl_img = eglCreateImageKHR(egl_dpy, EGL_NO_CONTEXT, EGL_LINUX_DMA_BUF_EXT, (EGLClientBuffer)0, egl_img_attr);

Why I need the pixel format to decompress images with libjpeg turbo

Background story: Right now I am working on an image encode and decode service. I will attach the service to a sensor (ex. camera) like to compress the image, send it over the network to receive it at the end with a client service, where the image is finaly decompressed and used. For this whole compression/decompression stuff I decided to use libJPEG Turbo, because it is the fastest library that I found.
Problem: In my mind a JPEG file is self describing and it should be possible to read it with every tool. But why I need for the compression and decompression with libJPEG Turbo additionally the pixel format (GRAY, RGB, BGR, RGBX, ...)? How can I extract/decompress the pixel format from the compressed JPEG (Byte Array)?
My code - compression (c++):
tjhandle _jpegCompressor = tjInitCompress();
tjCompress2(_jpegCompressor, buffer, width, 0, height, TJPF_RGB, &_compressedImage, &_jpegSize, TJSAMP_444, JPEG_QUALITY, TJFLAG_FASTDCT);
tjDestroy(_jpegCompressor);
My code - decompression (c++):
int width, height;
tjDecompressHeader(_jpegDecompressor, _compressedImage, _jpegSize, &width, &height);
size_t size = width*height*channels;
unsigned char* buffer = new unsigned char[size];
tjDecompress2(_jpegDecompressor, _compressedImage, _jpegSize, buffer, width, 0, height, TJPF_RGB, TJFLAG_FASTDCT);
tjDestroy(_jpegDecompressor);
Tests:
I tested it out and the only importance is that I use for compression and decompression the same pixel format. If I use both times TJPF_BGR (or TJPF_RGB) it works, but if I use for decompression a different pixel format the channels are flipped.
Interestingly If I use the tjDecompress function (not tjDecompress2) I need to compress with TJ_RGB and can decompress with TJ_RGB or TJ_BGR to get a right result... If I start to compress with TJ_BGR the color is flipped at the end. What do I missunderstand?
tjDecompress(_jpegDecompressor, _compressedImage, _jpegSize, buffer, width, 0, height, 3, TJFLAG_FASTDCT);

How to render a scaled SVG to a QImage?

There is a QSvgRenderer class in QtSvg module which can render image onto QPaintDevice. This one can be QImage. In that case we will create:
Image svgBufferImage(renderer.defaultSize(), QImage::Format_ARGB32);
But how to render to a QImage of different size than default from the SVG renderer? Since the SVG format image can be scaled without quality loss, is it possible to generate static images, like PNG, from SVG files using QSvgRenderer?
Does anyone have a better idea? Basically I need to create images like PNG from SVG files in different sizes.
Just give your QImage the desired size. The SVG renderer will scale to fit the whole image.
#include <QApplication>
#include <QSvgRenderer>
#include <QPainter>
#include <QImage>
// In your .pro file:
// QT += svg
int main(int argc, char **argv)
{
// A QApplication instance is necessary if fonts are used in the SVG
QApplication app(argc, argv);
// Load your SVG
QSvgRenderer renderer(QString("./svg-logo-h.svg"));
// Prepare a QImage with desired characteritisc
QImage image(500, 200, QImage::Format_ARGB32);
image.fill(0xaaA08080); // partly transparent red-ish background
// Get QPainter that paints to the image
QPainter painter(&image);
renderer.render(&painter);
// Save, image format based on file extension
image.save("./svg-logo-h.png");
}
This will create an 500x200 PNG image from the passed in SVG file.
Example output with an SVG image from the SVG logos page:
Here is complete answer:
QImage QPixmap::toImage()
If the pixmap has 1-bit depth, the returned image will also be 1 bit deep. Images with more bits will be returned in a format closely represents the underlying system. Usually this will be QImage::Format_ARGB32_Premultiplied for pixmaps with an alpha and QImage::Format_RGB32 or QImage::Format_RGB16 for pixmaps without alpha.
QImage img = QIcon("filepath.svg").pixmap(QSize(requiredsize)).toImage()
Also copy from above answer
// Save, image format based on file extension
image.save("./svg-logo-h.png");
If you have simple svg, like icon or small image, for replacing color for example, it's better to use NanoSVG lib
It has Qt wrapper, coded by me Qt Svg Render.
It's very small, only 2 headers and one cpp.
Usage:
QString svgPath="your svg Path";
QString pngPath="path for png";
QSize imageSize(your image Path size);
const QSize resultSize(size that you want to get);
QImage image(sizeW, sizeH, Format);//End of param
QFile svgFile(svgPath);
if (svgFile.open(QIODevice::ReadOnly))
{
int svgFileSize=svgFile.size();
QByteArray ba;
ba.resize(svgFileSize + 1);
svgFile.read(ba.data(), svgFileSize);
ba.back() = {};
if (auto const nsi(nsvgParse(ba.data(), "px", 96)); nsi)
{
image.fill(Qt::transparent);
QPainter p(&image);
p.setRenderHint(QPainter::Antialiasing, true);
drawSVGImage(&p, nsi, resultSize.width(), resultSize.height());
nsvgDelete(nsi);
image.save(pngPath);// if you want to save image
}
}

how to display data of yuv format without converting rgb in OpenGL ES?

I have being study about OpenGL ES for iOS.
I wonder that data of YUV format is can display without converting RGB.
In most, the yuv data have to convert RGB for display. But, converting process is very slow, Then, that is not display smoothly.
So, I would like to try to dispaly YUV data without convert to RGB.
Is it possible? If possible, what can I do?
Please, let me give a advice.
I think it is not possible in OpenGL ES to display YUV data without convert to RGB data.
You can do this very easily using OpenGL ES 2.0 shaders. I use this technique for my super-fast iOS camera app SnappyCam. The fragment shader would perform the matrix multiplication to take you from YCbCr ("YUV") to RGB. You can have each {Y, Cb, Cr} channel in a separate GL_LUMINANCE texture, or combine the {Cb, Cr} textures together using a GL_LUMINANCE_ALPHA texture if your chrominance data is already interleaved (Apple call this a bi-planar format).
See my related answer to the question YUV to RGBA on Apple A4, should I use shaders or NEON? here on StackOverflow.
You may also do this using the fixed rendering pipeline of ES 1.1, but I haven't tried it. I would look toward the the texture blending functions, e.g. given in this OpenGL Texture Combiners Wiki Page.
Is you are looking solution for IOS, iPhone application then there are solution for that.
This is way convert CMSampleBufferRef to UIImage when video pixel type is set as kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
-(UIImage *) imageFromSamplePlanerPixelBuffer:(CMSampleBufferRef) sampleBuffer{
#autoreleasepool {
// Get a CMSampleBuffer's Core Video image buffer for the media data
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
// Lock the base address of the pixel buffer
CVPixelBufferLockBaseAddress(imageBuffer, 0);
// Get the number of bytes per row for the plane pixel buffer
void *baseAddress = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0);
// Get the number of bytes per row for the plane pixel buffer
size_t bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer,0);
// Get the pixel buffer width and height
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
// Create a device-dependent gray color space
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceGray();
// Create a bitmap graphics context with the sample buffer data
CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8,
bytesPerRow, colorSpace, kCGImageAlphaNone);
// Create a Quartz image from the pixel data in the bitmap graphics context
CGImageRef quartzImage = CGBitmapContextCreateImage(context);
// Unlock the pixel buffer
CVPixelBufferUnlockBaseAddress(imageBuffer,0);
// Free up the context and color space
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
// Create an image object from the Quartz image
UIImage *image = [UIImage imageWithCGImage:quartzImage];
// Release the Quartz image
CGImageRelease(quartzImage);
return (image);
}
}
If you are looking for Mobile devices then i can provide others to.

how can I get information from every pixel from a CVImageBufferRef type?(OSX)

I'm trying to extract pixel data from individual frames of a QT Movie.
I think I need to use CV, because QTKit and NSImage would
be too slow...
I need to compare each pixel of the image in the buffer (CVImageBufferRef) containing the current frame of the webcam (iSight).
So I need speed.
Sorry for my bad english language, I'm Italian.
See the code in the question how to convert a CVImageBufferRef to UIImage, which is a bigger question but covers the same ground. Here's the base code you'd need (from the OP of that question):
CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
/*Lock the image buffer*/
CVPixelBufferLockBaseAddress(imageBuffer,0);
/*Get information about the image*/
uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddress(imageBuffer);
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
size_t width = CVPixelBufferGetWidth(imageBuffer);
size_t height = CVPixelBufferGetHeight(imageBuffer);
/*We unlock the image buffer*/
CVPixelBufferUnlockBaseAddress(imageBuffer,0);
Be sure to read the accepted answer for a fuller explanation of the data format.

Resources