Issues drawing multiple image textures with some disapearing - opengl-es

I'm looking to use OpenGl to blend 4 images to the screen. The first image contains the background and the other 3 are images contain some cartoon with transparency. My goal is to render those 4 images at once. At every call to render I'm updating the top 3 images with new images to compose the new frame.
I'm new to OpenGL and so far I'm able to achieve blending those images however I'm noticing some horrible issues when I render. I'm seeing some of the top 3 images are missing sometimes or some are rendered but look like they where cropped by the invisible man...
Each lines represents a different image.
Image cropped issue:
Image cropped and one image missing:
How it should look like:
Any help figuring out the issue I would greatly appreciate it! Below the code I'm using.
Here is the code I use to render the images.
void MoviePreview::prepareTexture (GLuint texture, GLint format, int w, int h)
{
// Bind to generated texture
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glShadeModel(GL_FLAT);
//glShadeModel(GL_SMOOTH);
glTexImage2D(GL_TEXTURE_2D, 0, format, w, h, 0, format, GL_UNSIGNED_BYTE, 0);
if (GL_RGBA == format)
{
//glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
}
// Crop the texture rectangle
GLint rect[] = {0, h, w, -h};
glTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_CROP_RECT_OES, rect);
glDrawTexiOES(0, 0, 0, w, h);
}
void MoviePreview::resize (int w, int h)
{
LOGI("native_gl_resize %d %d", w, h);
// init open gl
//glClearColor(1.0, 1.0, 1.0, 1.0);
glEnable(GL_TEXTURE_2D);
glEnable(GL_BLEND);
glDisable(GL_DITHER);
glDisable(GL_LIGHTING);
glDisable(GL_DEPTH_TEST);
glDisable(GL_COLOR_MATERIAL);
glDisable(GL_FOG);
glDisable(GL_CULL_FACE);
glDisable(GL_STENCIL_TEST);
glDisable(GL_COLOR_LOGIC_OP);
glDisable(GL_ALPHA_TEST);
//glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);//NEW
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);//NEW
//glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA);
// Generate one texture object
glGenTextures(MAX_TEXTURES, mTexture);
check_gl_error("glGenTextures");
int frameHeight = h;//image.rows;
int frameWidth = w;//image.cols;
// first texture is our background texture
prepareTexture(mTexture[0], GL_RGBA, mBg.cols, mBg.rows);
prepareTexture(mTexture[1], GL_RGBA, frameWidth, frameHeight);
prepareTexture(mTexture[2], GL_RGBA, frameWidth, frameHeight);
prepareTexture(mTexture[3], GL_RGBA, frameWidth, frameHeight);
mSurfaceWidth = w;
mSurfaceHeight = h;
}
void MoviePreview::render (int64_t* imageIds, const int images)
{
int i = 0;
double sleepDuration = 0;
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// TODO try to see if we can just get away from always loading the bg
// since it doesn't change often might be worth loading it once and that
// is it...
//glBindTexture(GL_TEXTURE_2D, mTexture[0]);
//glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, mBg.cols, mBg.rows, GL_RGBA, GL_UNSIGNED_BYTE, mBg.ptr());
//glDrawTexiOES(0, 0, 0, mSurfaceWidth, mSurfaceHeight);
// TODO pass the image batch loader
// load images
for (i=0; i<images; i++)
{
if (0 < imageIds[i])
{
sprintf(mTempPath, "%s/f1_%lld.png",mProjectPath.c_str(), imageIds[i]);
mImageLoader[i].loadImage(mTempPath);
}
}
if (0 < mFrameDuration)
{
// here we try to control the suggested frame rate
// set. We calculate since the last show image time
// if we should sleep or not...
if (0 < mLastDrawTimestamp) {
sleepDuration = mFrameDuration - (now_ms() - mLastDrawTimestamp);
}
if (0 < sleepDuration) {
usleep((long)sleepDuration*NANO_IN_MS);
}
}
// draw images
i = 0;
for (i=0; i<images; i++)
{
if (0 < imageIds[i])
{
cv::Mat img = mImageLoader[i].getImage();
if (!img.empty())
{
glBindTexture(GL_TEXTURE_2D, mTexture[i+1]);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, img.cols, img.rows, GL_RGBA, GL_UNSIGNED_BYTE, img.ptr());
glDrawTexiOES(0, 0, 0, img.cols, img.rows);
}
}
}
mLastDrawTimestamp = now_ms();
}

The issue ended up being with my image sources getting modified by another thread while OpenGl was trying to draw them...

Related

opengl es 2.0 android c++ glGetTexImage alternative

when testing on windows the code works as expected, but on android the glGetTexImage api doesn't exist, is there an other way of getting all the pixels from OpenGL without caching them before creating the texture?
this is the code:
void Texture::Bind(int unit)
{
glActiveTexture(GL_TEXTURE0 + unit);
glBindTexture(GL_TEXTURE_2D, mTextureID);
}
GLubyte* Texture::GetPixels()
{
Bind();
int data_size = mWidth * mHeight * 4;
GLubyte* pixels = new GLubyte[mWidth * mHeight * 4];
glGetTexImage(GL_TEXTURE_2D, 0, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
return pixels;
}
glGetTexImage doesn't exist in OpenGL ES.
In OpenGL ES, you have to attach the texture to a framebuffer and read the color plane from the framebuffer by glReadPixels
Bind();
int data_size = mWidth * mHeight * 4;
GLubyte* pixels = new GLubyte[mWidth * mHeight * 4];
GLuint textureObj = ...; // the texture object - glGenTextures
GLuint fbo;
glGenFramebuffers(1, &fbo);
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureObj, 0);
glReadPixels(0, 0, mWidth, mHeight, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &fbo);

How to render Image from file using the Qt OpenGL API

I'm trying to render an image from a *.png file on to screen using the Qt OpenGL API. My implement method creates the OpenGLWindow class, which is subclass of QOpenGLWindow and QOpenGLFunctions, then overrides 2 methods: initializeGL() and paintGL().
I have code, as below:
class OpenGLWindow : public QOpenGLWindow, protected QOpenGLFunctions
{
Q_OBJECT
public:
OpenGLWindow();
protected:
void initializeGL();
void paintGL() Q_DECL_OVERRIDE;
};
OpenGLWindow::OpenGLWindow()
: QOpenGLWindow(QOpenGLWindow::NoPartialUpdate)
{
}
void OpenGLWindow::initializeGL()
{
initializeOpenGLFunctions();
}
void OpenGLWindow::paintGL()
{
// Clear buffer with blue color
glClearColor(0, 0, 1, 0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// Load image from file
QImage image;
if(!image.load(":/side1.png"))
{
qDebug() << "load image fail";
return;
}
// Draw Texture
glEnable(GL_TEXTURE_2D); // Enable texturing
glActiveTexture(GL_TEXTURE1);
GLuint textureID;
glGenTextures(1, &textureID); // Obtain an id for the texture
glBindTexture(GL_TEXTURE_2D, textureID); // Set as the current texture
QImage tex = QGLWidget::convertToGLFormat(image);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, tex.width(), tex.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, tex.bits());
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glDisable(GL_TEXTURE_2D);
}
The result is the window only displays a blue screen, with an error that the image could not be displayed. I saw that the image was loaded successfully to QImage. Have I don't something wrong or am I missing something or not? Can somebody help me?
Result screen:

OpenGL FBO performance issue

i'm working on a school project and I've come across an issue with my FBO.
game is rendered in 2 passes:
1) I render to the shadow map texture using an FBO.
2) I render scene normally to the default FBO.
Issue is, for some reason, binding the FBO when i do the first pass slows down my game by roughly 200+ fps, I really don't know what could be wrong with the FBO since it's as barebones as possible.
If i were to render the shadow map without binding, so directly to the screen, it'd be 200 fps faster, so it's not an issue of rendering, it's an issue of binding the FBO I believe.
Anyways, here's the first pass function.
//renders to the depth buffer texture.
void firstPass()
{
static GLuint shadowID = Resources::getInstance().shadowShader;
shadowBuffer->bindForWriting();//make active frame buffer the shadow buffer
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glViewport(0, 0, shadowBuffer->TEXTURE_WIDTH, shadowBuffer->TEXTURE_HEIGHT);
//render to the shadow map from the point of view of the sunlight.
glUseProgram(shadowID);
... set some uniforms
scene->render(shadowID);
shadowBuffer->unBindForWriting();//set frame buffer to default
glUseProgram(0);
}
And here is my FBO class in its entirety.
ShadowMapFBO()
{
init();
}
void bindForWriting()
{
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, FBO);
}
void unBindForWriting()
{
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
}
void bindForReading(GLenum TextureUnit, GLuint texture)
{
glActiveTexture(TextureUnit);
glBindTexture(GL_TEXTURE_2D, texture);
}
void BindTexture(GLuint textureID, GLuint location)
{
glUniform1i(textureID, location);
}
void unBindForReading()
{
glBindTexture(GL_TEXTURE_2D, 0);
}
GLuint shadowTexture;
int TEXTURE_WIDTH = 2048;
int TEXTURE_HEIGHT = 2048;
GLuint FBO;
void init()
{
glGenFramebuffers(1, &FBO);
glBindFramebuffer(GL_FRAMEBUFFER, FBO);
shadowTexture = Util::createTexture(TEXTURE_WIDTH, TEXTURE_HEIGHT, true);
glFramebufferTexture(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, shadowTexture, 0);
glDrawBuffer(GL_NONE);
glReadBuffer(GL_NONE);
GLenum Status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
if (Status != GL_FRAMEBUFFER_COMPLETE)
printf("FB error, status: 0x%x\n", Status);
else
printf("Shadow Buffer created successfully.\n");
glBindFramebuffer(GL_FRAMEBUFFER, 0);
}
Here's how I create my depth buffer texture.
static GLuint createTexture(int width, int height)
{
GLuint textureId;
glGenTextures(1, &textureId);
glBindTexture(GL_TEXTURE_2D, textureId);
glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, width, height, 0, GL_DEPTH_COMPONENT, GL_FLOAT, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);
int i;
i = glGetError();
if (i != 0)
{
std::cout << "Error happened while loading the texture: " << i << std::endl;
}
glBindTexture(GL_TEXTURE_2D, 0);
return textureId;
}

Why will my android native activity sometimes display a bad image when rendering a bitmap?

I'm trying to use NDK to display a bitmap on a full screen. The requirement force me to develop with completely native code. I use Skia to draw a SkBitmap, then display it by Opengl APIs. When my application first time run on a real android device, it always works well. However, after I keep opening and closing the program for several times, it will show a bad image. Why does this happen?
Function engine_init_display is to initialize OpenGL ES and EGL, create bitmap and load texture.
static int engine_init_display(struct engine* engine){
// initialize OpenGL ES and EGL
/*
* Here specify the attributes of the desired configuration.
* Below, we select an EGLConfig with at least 8 bits per color
* component compatible with on-screen windows
*/
const EGLint attribs[] = {
EGL_SURFACE_TYPE, EGL_WINDOW_BIT,
EGL_DEPTH_SIZE, 16,
EGL_BLUE_SIZE, 8,
EGL_GREEN_SIZE, 8,
EGL_RED_SIZE, 8,
EGL_NONE
};
EGLint w, h, dummy, format;
EGLint numConfigs;
EGLConfig config;
EGLSurface surface;
EGLContext context;
EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
eglInitialize(display, 0, 0);
/* Here, the application chooses the configuration it desires. In this
* sample, we have a very simplified selection process, where we pick
* the first EGLConfig that matches our criteria */
eglChooseConfig(display, attribs, &config, 1, &numConfigs);
/* EGL_NATIVE_VISUAL_ID is an attribute of the EGLConfig that is
* guaranteed to be accepted by ANativeWindow_setBuffersGeometry().
* As soon as we picked a EGLConfig, we can safely reconfigure the
* ANativeWindow buffers to match, using EGL_NATIVE_VISUAL_ID. */
eglGetConfigAttrib(display, config, EGL_NATIVE_VISUAL_ID, &format);
ANativeWindow_setBuffersGeometry(engine->app->window, 0, 0, format);
surface = eglCreateWindowSurface(display, config, engine->app->window, NULL);
context = eglCreateContext(display, config, NULL, NULL);
if (eglMakeCurrent(display, surface, surface, context) == EGL_FALSE) {
LOGW("Unable to eglMakeCurrent");
return -1;
}
eglQuerySurface(display, surface, EGL_WIDTH, &w);
eglQuerySurface(display, surface, EGL_HEIGHT, &h);
engine->display = display;
engine->context = context;
engine->surface = surface;
engine->width = w;
engine->height = h;
engine->state.angle = 0;
engine->format=format;
// Initialize GL state.
glHint(GL_PERSPECTIVE_CORRECTION_HINT, GL_FASTEST);
glEnable(GL_CULL_FACE);
glShadeModel(GL_SMOOTH);
glDisable(GL_DEPTH_TEST);
SkBitmap bitmap;
GLvoid* bitmapBuffer;
createBitmap(bitmap,bitmapBuffer,width,height);
drawBitmap(bitmap,width,height);
glEnable(GL_TEXTURE_2D);
glGenTextures(1,&sTexture);
glBindTexture(GL_TEXTURE_2D,sTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid*)bitmapBuffer);
glDisable(GL_TEXTURE_2D);
glFinish();
clearBitmapBuffer(bitmap,bitmapBuffer);
//engine_draw_frame(engine);
return 0;
}
Function render is to display a bitmap
void render(struct engine* engine, int width, int height){
glViewport((engine->width-width)/2, (engine->height-height)/2, width, height);
glClearColorx((GLfixed)(0.1f * 65536),(GLfixed)(0.2f * 65536),(GLfixed)(0.3f * 65536), 0x10000);
glClear(GL_COLOR_BUFFER_BIT);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glMatrixMode(GL_TEXTURE);
glLoadIdentity();
glEnable(GL_TEXTURE_2D);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterx(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glBindTexture(GL_TEXTURE_2D,sTexture);
glFrontFace(GL_CW);
glTexCoordPointer(2, GL_FLOAT, 0, textureCoords);
glVertexPointer(3, GL_FLOAT, 0, vertices);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glDisable(GL_TEXTURE_2D);
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
glFlush();
eglSwapBuffers(engine->display, engine->surface);
}
And when the window is being closed, engine_term_display will be called
static void engine_term_display(struct engine* engine) {
if (engine->display != EGL_NO_DISPLAY) {
eglMakeCurrent(engine->display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT);
if (engine->context != EGL_NO_CONTEXT) {
eglDestroyContext(engine->display, engine->context);
}
if (engine->surface != EGL_NO_SURFACE) {
eglDestroySurface(engine->display, engine->surface);
}
eglTerminate(engine->display);
}
engine->display = EGL_NO_DISPLAY;
engine->context = EGL_NO_CONTEXT;
engine->surface = EGL_NO_SURFACE;
}
update
android_main is the main entry point of my native application. I find that when this main function has returned, the whole application is still running.
void android_main(struct android_app* state) {
struct engine engine;
// Make sure glue isn't stripped.
app_dummy();
memset(&engine, 0, sizeof(engine));
state->userData = &engine;
state->onAppCmd = engine_handle_cmd;
state->onInputEvent = engine_handle_input;
engine.app = state;
ANativeActivity_setWindowFlags(engine.app->activity, AWINDOW_FLAG_FULLSCREEN, 0);
if (state->savedState != NULL) {
// We are starting with a previous saved state; restore from it.
engine.state = *(struct saved_state*)state->savedState;
}
while(1){
int ident;
int events;
struct android_poll_source* source;
if((ident=ALooper_pollAll(-1, NULL, &events,(void**)&source))>=0){
// Process this event.
if (source != NULL) {
source->process(state, source);
}
if (state->destroyRequested != 0) {
engine_term_display(&engine);
return;
}
}
}
}
I fix it by myself, use exit(0) instead of return to finish the activity completely.
if (state->destroyRequested != 0) {
engine_term_display(&engine);
exit(0);
}

Render YUV video in OpenGL of ffmpeg using CVPixelBufferRef and Shaders

I'm using to render YUV frames of ffmpeg with the iOS 5.0 method "CVOpenGLESTextureCacheCreateTextureFromImage".
I'm using like the apple example GLCameraRipple
My result in iPhone screen is this: iPhone Screen
I need to know I'm doing wrong.
I put part of my code to find errors.
ffmpeg configure frames:
ctx->p_sws_ctx = sws_getContext(ctx->p_video_ctx->width,
ctx->p_video_ctx->height,
ctx->p_video_ctx->pix_fmt,
ctx->p_video_ctx->width,
ctx->p_video_ctx->height,
PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL);
// Framebuffer for RGB data
ctx->p_frame_buffer = malloc(avpicture_get_size(PIX_FMT_YUV420P,
ctx->p_video_ctx->width,
ctx->p_video_ctx->height));
avpicture_fill((AVPicture*)ctx->p_picture_rgb, ctx->p_frame_buffer,PIX_FMT_YUV420P,
ctx->p_video_ctx->width,
ctx->p_video_ctx->height);
My render method:
if (NULL == videoTextureCache) {
NSLog(#"displayPixelBuffer error");
return;
}
CVPixelBufferRef pixelBuffer;
CVPixelBufferCreateWithBytes(kCFAllocatorDefault, mTexW, mTexH, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, buffer, mFrameW * 3, NULL, 0, NULL, &pixelBuffer);
CVReturn err;
// Y-plane
glActiveTexture(GL_TEXTURE0);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
videoTextureCache,
pixelBuffer,
NULL,
GL_TEXTURE_2D,
GL_RED_EXT,
mTexW,
mTexH,
GL_RED_EXT,
GL_UNSIGNED_BYTE,
0,
&_lumaTexture);
if (err)
{
NSLog(#"Error at CVOpenGLESTextureCacheCreateTextureFromImage %d", err);
}
glBindTexture(CVOpenGLESTextureGetTarget(_lumaTexture), CVOpenGLESTextureGetName(_lumaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
// UV-plane
glActiveTexture(GL_TEXTURE1);
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
videoTextureCache,
pixelBuffer,
NULL,
GL_TEXTURE_2D,
GL_RG_EXT,
mTexW/2,
mTexH/2,
GL_RG_EXT,
GL_UNSIGNED_BYTE,
1,
&_chromaTexture);
if (err)
{
NSLog(#"Error at CVOpenGLESTextureCacheCreateTextureFromImage %d", err);
}
glBindTexture(CVOpenGLESTextureGetTarget(_chromaTexture), CVOpenGLESTextureGetName(_chromaTexture));
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
// Set the view port to the entire view
glViewport(0, 0, backingWidth, backingHeight);
static const GLfloat squareVertices[] = {
1.0f, 1.0f,
-1.0f, 1.0f,
1.0f, -1.0f,
-1.0f, -1.0f,
};
GLfloat textureVertices[] = {
1, 1,
1, 0,
0, 1,
0, 0,
};
// Draw the texture on the screen with OpenGL ES 2
[self renderWithSquareVertices:squareVertices textureVertices:textureVertices];
// Flush the CVOpenGLESTexture cache and release the texture
CVOpenGLESTextureCacheFlush(videoTextureCache, 0);
CVPixelBufferRelease(pixelBuffer);
[moviePlayerDelegate bufferDone];
RenderWithSquareVertices method
- (void)renderWithSquareVertices:(const GLfloat*)squareVertices textureVertices:(const GLfloat*)textureVertices
{
// Use shader program.
glUseProgram(shader.program);
// Update attribute values.
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
// Present
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER];
}
My fragment shader:
uniform sampler2D SamplerY;
uniform sampler2D SamplerUV;
varying highp vec2 _texcoord;
void main()
{
mediump vec3 yuv;
lowp vec3 rgb;
yuv.x = texture2D(SamplerY, _texcoord).r;
yuv.yz = texture2D(SamplerUV, _texcoord).rg - vec2(0.5, 0.5);
// BT.601, which is the standard for SDTV is provided as a reference
/* rgb = mat3( 1, 1, 1,
0, -.34413, 1.772,
1.402, -.71414, 0) * yuv;*/
// Using BT.709 which is the standard for HDTV
rgb = mat3( 1, 1, 1,
0, -.18732, 1.8556,
1.57481, -.46813, 0) * yuv;
gl_FragColor = vec4(rgb, 1);
}
Very thanks,
I imagine the issue is that YUV420 (or I420) is a tri-planar image format. I420 is an 8 bit Y plane followed by 8 bit 2x2 subsampled U and V planes. The code from GLCameraRipple is expecting NV12 format: 8-bit Y plane followed by an interleaved U/V plane with 2x2 subsampling. Given this I expect you will need three textures. luma_tex, u_chroma_tex, v_chroma_tex.
Also note that GLCameraRipple may also be expecting 'video range'. In other words the values for the planar format are luma=[16,235] chroma=[16,240].

Resources