How to draw a texture as a 2D background in OpenGL ES 2.0? - opengl-es

I'm just getting started with OpenGL ES 2.0, what I'd like to do is create some simple 2D output. Given a resolution of 480x800, how can I draw a background texture?
[My development environment is Java / Android, so examples directly relating to that would be best, but other languages would be fine.]

Even though you're on Android, I created an iPhone sample application that does this for frames of video coming in. You can download the code for this sample from here. I have a writeup about this application, which does color-based object tracking using live video, that you can read here.
In this application, I draw two triangles to generate a rectangle, then texture that using the following coordinates:
static const GLfloat squareVertices[] = {
-1.0f, -1.0f,
1.0f, -1.0f,
-1.0f, 1.0f,
1.0f, 1.0f,
};
static const GLfloat textureVertices[] = {
1.0f, 1.0f,
1.0f, 0.0f,
0.0f, 1.0f,
0.0f, 0.0f,
};
To pass through the video frame as a texture, I use a simple program with the following vertex shader:
attribute vec4 position;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main()
{
gl_Position = position;
textureCoordinate = inputTextureCoordinate.xy;
}
and the following fragment shader:
varying highp vec2 textureCoordinate;
uniform sampler2D videoFrame;
void main()
{
gl_FragColor = texture2D(videoFrame, textureCoordinate);
}
Drawing is a simple matter of using the right program:
glUseProgram(directDisplayProgram);
setting the texture uniform:
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, videoFrameTexture);
glUniform1i(uniforms[UNIFORM_VIDEOFRAME], 0);
setting the attributes:
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
glVertexAttribPointer(ATTRIB_TEXTUREPOSITON, 2, GL_FLOAT, 0, 0, textureVertices);
glEnableVertexAttribArray(ATTRIB_TEXTUREPOSITON);
and then drawing the triangles:
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

You don't really draw a background, instead you draw a rectangle (or, even more correctly: two triangles forming a rectangle) and set a texture to that. This isn't different at all from drawing any other object on screen.
There are plenty of places showing how this is done, maybe there's even an android example project showing this.
The tricky part is getting something to display in front of or behind something else. For this to work, you need to set up a depth buffer and enable depth testing (glEnable(GL_DEPTH_TEST)). And your vertices need to have a Z coordinate (and tell glDrawElements that your vertices are made up of three values, not two).
If you don't do that, objects will be rendered in the order their glDrawElements() functions are called (meaning whichever you draw last will end up obscuring the rest).
My advice is to not have a background image or do anything fancy until you get the hang of it. OpenGL ES 2.0 has kind of a steep learning curve, and tutorials on ES 1.x don't really help with getting 3D to work because they can use helper functions like gluPerspective, which 2.0 just doesn't have. Start with creating a triangle on a background of nothing. Next, make it a square. Then, if you want to go fancy already, add a texture. Play with positions. See what happens when you change the Z value of your vertices. (Hint: Not a lot, if you don't have depth testing enabled. And even then, if you don't have perspective projection, objects won't get smaller the farther they are away, so it will still seem as if nothing happened)
After a few days, it stops being so damn frustrating, and you finally "get it", mostly.

Related

Does the Windows Composition API support 2.5D projected rotation?

I have started to use the Windows Composition API in UWP applications to animate elements of the UI.
Visual elements expose RotationAngleInDegrees and RotationAngle properties as well as a RotationAxis property.
When I animate a rectangular object's RotationAngleInDegrees value around the Y axis, the rectangle rotates as I would expect but in a 2D application window, it does not appear to be displaying with a 2.5D projection.
Is there a way to get the 2.5D projection effect on rotations with the composition api?
It depends to the effect that you want to have. There is a fluent design app sample on GitHub and here is the link. You will be able to download the demo from the store. And you can get some idea from depth samples. For example, flip to reveal shows a way to rotate a image card and you can find source code from here. For more details please check the sample and the demo.
In general, the animation is to rotate based on X axis:
rectanglevisual.RotationAxis = new System.Numerics.Vector3(1f, 0f, 0f);
And then use rotate animation to rotate based on RotationAngleInDegrees.
It is also possible for you to do this directly on XAML platform by using PlaneProjection from image control.
As the sample that #BarryWang pointed me to demonstrates it is necessary to apply a TransformMatrix to the page (or a parenting container) before executing the animation to get the 2.5D effect with rotation or other spatial transformation animations with the composition api.
private void UpdatePerspective()
{
Visual visual = ElementCompositionPreview.GetElementVisual(MainPanel);
// Get the size of the area we are enabling perspective for
Vector2 sizeList = new Vector2((float)MainPanel.ActualWidth, (float)MainPanel.ActualHeight);
// Setup the perspective transform.
Matrix4x4 perspective = new Matrix4x4(
1.0f, 0.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f, 0.0f,
0.0f, 0.0f, 1.0f, -1.0f / sizeList.X,
0.0f, 0.0f, 0.0f, 1.0f);
// Set the parent transform to apply perspective to all children
visual.TransformMatrix =
Matrix4x4.CreateTranslation(-sizeList.X / 2, -sizeList.Y / 2, 0f) * // Translate to origin
perspective * // Apply perspective at origin
Matrix4x4.CreateTranslation(sizeList.X / 2, sizeList.Y / 2, 0f); // Translate back to original position
}

Perspective correct texturing of trapezoid in OpenGL ES 2.0

I have drawn a textured trapezoid, however the result does not appear as I had intended.
Instead of appearing as a single unbroken quadrilateral, a discontinuity occurs at the diagonal line where its two comprising triangles meet.
This illustration demonstrates the issue:
(Note: the last image is not intended to be a 100% faithful representation, but it should get the point across.)
The trapezoid is being drawn using GL_TRIANGLE_STRIP in OpenGL ES 2.0 (on an iPhone). It's being drawn completely facing the screen, and is not being tilted (i.e. that's not a 3D sketch you're seeing!)
I have come to understand that I need to perform "perspective correction," presumably in my vertex and/or fragment shaders, but I am unclear how to do this.
My code includes some simple Model/View/Projection matrix math, but none of it currently influences my texture coordinate values. Update: The previous statement is incorrect, according to comment by user infact.
Furthermore, I have found this tidbit in the ES 2.0 spec, but do not understand what it means:
The PERSPECTIVE CORRECTION HINT is not supported because OpenGL
ES 2.0 requires that all attributes be perspectively interpolated.
How can I make the texture draw correctly?
Edit: Added code below:
// Vertex shader
attribute vec4 position;
attribute vec2 textureCoordinate;
varying vec2 texCoord;
uniform mat4 modelViewProjectionMatrix;
void main()
{
gl_Position = modelViewProjectionMatrix * position;
texCoord = textureCoordinate;
}
// Fragment shader
uniform sampler2D texture;
varying mediump vec2 texCoord;
void main()
{
gl_FragColor = texture2D(texture, texCoord);
}
// Update and Drawing code (uses GLKit helpers from iOS)
- (void)update
{
float fov = GLKMathDegreesToRadians(65.0f);
float aspect = fabsf(self.view.bounds.size.width / self.view.bounds.size.height);
projectionMatrix = GLKMatrix4MakePerspective(fov, aspect, 0.1f, 50.0f);
viewMatrix = GLKMatrix4MakeTranslation(0.0f, 0.0f, -4.0f); // zoom out
}
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect
{
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glUseProgram(shaders[SHADER_DEFAULT]);
GLKMatrix4 modelMatrix = GLKMatrix4MakeScale(0.795, 0.795, 0.795); // arbitrary scale
GLKMatrix4 modelViewMatrix = GLKMatrix4Multiply(viewMatrix, modelMatrix);
GLKMatrix4 modelViewProjectionMatrix = GLKMatrix4Multiply(projectionMatrix, modelViewMatrix);
glUniformMatrix4fv(uniforms[UNIFORM_MODELVIEWPROJECTION_MATRIX], 1, GL_FALSE, modelViewProjectionMatrix.m);
glBindTexture(GL_TEXTURE_2D, textures[TEXTURE_WALLS]);
glUniform1i(uniforms[UNIFORM_TEXTURE], 0);
glVertexAttribPointer(ATTRIB_VERTEX, 3, GL_FLOAT, GL_FALSE, 0, wall.vertexArray);
glVertexAttribPointer(ATTRIB_TEXTURE_COORDINATE, 2, GL_FLOAT, GL_FALSE, 0, wall.texCoords);
glDrawArrays(GL_TRIANGLE_STRIP, 0, wall.vertexCount);
}
(I'm taking a bit of a punt here, because your picture does not show exactly what I would expect from texturing a trapezoid, so perhaps something else is happening in your case - but the general problem is well known)
Textures will not (by default) interpolate correctly across a trapezoid. When the shape is triangulated for drawing, one of the diagonals will be chosen as an edge, and while that edge is straight through the middle of the texture, it is not through the middle of the trapezoid (picture the shape divided along a diagonal - the two triangles are very much not equal).
You need to provide more than a 2D texture coordinate to make this work - you need to provide a 3D (or rather, projective) texture coordinate, and perform the perspective divide in the fragment shader, post-interpolation (or else use a texture lookup function which will do the same).
The following shows how to provide texture coordinates for a trapezoid using old-school GL functions (which are a little easier to read for demonstration purposes). The commented-out lines are the 2d texture coordinates, which I have replaced with projective coordinates to get the correct interpolation.
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0,640,0,480,1,1000);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
const float trap_wide = 600;
const float trap_narrow = 300;
const float mid = 320;
glBegin(GL_TRIANGLE_STRIP);
glColor3f(1,1,1);
// glTexCoord4f(0,0,0,1);
glTexCoord4f(0,0,0,trap_wide);
glVertex3f(mid - trap_wide/2,10,-10);
// glTexCoord4f(1,0,0,1);
glTexCoord4f(trap_narrow,0,0,trap_narrow);
glVertex3f(mid - trap_narrow/2,470,-10);
// glTexCoord4f(0,1,0,1);
glTexCoord4f(0,trap_wide,0,trap_wide);
glVertex3f(mid + trap_wide/2,10,-10);
// glTexCoord4f(1,1,0,1);
glTexCoord4f(trap_narrow,trap_narrow,0,trap_narrow);
glVertex3f(mid + trap_narrow/2,470,-10);
glEnd();
The third coordinate is unused here as we're just using a 2D texture. The fourth coordinate will divide the other two after interpolation, providing the projection. Obviously if you divide it through at the vertices, you'll see you get the original texture coordinates.
Here's what the two renderings look like:
If your trapezoid is actually the result of transforming a quad, it might be easier/better to just draw that quad using GL, rather than transforming it in software and feeding 2D shapes to GL...
What you are trying here is Skewed texture. A sample fragment shader is as follows :
precision mediump float;
varying vec4 vtexCoords;
uniform sampler2D sampler;
void main()
{
gl_FragColor = texture2DProj(sampler,vtexCoords);
}
2 things which should look different are :
1) We are using varying vec4 vtexCoords; . Texture co-ordinates are 4 dimensional.
2) texture2DProj() is used instead of texture2D()
Based on length of small and large side of your trapezium you will assign texture co-ordinates. Following URL might help :
http://www.xyzw.us/~cass/qcoord/
The accepted answer gives the correct solution and explanation but for those looking for a bit more help on the OpenGL (ES) 2.0 pipeline...
const GLfloat L = 2.0;
const GLfloat Z = -2.0;
const GLfloat W0 = 0.01;
const GLfloat W1 = 0.10;
/** Trapezoid shape as two triangles. */
static const GLKVector3 VERTEX_DATA[] = {
{{-W0, 0, Z}},
{{+W0, 0, Z}},
{{-W1, L, Z}},
{{+W0, 0, Z}},
{{+W1, L, Z}},
{{-W1, L, Z}},
};
/** Add a 3rd coord to your texture data. This is the perspective divisor needed in frag shader */
static const GLKVector3 TEXTURE_DATA[] = {
{{0, 0, 0}},
{{W0, 0, W0}},
{{0, W1, W1}},
{{W0, 0, W0}},
{{W1, W1, W1}},
{{0, W1, W1}},
};
////////////////////////////////////////////////////////////////////////////////////
// frag.glsl
varying vec3 v_texPos;
uniform sampler2D u_texture;
void main(void)
{
// Divide the 2D texture coords by the third projection divisor
gl_FragColor = texture2D(u_texture, v_texPos.st / v_texPos.p);
}
Alternatively, in the shader, as per #maverick9888's answer, You can use texture2Dproj though for iOS / OpenGLES2 it still only supports a vec3 input...
void main(void)
{
gl_FragColor = texture2DProj(u_texture, v_texPos);
}
I haven't really benchmarked it properly but for my very simple case (a 1d texture really) the division version seems a bit snappier.

My triangle doesn't render when I use OpenGL Core Profile 3.2

I have a Cocoa (OSX) project that is currently very simple, I'm just trying to grasp the general concepts behind using OpenGL. I was able to get a triangle to display in my view, but when I went to write my vertex shaders and fragment shaders, I realized I was running the legacy OpenGL core profile. So I switched to the OpenGL 3.2 profile by setting the properties in the pixel format of the view in question before generating the context, but now the triangle doesn't render, even without my vertex or fragment shaders.
I have a controller class for the view that's instantiated in the nib. On -awakeFromNib it sets up the pixel format and the context:
NSOpenGLPixelFormatAttribute attr[] =
{
NSOpenGLPFAOpenGLProfile, NSOpenGLProfileVersion3_2Core,
0
};
NSOpenGLPixelFormat *glPixForm = [[NSOpenGLPixelFormat alloc] initWithAttributes:attr];
[self.mainView setPixelFormat:glPixForm];
self.glContext = [self.mainView openGLContext];
Then I generate the VAO:
glGenVertexArrays(1, &vertexArrayID);
glBindVertexArray(vertexArrayID);
Then the VBO:
glGenBuffers(1, &buffer);
glBindBuffer(GL_ARRAY_BUFFER, buffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(g_vertex_buffer_data), g_vertex_buffer_data, GL_STATIC_DRAW);
g_vertex_buffer_data, the actual data for that buffer is defined as follows:
static const GLfloat g_vertex_buffer_data[] = {
-1.0f, -1.0f, 0.0f,
1.0f, -1.0f, 0.0f,
0.0f, 1.0f, 0.0f,
};
Here's the code for actually drawing:
[_glContext setView:self.mainView];
[_glContext makeCurrentContext];
glViewport(0, 0, [self.mainView frame].size.width, [self.mainView frame].size.height);
glEnableVertexAttribArray(0);
glBindBuffer(GL_ARRAY_BUFFER, self.vertexBuffer);
glVertexAttribPointer(
0,
3,
GL_FLOAT,
GL_FALSE,
0,
(void*)0
);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glDrawArrays(GL_TRIANGLES, 0, 3); // Starting from vertex 0; 3 vertices total -> 1 triangle
glDisableVertexAttribArray(0);
glFlush();
This code draws the triangle fine if I comment out the NSOpenGLPFAOpenGLProfile, NSOpenGLProfileVersion3_2Core, in the NSOpenGLPixelFormatAttribute array, but as soon as I enable OpenGL Core Profile 3.2, it just displays black. Can anyone tell me what I'm doing wrong here?
EDIT: This issue still happens whether I turn my vertex and fragment shaders on or not, but here are my shaders in case it is helpful:
Vertex shader:
#version 150
in vec3 position;
void main() {
gl_Position.xyz = position;
}
Fragment shader:
#version 150
out vec3 color;
void main() {
color = vec3(1,0,0);
}
And right before linking the program, I make this call to bind the attribute location:
glBindAttribLocation(programID, 0, "position");
EDIT 2:
I don't know if this helps at all, but I just stepped through my program, running glGetError() and it looks like everything is fine until I actually call glDrawArrays(), then it returns GL_INVALID_OPERATION. I'm trying to figure out why this could be occurring, but still having no luck.
I figured this out, and it's sadly a very stupid mistake on my part.
I think the issue is that you need a vertex shader and a fragment shader when using 3.2 core profile, you can't just render without them. The reason it wasn't working with my shaders was...wait for it...after linking my shader program, I forgot to store the programID in the ivar in my class, so later when I call glUseProgram() I'm just calling it with a zero parameter.
I guess one of the main sources of confusion was the fact that I expected the 3.2 core profile to work without any vertex or fragment shaders.

glMaterialfv not working for me

This is OpenGL on iPhone 4.
Im drawing scene using light and materials. Here is snippet of my code:
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustumf(-1, 1, -1, 1, -1, 1);
CGFloat ambientLight[] = { 0.5f, 0.5f, 0.5f, 1.0f };
CGFloat diffuseLight[] = { 1.0f, 1.0f, 1.0f, 1.0f };
CGFloat direction[] = { 0.0f, 0.0f, -20.0f, 0 };
glEnable(GL_LIGHT0);
glLightfv(GL_LIGHT0, GL_AMBIENT, ambientLight);
glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuseLight);
glLightfv(GL_LIGHT0, GL_POSITION, direction);
glShadeModel(GL_FLAT);
glEnable(GL_LIGHTING);
glDisable(GL_COLOR_MATERIAL);
float blankColor[4] = {0,0,0,1};
float whiteColor[4] = {1,1,1,1};
float blueColor[4] = {0,0,1,1};
glMaterialfv(GL_FRONT, GL_DIFFUSE, blueColor);
glEnable(GL_CULL_FACE);
glVertexPointer(3, GL_FLOAT, 0, verts.pdata);
glEnableClientState(GL_VERTEX_ARRAY);
glNormalPointer(GL_FLOAT, 0, normals.pdata);
glEnableClientState(GL_NORMAL_ARRAY);
glDrawArrays (GL_TRIANGLES, 0, verts.size/3);
Problem is that instead of seeing BLUE diffuse color I see it white. It fades out if I rotate model's side but I can't understand why its not using my blue color.
BTW if I change glMaterialfv(GL_FRONT, GL_DIFFUSE, blueColor) to glMaterialfv(GL_FRONT_AND_BACK, GL_DIFFUSE, blueColor) then I do see blue color. If I do it glMaterialfv(GL_FRONT, GL_DIFFUSE, blueColor); and then glMaterialfv(GL_BACK, GL_DIFFUSE, blueColor); I see white color again. So it looks like GL_FRONT_AND_BACK shows it but rest of combinations show white. Anyone can explain it to me?
This is because of clockwise
10.090 How does face culling work? Why doesn't it use the surface normal?
OpenGL face culling calculates the signed area of the filled primitive in window coordinate space. The signed area is positive when the window coordinates are in a counter-clockwise order and negative when clockwise. An app can use glFrontFace() to specify the ordering, counter-clockwise or clockwise, to be interpreted as a front-facing or back-facing primitive. An application can specify culling either front or back faces by calling glCullFace(). Finally, face culling must be enabled with a call to glEnable(GL_CULL_FACE); .
OpenGL uses your primitive's window space projection to determine face culling for two reasons. To create interesting lighting effects, it's often desirable to specify normals that aren't orthogonal to the surface being approximated. If these normals were used for face culling, it might cause some primitives to be culled erroneously. Also, a dot-product culling scheme could require a matrix inversion, which isn't always possible (i.e., in the case where the matrix is singular), whereas the signed area in DC space is always defined.
However, some OpenGL implementations support the GL_EXT_ cull_vertex extension. If this extension is present, an application may specify a homogeneous eye position in object space. Vertices are flagged as culled, based on the dot product of the current normal with a vector from the vertex to the eye. If all vertices of a primitive are culled, the primitive isn't rendered. In many circumstances, using this extension
from here
Also you can read here

Why am I not able to attach this texture uniform to my GLSL fragment shader?

In my Mac application, I define a rectangular texture based on YUV 4:2:2 data from an attached camera. Using standard vertex and texture coordinates, I can draw this to a rectangular area on the screen without any problems.
However, I would like to use a GLSL fragment shader to process these image frames on the GPU, and am having trouble passing in the rectangular video texture as a uniform to the fragment shader. When I attempt to do so, the texture simply reads as black.
The shader program compiles, links, and passes validation. I am receiving the proper address for the uniform from the shader program. Other uniforms, such as floating point values, pass in correctly and the fragment shader responds to changes in these values. The fragment shader receives the correct texture coordinates. I've also sprinkled my code liberally with glGetError() and seen no errors anywhere.
The vertex shader is as follows:
void main()
{
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
gl_FrontColor = gl_Color;
gl_TexCoord[0] = gl_TextureMatrix[0] * gl_MultiTexCoord0;
}
and the fragment shader is as follows:
uniform sampler2D videoFrame;
void main()
{
gl_FragColor = texture2D(videoFrame, gl_TexCoord[0].st);
}
This should simply display the texture on my rectangular geometry.
The relevant drawing code is as follows:
static const GLfloat squareVertices[] = {
-1.0f, -1.0f,
1.0f, -1.0f,
-1.0f, 1.0f,
1.0f, 1.0f,
};
const GLfloat textureVertices[] = {
0.0, videoImageSize.height,
videoImageSize.width, videoImageSize.height,
0.0, 0.0,
videoImageSize.width, 0.0
};
CGLSetCurrentContext(glContext);
if(!readyToDraw)
{
[self initGL];
readyToDraw = YES;
}
glViewport(0, 0, (GLfloat)self.bounds.size.width, (GLfloat)self.bounds.size.height);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glClearColor(0.5f, 0.5f, 0.5f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
glEnable(GL_TEXTURE_2D);
glGenTextures(1, &textureName);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_RECTANGLE_EXT, textureName);
glTexImage2D(GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA, videoImageSize.width, videoImageSize.height, 0, GL_YCBCR_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE, videoTexture);
glUseProgram(filterProgram);
glUniform1i(uniforms[UNIFORM_VIDEOFRAME], 0);
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glVertexPointer(2, GL_FLOAT, 0, squareVertices);
glTexCoordPointer(2, GL_FLOAT, 0, textureVertices);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
[super drawInCGLContext:glContext pixelFormat:pixelFormat forLayerTime:interval displayTime:timeStamp];
glDeleteTextures(1, &textureName);
This code resides within a CAOpenGLLayer, where the superclass's -drawInCGLContext:pixelFormat:forLayerTime: displayTime: simply runs glFlush().
The uniform address is read using code like the following:
uniforms[UNIFORM_VIDEOFRAME] = glGetUniformLocation(filterProgram, "videoFrame");
As I said, if I comment out the glUseProgram() and glUniform1i() lines, this textured rectangle draws properly. Leaving them in leads to a black rectangle being drawn.
What could be preventing my texture uniform from being passed into my fragment shader?
Not sure about the GLSL version you're using, but from 1.40 upwards there's the type sampler2DRect specifically for accessing non-power-of-two textures. Might be what you're looking for, however I don't know how rectangular textures were handled before glsl 1.40.

Resources