How do I display a non-power-of-2-texture as sprite in OpenGL (ES) with no stretching? - opengl-es

I want to draw an arbitrary size sprite as a png, say something TOTALLY CRAZY like 56 wide x 30 high. Not a power of 2 in either dimension. Also I may want to draw another different sprite that's 72 wide x 33 high. Indicating this because no 'tricks' are acceptable here, I need to handle general case.
So I have this png (with transparency) and I want to draw it as a sprite with absolutely no stretching, interpolation, etc. I want it to map 1:1 with pixels on the screen. Pretend these sprites are pixel art (they may or may not be but they are drawn for the exact resolution).
I understand sprite drawing - 2 triangles as a quad, rendering my texture to that via texture coordinate mapping - however I only understand it for powers of 2. And I also do not understand how to size my 'quad' and set up my GL matricies such that I can make the sprite be the exact same size in pixels as my sprite
I'm using OpenGL ES so using the new extensions to use non-power-of-2 textures is not acceptable.
I'm only drawing 3-10 sprites at a time so I'm open to suggestions about efficiency but I'm most interested in 'accurate' looking results.

To get a pixel-perfect coordinate system, you could set up your matrices as follows:
glViewport(0, window_width, 0, window_height);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(0, window_width, 0, window_height, 0, 1);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
This ensures a 1:1 mapping of OpenGL coordinates to pixels, and puts your origin in the bottom left corner.
As to your texture, if you don't want to rely on the non-power-of-two extension, you must pad it to be a power of two. Even though the OpenGL ES spec doesn't say it has to be square as well (as far as I know), I've seen strange things happen on Android with texture sizes like 512 x 128, so it might be best to stick to square power-of-two textures.
The thing to note is that texture coordinates in OpenGL are always between 0 and 1, no matter what the pixel size of your texture is. So if your sprite is 48 pixels wide, and you padded it to a texture of 64 x 64, then it will span the x coordinates from 0 to 0.75:
1 +--------+
| |
| |
+-----+ |
|\_O_/| |
| O | |
| / \ | |
0 +-----+--+
0 0.75 1 <- texture coordinates
0 48 64 <- pixels
So you have to set up these texture coordinates for the corners of your quad. Because we have a pixel-perfect projection, the quad must also be exactly 48 pixels wide.
In conclusion, to draw your sprite at position x, y (in pixels from the bottom left), do something like this:
float texcoord_x = (float)sprite_width / texture_width;
float texcoord_y = (float)sprite_height / texture_height;
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture_id);
glBegin(GL_QUADS);
// bottom left
glTexCoord2f(0, 0);
glVertex2i(x, y);
// bottom right
glTexCoord2f(texcoord_x, 0);
glVertex2i(x + sprite_width, y);
// top right
glTexCoord2f(texcoord_x, texcoord_y);
glVertex2i(x + sprite_width, y + sprite_height);
// top left
glTexCoord2f(0, texcoord_y);
glVertex2i(x, y + sprite_height);
glEnd();
I know OpenGL ES doesn't have glVertex2i and so on, so you'll have to put these coordinates into a buffer. And it doesn't allow quads, so you'll have to split it up into triangles. But this is the basic idea.

Related

How to convert a screen coordinate into a translation for a projection matrix?

(More info at end)----->
I am trying to render a small picture-in-picture display over my scene. The PiP is just a smaller texture, but it is intended to reveal secret objects in the scene when it is placed over them.
To do this, I want to render my scene, then render the SAME scene on the smaller texture, but with the exact same positioning as the main scene. The intended result would be something like this:
My problem is... I cannot get the scene on the smaller texture to match up 1:1. I keep trying various kludges, but ultimately I suspect that I need to do something to the projection matrix to pan it over to the location of the frame. I can get it to zoom correctly...just can't get it to pan.
Can anyone suggest what I need to do to my projection matrix to render my scene 1:1 (but panned by x,y) onto a smaller texture?
The data I have:
Resolution of the full-screen framebuffer
Resolution of the smaller texture
XY coordinate where I want to draw the smaller texture as an overlay sprite
The world/view/projection matrices from the original full-screen scene
The viewport from the original full-screen scene
(Edit)
Here is the function I use to produce the 3D camera:
void Make3DCamera(Vector theCameraPos, Vector theLookAt, Vector theUpVector, float theFOV, Point theRez, Matrix& theViewMatrix,Matrix& theProjectionMatrix)
{
Matrix aCombinedViewMatrix;
Matrix aViewMatrix;
aCombinedViewMatrix.Scale(1,1,-1);
theCameraPos.mZ*=-1;
theLookAt.mZ*=-1;
theUpVector.mZ*=-1;
aCombinedViewMatrix.Translate(-theCameraPos);
Vector aLookAtVector=theLookAt-theCameraPos;
Vector aSideVector=theUpVector.Cross(aLookAtVector);
theUpVector=aLookAtVector.Cross(aSideVector);
aLookAtVector.Normalize();
aSideVector.Normalize();
theUpVector.Normalize();
aViewMatrix.mData.m[0][0] = -aSideVector.mX;
aViewMatrix.mData.m[1][0] = -aSideVector.mY;
aViewMatrix.mData.m[2][0] = -aSideVector.mZ;
aViewMatrix.mData.m[3][0] = 0;
aViewMatrix.mData.m[0][1] = -theUpVector.mX;
aViewMatrix.mData.m[1][1] = -theUpVector.mY;
aViewMatrix.mData.m[2][1] = -theUpVector.mZ;
aViewMatrix.mData.m[3][1] = 0;
aViewMatrix.mData.m[0][2] = aLookAtVector.mX;
aViewMatrix.mData.m[1][2] = aLookAtVector.mY;
aViewMatrix.mData.m[2][2] = aLookAtVector.mZ;
aViewMatrix.mData.m[3][2] = 0;
aViewMatrix.mData.m[0][3] = 0;
aViewMatrix.mData.m[1][3] = 0;
aViewMatrix.mData.m[2][3] = 0;
aViewMatrix.mData.m[3][3] = 1;
if (gG.mRenderToSprite) aViewMatrix.Scale(1,-1,1);
aCombinedViewMatrix*=aViewMatrix;
// Projection Matrix
float aAspect = (float) theRez.mX / (float) theRez.mY;
float aNear = gG.mZRange.mData1;
float aFar = gG.mZRange.mData2;
float aWidth = gMath.Cos(theFOV / 2.0f);
float aHeight = gMath.Cos(theFOV / 2.0f);
if (aAspect > 1.0) aWidth /= aAspect;
else aHeight *= aAspect;
float s = gMath.Sin(theFOV / 2.0f);
float d = 1.0f - aNear / aFar;
Matrix aPerspectiveMatrix;
aPerspectiveMatrix.mData.m[0][0] = aWidth;
aPerspectiveMatrix.mData.m[1][0] = 0;
aPerspectiveMatrix.mData.m[2][0] = gG.m3DOffset.mX/theRez.mX/2;
aPerspectiveMatrix.mData.m[3][0] = 0;
aPerspectiveMatrix.mData.m[0][1] = 0;
aPerspectiveMatrix.mData.m[1][1] = aHeight;
aPerspectiveMatrix.mData.m[2][1] = gG.m3DOffset.mY/theRez.mY/2;
aPerspectiveMatrix.mData.m[3][1] = 0;
aPerspectiveMatrix.mData.m[0][2] = 0;
aPerspectiveMatrix.mData.m[1][2] = 0;
aPerspectiveMatrix.mData.m[2][2] = s / d;
aPerspectiveMatrix.mData.m[3][2] = -(s * aNear / d);
aPerspectiveMatrix.mData.m[0][3] = 0;
aPerspectiveMatrix.mData.m[1][3] = 0;
aPerspectiveMatrix.mData.m[2][3] = s;
aPerspectiveMatrix.mData.m[3][3] = 0;
theViewMatrix=aCombinedViewMatrix;
theProjectionMatrix=aPerspectiveMatrix;
}
Edit to add more information:
Just playing and tweaking numbers, I have come to a "close" result. However the "close" result requires a multiplication by some kludge numbers, that I don't understand.
Here's what I'm doing to to perspective matrix to produce my close result:
//Before calling Make3DCamera, adjusting FOV:
aFOV*=smallerTexture.HeightF()/normalRenderSize.HeightF(); // Zoom it
aFOV*=1.02f // <- WTH is this?
//Then, to pan the camera over to the x/y position I want, I do:
Matrix aPM=GetCurrentProjectionMatrix();
float aX=(screenX-normalRenderSize.WidthF()/2.0f)/2.0f;
float aY=(screenY-normalRenderSize.HeightF()/2.0f)/2.0f;
aX*=1.07f; // <- WTH is this?
aY*=1.07f; // <- WTH is this?
aPM.mData.m[2][0]=-aX/normalRenderSize.HeightF();
aPM.mData.m[2][1]=-aY/normalRenderSize.HeightF();
SetCurrentProjectionMatrix(aPM);
When I do this, my new picture is VERY close... but not exactly perfect-- the small render tends to drift away from "center" the further the "magic window" is from the center. Without the kludge number, the drift away from center with the magic window is very pronounced.
The kludge numbers 1.02f for zoom and 1.07 for pan reduce the inaccuracies and drift to a fraction of a pixel, but those numbers must be a ratio from somewhere, right? They work at ANY RESOLUTION, though-- so I have have a 1280x800 screen and a 256,256 magic window texture... if I change the screen to 1024x768, it all still works.
Where the heck are these numbers coming from?
If you don't care about sub-optimal performance (i.e., drawing the whole scene twice) and if you don't need the smaller scene in a texture, an easy way to obtain the overlay with pixel perfect precision is:
Set up main scene (model/view/projection matrices, etc.) and draw it as you are now.
Use glScissor to set the rectangle for the overlay. glScissor takes the screen-space x, y, width, and height and discards anything outside that rectangle. It looks like you have those four data items already, so you should be good to go.
Call glEnable(GL_SCISSOR_TEST) to actually turn on the test.
Set the shader variables (if you're using shaders) for drawing the greyscale scene/hidden objects/etc. You still use the same view and projection matrices that you used for the main scene.
Draw the greyscale scene/hidden objects/etc.
Call glDisable(GL_SCISSOR_TEST) so you won't be scissoring at the start of the next frame.
Draw the red overlay border, if desired.
Now, if you actually need the overlay in its own texture for some reason, this probably won't be adequate...it could be made to work either with framebuffer objects and/or pixel readback, but this would be less efficient.
Most people completely overcomplicate such issues. There is absolutely no magic to applying transformations after applying the projection matrix.
If you have a projection matrix P (and I'm assuming default OpenGL conventions here where P is constructed in a way that the vector is post-multiplied to the matrix, so for an eye space vector v_eye, we get v_clip = P * v_eye), you can simply pre-multiply some other translate and scale transforms to cut out any region of interest.
Assume you have a viewport of size w_view * h_view pixels, and you want to find a projection matrix which renders only a tile w_tile * h_tile pixels , beginning at pixel location (x_tile, y_tile) (again, assuming default GL conventions here, window space origin is bottom left, so y_tile is measured from the bottom). Also note that the _tile coordinates are to be interpreted relative to the viewport, in the typical case, that would start at (0,0) and have the size of your full framebuffer, but this is by no means required nor assumed here.
Since after applying the projection matrix we are in clip space, we need to transform our coordinates from window space pixels to clip space. Note that clip space is a 4D homogeneous space, but we can use any w value we like (except 0) to represent any point (as a point in the 3D space we care about forms a line in the 4D space we work in), so let's just use w=1 for simplicity's sake.
The view volume in clip space is denoted by the [-w,w] range, so in the w=1 hyperplane, it is [-1,1]. Converting our tile into this space yields:
x_clip = 2 * (x_tile / w_view) -1
y_clip = 2 * (y_tile / h_view) -1
w_clip = 2 * (w_tile / w_view) -1
h_clip = 2 * (h_tile / h_view) -1
We now just need to translate the objects such that the center of the tile is moved to the center of the view volume, which by definition is the origin, and scale the w_clip * h_clip sized region to the full [-1,1] extent in each dimension.
That means:
T = translate(-(x_clip + 0.5*w_clip), -(y_clip + 0.5 *h_clip), 0)
S = scale(2.0/w_clip, 2.0/h_clip, 1.0)
We can now create the modified projection matrix P' as P' = S * T * P, and that's all there is. Rendering with P' instead of P will render exactly the region of your tile to whatever viewport you are using, so for it to be pixel-exact with respect to your original viewport, you must now render with a viewport which is also w_tile * h_tile pixels big.
Note that there is also another approach: The viewport is not clamped against the framebuffer you're rendering to. It is actually valid to provide negative values for x and y. If your framebuffer for rendering your tile into is exactly w_tile * h_tile pixels, you simply could set glViewport(-x_tile, -y_tile, x_tile + w_tile, y_tile + h_tile) and render with the unmodified projection matrix P instead.

OpenGL transparency in texture when render with stencil buffer

The question has been updated thanks to the comments.
Screenshot of how textures overlap
To draw 2 points with brush texture using the stencil buffer to avoid textures transparency overlap, the following code is used:
glEnable(GL_STENCIL_TEST.gluint)
glClear(GL_STENCIL_BUFFER_BIT.gluint | GL_DEPTH_BUFFER_BIT.gluint)
glStencilOp(GL_KEEP.gluint, GL_KEEP.gluint, GL_REPLACE.gluint)
glStencilFunc(GL_ALWAYS.gluint, 1, 1)
glStencilMask(1)
glDrawArrays(GL_POINTS.gluint, 0, 1)
glStencilFunc(GL_NOTEQUAL.gluint, 1, 1)
glStencilMask(1)
glDrawArrays(GL_POINTS.gluint, 1, 1)
glDisable(GL_STENCIL_TEST.gluint)
And stencil buffer works, however, each point fill a full rectangle in the stencil buffer, but a texture image has transparency. So maybe texture used in the wrong way?
The texture is loaded like this
glGenTextures(1, &gl_id)
glBindTexture(GL_TEXTURE_2D.gluint, gl_id)
glTexParameteri(GL_TEXTURE_2D.gluint, GL_TEXTURE_MIN_FILTER.gluint, GL_LINEAR)
glTexImage2D(GL_TEXTURE_2D.gluint, 0, GL_RGBA, gl_width.int32, gl_height.int32, 0, GL_RGBA.gluint, GL_UNSIGNED_BYTE.gluint, gl_data)
Blending set as
glEnable(GL_BLEND.gluint)
glBlendFunc(GL_ONE.gluint, GL_ONE_MINUS_SRC_ALPHA.gluint)
Could you please advice where to look in order to fill 1s in stencil buffer by exactly not transparent area of brush image?
I recommend to discard the transparent parts of the texture in the fragment shader. A fragment can be completely skipped in the fragment shader by the discard keyword.
See Fragment Shader - Special operations.
Use a small threshold and discard a fragment, if the alpha channel of the texture color is below the threshold:
vec4 texture_color = .....;
float threshold = 0.01;
if ( texture_color.a < threshold )
discard;
An other possibility would be to use an Alpha test. This would be only available in OpenGL compatibility profile, but not in core profile or OpenGL ES.
See Khronos OpenGL-Refpages glAlphaFunc:
The alpha test discards fragments depending on the outcome of a comparison between an incoming fragment's alpha value and a constant reference value.
With the following alpha test, the fragments whos alpha channel is below the threshold are discarded:
float threshold = 0.01;
glAlphaFunc(GL_GEQUAL, threshold);
glEnable(GL_ALPHA_TEST)

Geometry Shader Quad Post Processing

Using directx 11, I'm working on a graphics effect system that uses a geometry shader to build quads in world space. These quads then use a fragment shader in which the main texture is the rendered scene texture. Effectively producing post process effects on qorld space quads. The simplest of which is a tint effect.
The vertex shader only passes the data through to the geometry shader.
The geometry shader calculates extra vertices based on a normal. Using cross product, I find the x and z axis and append the tri-stream with 4 new verts in each diagonal direction from the original position (generating a quad from the given position and size).
The pixel shader (tint effect) simply multiplies the scene texture colour with the colour variable set.
The quad generates and displays correctly on screen. However;
The problem that I am facing is the mapping of the uv coordinates fails to align with the image on the back buffer. That is, when using the tint shader with half alpha as the given colour you can see the image displayed on the quad does not overlay the image on the back buffer perfectly, unless the quad facing towards the camera. The closer the quad normal matches the cameras y axis, the more the image is skewed.
I am currently using the formula below to calculate the uv coordinates:
float2 uv = vert0.position.xy / vert0.position.w;
vert0.uv.x = uv.x * 0.5f + 0.5f;
vert0.uv.y = -uv.y * 0.5f + 0.5f;
I have also used the formula below, which resulted (IMO) in the uv's not taking perspective into concideration.
float2 uv = vert0.position.xy / SourceTextureResolution;
vert0.uv.x = uv.x * ASPECT_RATIO + 0.5f;
vert0.uv.y = -uv.y + 0.5f;
Question:
How can I obtain screen space uv coordinates based on a vertex position generated in the geometry shader?
If you would like me to elaborate on any points please ask and i will try my best :)
Thanks in advance.

glDrawPixels() with 0.375 translation

I've noticed some strange behaviour with glDrawPixels() when using a 0.375 translation. This is my GL initialization:
width = 640; height = 480;
glViewport(0, 0, width, height);
glMatrixMode(GL_PROJECTION);
glLoadIdentity( );
glOrtho(0, width, height, 0, 0, 1);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity( );
glTranslatef(0.375, 0.375, 0.0);
Now I want to draw a 640x30 pixel buffer to the very last 30 rows of my GL window. Hence, I do the following:
glRasterPos2i(0, 480);
glDrawPixels(640, 30, GL_RGBA, GL_UNSIGNED_BYTE, pixelbuffer);
Unfortunately, nothing gets drawn using this code. glGetError() also returns 0. The interesting thing is that as soon as I remove the call to glTranslatef(0.375, 0.375, 0.0) everything works fine!
So could somebody explain to me why this 0.375 translation on both axes confuses glDrawPixels()? Is this somehow rounded to 1.0 internally making my call to glDrawPixels() suddenly want to draw beyond the context's boundaries and thus it gets clipped by OpenGL? This is the only explanation I can think of but I don't understand why OpenGL should round a 0.375 translation to 1.0... it should be rounded down to 0.0 instead, shouldn't it?
The point (0,480) actually straddles one of your clipping planes given your projection matrix. Your sub-pixel shift hack pushes the point beyond the breaking point and the raster position is clipped. In GL, glRasterPos (...) will invalidate all following raster operations as long as the initial position is clipped (which in this case, it is).
You could try glRasterPos2i (0, 479). This is altogether more meaningful given the dimensions of your window anyway. You could also drop the whole charade and use glWindowPos2i (...) instead of relying on your projection and modelview matrices to position the raster coordinate in window-space.
I can't answer your question on why glTranslatef stops glDrawPixels from working, but I can tell you that isn't the way to select where to draw. Check the man page for glDrawPixels for a bit more info. It will tell you about glRasterPos and glWindowPos

Texture mapping in LWJGL

This have been driving me crazy for the past couple of days.
I'm animating a spritesheet, and it actually works out fine on my 96px 384px texture with this code:
glBegin(GL_QUADS);
glTexCoord2f((frameCount*24.0f)/imgWidth, (row*24.0f)/imgHeight); glVertex3f(0+x, 0+y, -0.001f*(y+32));
glTexCoord2f((frameCount*24.0f)/imgWidth, ((row+1)*24.0f)/imgHeight); glVertex3f(0+x, 32+y, -0.001f*(y+32));
glTexCoord2f(((frameCount+1)*24.0f)/imgWidth, ((row+1)*24.0f)/imgHeight); glVertex3f(32+x, 32+y, -0.001f*(y+32));
glTexCoord2f(((frameCount+1)*24.0f)/imgWidth, (row*24.0f)/imgHeight); glVertex3f(32+x, 0+y, -0.001f*(y+32));
glEnd();
Problem is though, that when I load in a 32px 32px texture, it looks weird! I suspect that the number 24.0f should be different according to the texture size, but I can't figure out how.
Second question: How does this method affect the performance, are there better ways of doing it?
The texture coordinate for the x-axis (width or u value) should be:
frameCount * (frameWidth / imgWidth)
with frameWidth being the width of each frame in your texture and imgWidth being the total width of the texture.
The texture coordinate for the y-axis (height or v value) should be:
frameCount * (frameHeight / imgHeight)
with frameHeight being the height of each frame in your texture and imgHeight being the total height of the texture (in this case they are probably the same since each frame texture has same height as the entire texture here - or that's what I'm assuming by looking at your code).
If you want the code to be more efficient, you can precompute the multiplications that happen multiple times for each quad. So you can probably precompute:
float widthFraction = frameWidth / imgWidth;
float heightFraction = frameHeight / imgHeight;
The same applies for the vertex coordinate calculations, by the way.
Over hundreds of thousands of vertices, this will definitely speed the computations up a bit, but you should compare the two methods to see how much.

Resources